If you're talking cardiovascular fitness, a couple cigars a week will make close to zero difference (providing you aren't inhaling). Smoke a cigar every day, and the volume of smoke that will enter your lungs passively will make a difference. Wether you feel a noticeable difference is down to baseline fitness levels, exercise modality, and your breathing technique.
If you're talking weights it will only have a difference if your intensity is high, where CV fitness becomes a factor. Rest periods may also need to increase between sets, though there are so many variables that affect weightlifti