Does Halfwheel ever have anything nice to say about CCs


Recommended Posts

  • Replies 169
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Hi Everyone, Charlie from halfwheel here. Thanks to Kevin for pointing out the thread to me via email. It's certainly an interesting conversation and a pretty educated one about the reviewing app

This thread has been quite enlightening, and I appreciate everyone's thoughts. Charlie has done a great job in detailing everything on his end, but here are a few things from my perspective for t

The luxury of HW is the sheer amount of reviews they do allows for a pretty good trend analysis.  Only 11 (regular production/LCDH/ER/EL/Anejados) CCs produced in the last 10 years have reached 9

Consider this…Last year, there was only one Cuban on their top 25 list. The year before, there were no Cubans on the list at all. Zero. Nothing for 2019, yet there were two non-Cuban Cohibas on the list. Sit on that for a moment. 

 

Furthermore, the most recent reviews they’ve done of Cubans have been terribly inaccurate. 

 

Here’s my take-away: they penalize Cubans because they don’t taste like the Nicaraguans they’re mostly used to. Just like some wine drinkers will penalize a Burgundy for not tasting like a California Cab. 

 

It has nothing to do with advertising dollars or anything like that. It’s simply a question of inexperienced palates not understanding what they’re smoking. They’d probably deny it, but they’re Cuban rookies who smoke an EL or RE every once in a while. That’s all.

 

  • Like 1
  • Confused 2
Link to comment
Share on other sites

12 minutes ago, charlieminato said:

Some other tidbits:

  • @NSXCIGAR did some great work analyzing our numbers but I think his decision to filter out the high-end stuff is flawed, particularly given that for the first few years of our website, when we were disproportionately reviewing the higher-end Cuban and non-Cuban cigars. Do I think that would reverse what the numbers say? No. 
  • One thing that I find interesting whenever people focus on the scores is that most of our readers probably have no idea what our scores are supposed to mean. For reference, an 88-point score is a cigar we'd recommend buying a box of 20 of based on its performance. A 91 at halfwheel is a really tough score to get, it's not the same as a 91 elsewhere, including probably on your own scoring system. 

I think Aaron's article at DP on when a 90 isn't a 90 shows Halfwheel is one of the most consistent scoring sites out there and hasn't suffered from score creep. 

We don't chat often enough (or maybe it's too often for you!) but the summary commentary on reviews as of late seems to be that the scoring system could use a tweak. Maybe I'm reading between the lines too much. 

Link to comment
Share on other sites

@charlieminato thanks for your thoughtful response. I enjoy your reviews and the hard work you put into it.
 

There are some very knowledgeable guys in here and I love to see the back-n-forth. Didn’t someone say they were making popcorn? 

  • Like 1
Link to comment
Share on other sites

41 minutes ago, charlieminato said:

When we are able to find ways to get our hands on Reservas/Gran Reservas, Cuba has a strong shot of making that list. If we are just going to evaluate all the other new items when they are fresh, well, you guys know how that goes. One thing I really struggle with when I hear the complaints about not giving the Cuban cigars enough time (in years) is that we just don't know how long that's going to be and there's no easy way to predict that. One other thing, we aren't trying to be wine reviewers telling you how good we think the cigar is going to be in six years. It's how good is the cigar when we were smoking it.

  • Someone mentioned vendors might be to blame. We buy from a wide range of vendors, sometimes stores and sometimes people, including some that have commented on this thread. At the end of the day, we are trying to replicate the real world scenarios of smoking these cigars and not trying to find the unicorn boxes that were hand-selected and most consumers will never see. (That's why we stopped reviewing festival cigars, which tend to be very different from the cigars that come out.)

 

Thank you for the detailed explanation - I do agree with your comments, good job.

  • Like 1
Link to comment
Share on other sites

4 minutes ago, NSXCIGAR said:

Absolute NC scores run the gamut as well and I don't see a statistical trend in terms of absolute score or average score among all NCs. I do however see a trend in the probability and clustering of scores for NCs relative to CCs. Again, it appears the probability a NC is going to score 88-93 is higher with sample distribution taken into account. In other words, the average NC is more likely to score in the 88-93 range than the average CC. 

I think you would find the work Aaron did breaking down the score for the year as well as the year over year view interesting. While it isn't broken out into CC vs NC (our group doesn't think that way for reviews) I think it goes a long way to giving you some of the analysis you're after.

https://developingpalates.com/editorials/cigar-editorials/cigar-editorial-90-not-90-3-years-later/

Link to comment
Share on other sites

1 hour ago, Carrie Nation said:

They’d probably deny it, but they’re Cuban rookies who smoke an EL or RE every once in a while. That’s all.

This simply isn't true, sorry.

 

1 hour ago, Bri Fi said:

Didn’t someone say they were making popcorn? 

Ha! 

  • Like 1
Link to comment
Share on other sites

17 minutes ago, NSXCIGAR said:

True vintage or ultra-premium Cuban production just isn't a fair representation of CCs in general.

 

This is where our data gets a bit problematic. There's basically four sets of data in here:

  • Pre-halfwheel Smoking Stogie reviews (which is mostly just bonkers stuff and Tatuaje/Illusione/Viaje LEs with some random "normal" things mixed in for inexplicable reasons.)
  • Pre-halfwheel TheCigarFeed reviews (lots of lanceros)
  • halfwheel Scoring V1 (high focus on limited edition and prerelease NC cigars, less focus on CCs in general, mixture of new/old CCs/NCs)
  • halfwheel Scoring V2 (mostly new cigars outside of CC ELs/ERs/ETC)

I'm not sure that I could argue that our reviews are a fair representation of the non-Cuban cigar market in general. It's a decent—but skewed—representation of new cigars in a given year, but it's not indicative of what one would expect to see in any store. (For example, there are almost no reviews from mainstays like Ashton and Padrón, few reviews or regular production Cuban cigars, lots of random ERs, etc...)

Quote

What I'm having trouble reconciling is that it appears much more likely that a NC is going to occupy the higher ranges of scores (88-93) than a CC.

I get that the numbers can conclude that a Cuban cigar only has a 5 percent chance of getting 91+ with your filters. I'm not sure I'm surprised, quite frankly, I'm guessing that if we gave the same exact cigars and scoring method to four people that commented in that thread, I'm guessing the numbers would be pretty similar unless there are members here that want to argue that fresh Cuban cigars are as enjoyable today than the fresh non-Cuban cigars we often review.

One other thing I didn't mention earlier, we aren't trying to make the numbers look nice/funny/special/etc. Even without "fudging the numbers" there's very easy ways to manipulate it. Some publications will only put their highest scores in the magazines, meaning you never really see the lower scores. Other reviewers will openly admit that they won't publish scores below X.

We try to just be honest and don't really care about the ramification of a high/low score in regards to our audience, advertisers, professional relationships, Developing Palate's analysis, etc.

--

I appreciate the time you've been spent thinking about this. I agree with about 99 percent of the conclusions because the numbers are the numbers. 

  • Like 2
Link to comment
Share on other sites

Where is the OP?
Farted and left the room it seems...

Sent by spooky action at a distance

  • Haha 2
Link to comment
Share on other sites

50 minutes ago, charlieminato said:

I get that the numbers can conclude that a Cuban cigar only has a 5 percent chance of getting 91+ with your filters.

I wouldn't interpret that in that way. Yes, only 5% of CCs score 91 but I don't think that's a pertinent comparative analysis to NCs. I The likelihood of an NC scoring 91 is probably about the same but as I describe below would be slightly higher than a CC.

I would put much more stock in the simple conclusion that it appears more likely a given NC will score 88-93 than a given CC. That's where I think the bias shows itself. Again, I don't know what the bias is or why it's there but it does appear more likely a given NC will score in that range than a CC. In other words, if 5% of CCs score 91, 5 + % of NCs would score 91. 

 

1 hour ago, Cigar Surgeon said:

I think you would find the work Aaron did breaking down the score for the year as well as the year over year view interesting. While it isn't broken out into CC vs NC (our group doesn't think that way for reviews) I think it goes a long way to giving you some of the analysis you're after.

https://developingpalates.com/editorials/cigar-editorials/cigar-editorial-90-not-90-3-years-later/

That's focusing on the scores themselves, not relative score numbers like I am. Absolute scores don't matter to me, nor do scores between publications. What I'm looking at is how that score is arrived at and comparing scores between categories (CC vs NC in this case) for one publication (HW) which takes how they arrived at the scores out of the equation. The score is the score and I'm looking at that. The methodology comes in later if any biases are detected.

I've isolated a category which I've labelled the "golden" range of HW scores of 88-93. It appears less likely that any given CC will score in that range than any given NC. That to me confirms a bias. There's no other explanation. There could be many reasons for a bias--I'm not drawing any conclusions but the fact remains that it is less likely a CC will score 88-93 than a NC.

We could go further and look at country by country, age of cigar and all sorts of things to isolate the bias. For example, it might turn out the bias disappears when age of the cigar is adjusted for. There you have it--perhaps the CCs have less age or more age and that accounts for the comparative difference. We just don't know. The easiest, most bird's eye look shows there is a correlation between CC and NC and scores in the 88-93 range. There's other data between the two that could account for the differences but that would require a full data input and deeper statistical dive than I have time for.

  • Like 1
Link to comment
Share on other sites

54 minutes ago, NSXCIGAR said:

That's focusing on the scores themselves, not relative score numbers like I am. Absolute scores don't matter to me, nor do scores between publications. What I'm looking at is how that score is arrived at and comparing scores between categories (CC vs NC in this case) for one publication (HW) which takes how they arrived at the scores out of the equation. The score is the score and I'm looking at that. The methodology comes in later if any biases are detected.

I've isolated a category which I've labelled the "golden" range of HW scores of 88-93. It appears less likely that any given CC will score in that range than any given NC. That to me confirms a bias. There's no other explanation. There could be many reasons for a bias--I'm not drawing any conclusions but the fact remains that it is less likely a CC will score 88-93 than a NC.

We could go further and look at country by country, age of cigar and all sorts of things to isolate the bias. For example, it might turn out the bias disappears when age of the cigar is adjusted for. There you have it--perhaps the CCs have less age or more age and that accounts for the comparative difference. We just don't know. The easiest, most bird's eye look shows there is a correlation between CC and NC and scores in the 88-93 range. There's other data between the two that could account for the differences but that would require a full data input and deeper statistical dive than I have time for.

It was less about publications and more about showing the bell chart for HW specifically, as well as how the trend changes year over year.

If you lump all scores, all time together to make some sort of analysis you're ignoring the impact that a particular years releases might have on scoring.

Playing Devil's advocate: What if 2017 happened to be a bad release year for CCs and HW's scoring reflected that trend?

When we do our own internal analysis at the end of the year we break out:

  • Average scoring by reviewer
  • Number of cigars reviewed by country
  • Average score by reviewer, by country
  • Number of cigars by price range
  • Average score by reviewer, by price range
  • Number of cigars by sponsor / non-sponsor
  • Average score by reviewer, by sponsor / non-sponsor
  • Number of cigars by sample provided / purchased
  • Average score by reviewer, by provided / purchased

Then we post the whole thing online.

Charlie does a great job of posting their data breakdown (each year I believe) and this is probably the information you're after in terms of relative scoring by country:

https://halfwheel.com/2020-reviews-in-review/385913/

 

  • Like 2
Link to comment
Share on other sites

41 minutes ago, charlieminato said:

There are a number of other explanations, starting with that Cuban cigars could be worse.

I agree, that could certainly explain it! But as all reviews are subjective reviews, at least HW specifically finds CCs less likely to score well, and that's what I'm trying to get to the bottom of. We also have the fact that the highest-scorers have been CCs, albeit super-premiums or rare vintage.

As we both point out, the bias may be due to extraneous factors and further data has to be compared. Are the CCs being smoked before being acclimated, what are the storage conditions, etc. I'd certainly give HW the benefit of the doubt in terms of objectivity but the fact remains there is an imbalance in score performance between NCs and CCs. A deeper dive into the data would possibly even uncover a bias between NC countries like Nicaragua vs. DR. But NC vs CC is a very broad category analysis and I don't think you'd find much support (subjective opinions, of course) that NCs are in general more likely to be better cigars than CCs--at least not in the last 10 years. Being generous I would imagine them being equal in terms of overall consumer evaluation. I'm discounting overall sales since we know how that shakes out and I'm trying to be as generous to NCs as I can. 

My point is that while it's possible that the answer is just "CCs are worse" I personally don't think that's a plausible explanation. It is possible that they're worse to the HW reviewers, but that's the heart of this entire post. If that's true we need to take that into consideration. Why the bias is occurring is for HW to find out. Knowing there is one just means we as readers and consumers have to take that into consideration. 

  • Like 2
Link to comment
Share on other sites

55 minutes ago, Cigar Surgeon said:

If you lump all scores, all time together to make some sort of analysis you're ignoring the impact that a particular years releases might have on scoring.

Playing Devil's advocate: What if 2017 happened to be a bad release year for CCs and HW's scoring reflected that trend?

My sample size of 10 years, regular production should be large enough to negate those kinds of anomalies. With a smaller sample size I would agree but HW's data sets are large enough to be able to smooth out those kinds of trends. NCs have bad runs, too. 

And again, I don't care about year-over-year trends. I'm looking at the history of all the HW reviews and the comparative performances of cigars over that time. I suppose the 10-year period is a 10-year "trend" but I'm selecting a period that is big enough to serve as an effective total history.

Link to comment
Share on other sites

Thanks everyone for the thorough discussion.I learned a lot. I am not as seasoned as many of you here and with that comes some naivety but maybe on he flip side an outsiders fresh perspective. So I really appreciate HalfWheel.  I enjoy Charlie’s and the others writing style. I am procigar so I like to keep up on the news which they do a great job with and I have bought much cigar paraphernalia and NCs after their reviews. I appreciate Charlie’s response and detailed thoughts...So what I have garnered is that

my suspicions are probably true. Stats can be played with but NSXCIGAR presented

some interesting numbers to think about. There is some bias going on that can be speculated but all the confounding variables cannot be determined. Although for example I respect HWs integrity , one has to be careful about reviewing a product one obtains revenue from.I don’t believe anything is a conscience bias though but it’s still a flaw.I totally understand HWs need to compare apples to apples . Charlie’s point of when do you wait 1 month, 4 months , 2 years for a CC to be ready to review is accurate but we all know for the most part you do need to wait some period. Although I’ve smoked some 2020s ROTT most are better with age. Comparing a CC and NC at say 30days may not be apples to apples.

I also don’t love assigning points but if they are published we are going to look at them. I think some sites/publications think we all want them. I come from a wine background and lived through the misrepresentation that a point system produces vs just a tasting note....anyway I will continue to read HW for non Cuban insight and just listen to the FOH forum and my other CC avenues for my CC needs. Thanks all

 

 

 

 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.

Community Software by Invision Power Services, Inc.