I did a little detective work and found an entry that fits your description in the Tampa region of NHC. It was judged by a mead certified Grand Master 3 and a certified mead judge. As an organizer and judge director I can’t assign it much better than that.
2 Likes
That ain’t too shabby, but let’s not discount another fact: mead is not cider; cider is not mead.
I’m not saying anyone did anything right or wrong. I’m not saying BJCP is awesome or terrible. I’m definitely not blaming AHA for one iota. Just stating a fact for additional consideration.
Of the three competitions in which this cider was submitted. One received a double gold with scores of 41 and 43. Regionals 32 and 39. State Fair received a gold score of 38 and 39. If you run these numbers through any online descriptive statistical calculator, you will discover that I contend that the score of 32 is an outlier. An outlier is calculated as:
Upper Fence=𝑄3+1.5×𝐼𝑄𝑅
Lower Fence=𝑄1−1.5×𝐼𝑄𝑅
Q3=41 and Q1=38. IQR=3.
Based on the judge’s comments, scoring it 32, and their judge ranking, it seems that something is wrong.
1 Like
I use the statistical method as well. So yes. Throw out the outlier. The average or dare I say median of the scores that remain is closer to the truth.
Just use the median instead of the average and you’re already mitigating the impact of outliers.
The point is that judges can destroy your entry for attaining its proper score, and there is no attempt to statistically weed out judges within the Regionals for being “outliers” with their scoring. How many other entries did this judge also improperly score entries? Regionals come once a year, and to get such a score that is so sloppily determined diminishes everyone’s results from being so far off. The AHA needs to take the time to judge the judges through statistical analysis, as to how they scored for across the board. For every score they have, there is another score by another judge. The difference between all those scores should show a pattern. I can only analyze my entry across multiple competitions, but the AHA can look at the scores within a competition.
Yeah… sorry man. No competition is going to do that. Ever. Not a comp, not an organizer, not the BJCP, and certainly not the AHA. Sorry.
1 Like
It sounds like you had a bad experience at regionals. Anyone that enters competitions has had similar things happen to them. I’ve had judges put in writing my Crem Ale wasn’t creamy enough, maybe they meant the mouthfeel was too light, all you can do is move on to the next competition and hope for better results. Honestly this wonderful hobby has peaked and is on the way down for a number of reasons. Until we have judges lined out the door at a competition you get what you get all we can hope for is to not get a pro brewer and a cicerone judging together. Keep brewing, keep entering comps and remember a 33% medal rate on your entries is pretty good.
1 Like
But the AHA could give all the numbers to an undergraduate in a math program in a local college/university with a couple cases of beer and that student could crunch the numbers. They could probably build a simple system to identify the judges that are not operating up to spec to better understand if there are issues.
And then do what?
Take away a certification they have no jurisdiction in?
Not allow judges to judge?
Make them increase/decrease their scores?
And what position at “the AHA” does this?
Is this a volunteer position?
Paid position?
What are the requirements?
But the bottom line is, a statistical analysis on subjective scoring is meaningless
1 Like
Not saying that at all, but if the judges don’t understand how well they are doing - how do they know how good or bad they are doing? The judges need to also be scored. They need to correct their flaws.
I wish the moderators would lock this thread.
2 Likes
We could, but nothing about this conversation violates the rules around here and I think it would be better for folks to be vent and potentially come to solutions (within reason - no violence or being an overt jerk). Shutting off all dissent, even when the majority of us feel it’s unwarranted feels a bit too far.
As I said way back up there, somewhere, I get the frustration that Ryeff is feeling and those of us who’ve been around this block a few times know the system isn’t perfect (too many human fingers in the making of).
Let’s take this out of the realm of taste and look at professional reffing. Those folks get grades and statistical analysis out the wazoo performed on them and yet Angel Hernandez and Joe West remained professional umps for decades.
3 Likes
6 Likes
One aspect of the program that the AHA uses for its competition (Beer Awards Program) is that an entrant can use it to send feedback on the judges and the score sheets. These comments and ratings go to the organizer of the regional competition who can use that information to look into a judge’s performance and either talk to the judge about it or use the information in future scheduling.
2 Likes