From what I witnessed today, I can honestly say that it is difficult for BCJP judges remain unbiased through an entire flight. The lead judge has a huge amount of control on the final score of a beer, and his/her biases do impact a beer’s final score. For example, a lead judge with whom I worked in one flight definitely had a bias against beers that were bottled from a soda keg. I witnessed another table score every beer in 8C lower than every beer in 9C (both styles were grouped into the same flight). That situation is statistically impossible without bias.
Even trained full time professionals are subjected to their internal biases. Volunteer judges who judge a few times per year don’t stand a chance.
Although, I would disagree with your comment about the impossibility of every one of a substyle scoring lower than the other. I’ve had some flights where one substyle was absolutely mediocre at best and another where the worst beer was mediocre, so… not impossible.
The crazy thing is that I asked to taste the beers in that flight. A beer in 9C that reeked of ballpoint pen extract tang scored higher than a beer in 8C that was commercial quality. The lead judge’s justification was that the 8C beers were slightly malt forward. This experience taught me that BCJP-sanctioned competition results should be taken with a grain of salt. Anyone who has ever tasted a “real deal” English ESB on tap knows that “malt forward” is acceptable within the style.
Anytime you get one set of criteria and two humans to apply that criteria you will get 3 different opinions.
The best way to influence judging is to become one. I am a novice, but I have judged for a number of competitions and I have entered many more. You can look at your score sheets. The BJCP judges are always identified by their ID number. In many cases, it has not been the BJCP judge giving my beer the lowest score, but rather a non BJCP participant, and I know that they are not the lead. I have also strongly disagreed with the score at a table where I was judging. I was allowed to present my argument and the lead judge altered his score to bring our table scores to within the 7 point requirement.
Judging is hard work. I personally do not enjoy it. 98% of the judges I’ve worked with take the job VERY seriously, work hard and try to render helpful, fair and objective scoresheets. You may have met one of the 2%, I don’t know, but don’t condemn the process based on this one experience. Get more involved.
So one competition, a single judge out of 8,000, and you can pass judgement on the judging skills other 7,999?
you said some other stuff, but that’s what I took from your post.
No, it gets worse. The BCJP judges at my table could not tell the difference between acetaldehyde and ethyl heptanoate (the condensation of ethanol and heptanoic acid). They flagged a really nice sweet smelling beer as suffering from acetaldehyde. I said, “are you sure that you do not grape?” That’s when everyone at the table noticed it. One said,“I smell Concord grape.” I said, “Bingo! You just smelled ethyl heptanoate.” Silence fell over the room because no one had heard of that ester. Ethyl heptanaote is the ester that is used in artificial grape flavoring. Ethyl heptanoate smells like Concord grape or grape lollipop. Many heirloom British yeast strains produce this ester.
With that said, the major problem that I have with this situation is that inexperienced brewers tend to take contest remarks as gospel. Flagging ethyl heptanoate as acetaldehyde is a major faux pas that only confuses the entrant.
Sounds like you had some very insightful knowledge to share. Like I said, get involved, or stay involved. You can help make the competitions in your area better.
Maybe in an ideal world there would be four judges. Drop the high, drop the low, and average the remaining two. Will need way more judges if this were to be the norm.
Like another system we’re familiar with, its not perfect but it’s the best we’ve got. I agree with jumping in, and I’m in the process of doing so myself. Personally, not to change anything but just to help and to learn.
The best in any endeavor can enlighten the worst. I just hope that the learning is passed on as new experience are encountered. I have considered going through BJCP, but don’t think I have the time and enough of a discerning palate for the job. So thank you to the best and also thank you to the worst - may you meet and become better for the experience!
you seem to know a lot. but you didn’t already know this bolded part?
Judges do their best. None of us know everything. I struggle detecting low levels of acetaldehyde and sometimes DMS. I’m hyper sensitive to diacetyl. And I’m still a pretty decent judge.
And maybe “silence fell over the room” because they were waiting for further explanation from someone they thought they could learn something from? Or perhaps the “this is what it is, period” tone made them not want to reply?
yeah we’re gonna have to disagree on the seriousness of what happened…
IMO your comments have an “I was right and they were wrong” tone. Since you’ve also lumped all judges in together with your comments, perhaps you could offer some feedback to address the issues you believe exist. If I were a new brewer, not a judge, and took your comments “as gospel” I’d likely come to the conclusion that BJCP judges were worthless and we should blow up the judging process. So, what should be done?
cheers–
–Michael
It sounds like you are quite knowledgeable about ester identification - which is awesome. That said - I don’t think it is a MAJOR faux pas to be unable to flag ethyl heptanoate and its prominence in heirloom british ale yeasts… Christ, I have never even heard of it. I brew british beers all the time - good ones. It is certainly a great bit of knowledge to learn about - but, I think you are going to far with this. Why would everyone be expected to know this?
These people are VOLUNTEERS. This is a HOBBY. I would fully expect the people judging my beers to be learning as well. I would hope that judges would do their best, continue to learn (like in this situation) and offer as constructive a criticism as the can at their current level of knowledge.
Anyone who enters a beer here and there and expects definitive feedback off of a single entry is doing it wrong. If beginner brewers are taking 1 set of sheets as “gospel” they need to be told that they are not going to get what they are looking for. If someone really wants some in-depth feedback and troubleshooting on a beer for the purpose of modifying recipe’s/process, etc. - they better be sending that beer into 3-5 competitions for sure. In my experience, that is how you get consistent criticisms to rise to the top and rule out what might be a single, individual perception.
In my experience over the past couple years, I would say about 25% of the feedback I get is extraordinarily helpful. About 50% of it provides one or two valuable things I can take away/or combine with other feedback to make a difference in my beer. About 25% is not that helpful. That breakdown makes it all about 100% more helpful than having friends and family say “ooooohhhhh, this is good.”
Thanks to all the BJCP judges who DONATE their time - even the beginners who are still learning as they go. I appreciate it and it has made my beer a lot better. Hope to join the ranks in the next year or so.
I am not a judge but knowing several I know that it is a tough job tasting beer all day, or in some cases over several days. Some samples tremendous and some are down right nasty.
I think the judges, whether they are nationally ranked or just getting started, do their very best to fairly evaluate each and every beer, but yes, regardless of what score you get or comments you receive, take it for what it’s worth-an opinion.
In the end, if you like your beer and are proud of it then enjoy it! Take the feedback you receive and apply the parts that you think could improve the beer and forget about the comments you might not agree with. I also find that when possible, enter the same beer in several comps to get a better overall feel of the beer.
It is also important to provide feedback to the judges on their evaluations. I have emailed several judges from various comps to get a better understanding of their comments and attach their score sheets so they can remember what they wrote as well, even if they don’t recall the specific beer. For example: In my most recent comp my ESB received a 35 which I was happy with. The problem though was after reading one judges comments I found that his understanding of the style was lacking. Each thing I got dinged for is clearly stated in the guidelines as being true to style and more than acceptable. In a polite manner I emailed him a copy of the sheet and a suggestion that he re-read the style guideline so that he can improve his understanding and improve his judging knowledge. This was a “recognized” judge and his score brought my average down.
Judges are constantly learning their trade just like brewers. They take the time to volunteer but I also believe that as a brewer it is my right to provide them feedback as well, both positive and negative so they too can improve their trade.
As previously mentioned, you are dealing with the personal abilities of each judge to properly sample your beer and each person’s ability to note certain characteristics will be different. Each person’s threshold of taste will be different. I can easily detect most general flaws in a beer but when it comes to discerning the nuances of certain hops or malt profiles I fall short. In many cases I can even tell you what yeast was used in a beer while others can’t, every one has strengths and weaknesses. It is unfair to lump all judges together and call them biased.
+1 duboman. Well said.
Outside of personal bias creeping into decisions, I believe that the BJCP program does an excellent job of teaching style parameters. All of the national and most of the recognized judges with whom I interacted at the competition knew style parameters very well. In my humble opinion, the area in which the program could use some improvement is teaching the accurate identification of fermentation metabolic byproducts, especially esters. Beer after beer, any hint of apple was flagged as suffering from incomplete fermentation. However, the presence of apple does not always mean that a beer is suffering from incomplete fermentation. The type of apple matters. Green apple is acetaldehyde. Ripe red apple is ethyl hexanoate. Acetaldehyde is the result of incomplete fermentation or the oxidation of ethanol; therefore, it can be considered to be a flaw. On the other hand, ethyl hexanoate is an acceptable ester that is present in many ales to some extent. For example, the Scottish and Newcastle strain throws ethyl hexanoate at above taste threshold levels (it’s an integral component of the Newcastle Brown Ale flavor profile).
Seems like it would be super useful to take a tasting course with you leading. You obviously have a strong grasp of yeast metabolic byproducts and their sensory identification and that takes a lot if study, practice, and understanding. I for one would take that course
Again, you are flagging your experience with two people at a table as what every BJCP judge does.
Using scientific descriptors on a score sheet is poor form IMO. If the entrant doesn’t relate to what you write it just isn’t useful information. Writing down “Concord Grape Ester” is far more useful in the long run “ethyl heptanoate.”
I’d suggest you take the online entrance exam and then the tasting exam. Also spend more time at various competitions so you have more than a singular experience.
To me, at this time you are a relatively new and anonymous poster, so, why don’t you give your background? You seem awful sure of yourself. How do you know you were right and they were wrong? Are we witnessing the coming of the next Gordon Strong?
Putting the snarkiness aside, judging can be subjective. BJCP judges across the board do a great job of trying to get it right and they usually do. A head (lead) judge’s job is not to disregard or change the other’s perception of the beer but to come to a consensus with the other, often less experienced judge. There are few judges out there that can get all of the nuances to every style. I am a national BJCP judge and I have been paired with judges that have no credentials that were amazingly good and on rare occasions some that thought they knew it all, but didn’t. Most judges are aware of the shortcomings of their palates and work very hard to get it right. In spite of inherent limitations of their senses they usually get the best 3 to the medals.
Remember, homebrewing and judging is a hobby. There is no money awarded. 99% of the time it is for bragging rights and feedback. When I get back 2 different interpretations of a beer that I entered, I read them both and try to see what the difference was and compare them to my notes. I usually learn something.
Other that getting a mass spectrometer for your homebrew club, I think the system works well within its obvious limitations.
Mac
A well-trained judge can tell the difference between apple-like ester and acetaldehyde. But only a minority know the former as ethyl hexanoate, and even fewer will refer to it as such.