Bias in BJCP judging?

[denny]

@mdixon

Again, you are flagging your experience with two people at a table as what every BJCP judge does.

Using scientific descriptors on a score sheet is poor form IMO. If the entrant doesn’t relate to what you write it just isn’t useful information. Writing down “Concord Grape Ester” is far more useful in the long run “ethyl heptanoate.”

I’d suggest you take the online entrance exam and then the tasting exam. Also spend more time at various competitions so you have more than a singular experience.

+1.060 x2 to Dixon!

[klickitat_jim]

Typically I learn less if the teacher’s goal is to show me how smart he or she is. Sometimes I just tune them out all together.

[system]

@mdixon

Using scientific descriptors on a score sheet is poor form IMO. If the entrant doesn’t relate to what you write it just isn’t useful information. Writing down “Concord Grape Ester” is far more useful in the long run “ethyl heptanoate.”

In this instance, the problem was not that the judges could not name the chemical compound.  It’s that they were too quick to judge the fruity aroma as acetaldehyde, which would have resulted in a lower score.

[mdixon]

With most judges a fruity ester would never be confused as acetaldehyde however people have different aroma and taste experiences and often perceive things at different levels. So you may pick up a phenolic and believe it is at a very high level while the judge across from you does not pick up anything. The sign of an experienced judge is one who knows at what level they pick up a particular trait and if they are hypersensitive know how to relate that to the average judge. That comes with time and experience.

[hopfenundmalz]

@mdixon

With most judges a fruity ester would never be confused as acetaldehyde however people have different aroma and taste experiences and often perceive things at different levels. So you may pick up a phenolic and believe it is at a very high level while the judge across from you does not pick up anything. The sign of an experienced judge is one who knows at what level they pick up a particular trait and if they are hypersensitive know how to relate that to the average judge. That comes with time and experience.

Then there are those that say Bud has acetaldehyde. There are a lot of those.

[mtnrockhopper (Jimmy K)]

This seems to be spiraling to nowhere. Judges and judging are not perfect. It never will be, just look at controversies in judged competitions with far more concrete, measurable metrics.

[klickitat_jim]

Amen. I’m on your side with this. Brand new baby judge who is just glad to be a part of it. I can only fix me, so I plan to pay attention and learn and improve along the way. Meanwhile not getting to bent about broad stroke criticism of the whole package

[darwin18]

Did you communicate your concerns of possible biases to the competition coordinator, head judge, or did you just come onto an internet forum and post your complaints?  If you saw definite bias then it really is your responsibility to communicate to those who are in charge of and run the competition so that they can address it and ensure that it doesn’t happen in future competitions.

FWIW, I would be much more concerned with the judge that had a bias against keg conditioned beers than a lack of flavor/aroma knowledge.  The first is a conscious effort while the second is just a lack of knowledge that it easily addressed.  Next time just call the head judge over and ask him his thoughts and see if he agrees.  Not every judge is an expert in every category.  In my opinion, you missed a really good opportunity to help share your knowledge of the styles with the people you were judging with.

It’s not rare either for the entries for one subcategory to score poorly against the entries in another when they’re grouped into one flight.  Honestly, sometimes you just a category or subcategory where the entries are just not good.  Have you ever seen a competition results page where a 2nd and 3rd place for a flight but not a 1st?  That’s an example of poor quality in that particular flight.

[johnf]

From what I witnessed today, I can honestly say that it is difficult for BCJP judges remain unbiased through an entire flight.  The lead judge has a huge amount of control on the final score of a beer, and his/her biases do impact a beer’s final score.  For example, a lead judge with whom I worked in one flight definitely had a bias against beers that were bottled from a soda keg.  I witnessed another table score every beer in 8C lower than every beer in 9C (both styles were grouped into the same flight).  That situation is statistically impossible without bias.

How many 8Cs were there and how many 9Cs were there. Probably not many if the categories were combined. I think you are taking some pretty egregious liberties with the word “impossible”.

I do agree there are a lot false positives on acetaldehyde. I do not think this is at all limited to the BJCP. Budweiser is the perfect example of this. It does not have above threshold acetaldehyde and yet most people who know the word acetaldehyde think it does because the word is overassociated with apple aroma.

The observation about relative scoring of 8C and 9C beers strikes me as typical lack of intuition about probabilities combined with hyperbole and best and data mining at worse. If you go into a situation with the intention of observing something that validates your beliefs, you probably will.

[hopfenundmalz]

@johnf

From what I witnessed today, I can honestly say that it is difficult for BCJP judges remain unbiased through an entire flight.  The lead judge has a huge amount of control on the final score of a beer, and his/her biases do impact a beer’s final score.  For example, a lead judge with whom I worked in one flight definitely had a bias against beers that were bottled from a soda keg.  I witnessed another table score every beer in 8C lower than every beer in 9C (both styles were grouped into the same flight).  That situation is statistically impossible without bias.

How many 8Cs were there and how many 9Cs were there. Probably not many if the categories were combined. I think you are taking some pretty egregious liberties with the word “impossible”.

I do agree there are a lot false positives on acetaldehyde. I do not think this is at all limited to the BJCP. Budweiser is the perfect example of this. It does not have above threshold acetaldehyde and yet most people who know the word acetaldehyde think it does because the word is overassociated with apple aroma.

The observation about relative scoring of 8C and 9C beers strikes me as typical lack of intuition about probabilities combined with hyperbole and best and data mining at worse. If you go into a situation with the intention of observing something that validates your beliefs, you probably will.

It is often stated as fact that Bud has acetaldehyde in its aroma. As you state, it has levels below threshold.
Mitch Steele and others have stated that the Bud yeast has high levels of apple esters, which causes confusion. Go to the paragraph with the Acetaldehyde heading.
http://hoptripper.com/what-is-quality/

[reverseapachemaster]

The most obvious difference between apple-like esters and acetaldehyde is I get a very tin-like metallic taste with acetaldehyde, especially at higher levels. Bud definitely does not have that.

[system]

I decided to take Alewyfe’s suggestion to heart.  It’s definitely easier to throw stones than it is step in and attempt to make a difference.  I took and passed the BJCP Entrance Exam today.  Now, I just need to pass the BJCP Beer Judging Exam within the next year.

[hopfenundmalz]

I decided to take Alewyfe’s suggestion to heart.  It’s definitely easier to throw stones than it is step in and attempt to make a difference.  I took and passed the BJCP Entrance Exam today.  Now, I just need to pass the BJCP Beer Judging Exam within the next year.

Excellent. +1

[morticaixavier]

I decided to take Alewyfe’s suggestion to heart.  It’s definitely easier to throw stones than it is step in and attempt to make a difference.  I took and passed the BJCP Entrance Exam today.  Now, I just need to pass the BJCP Beer Judging Exam within the next year.

way to go. I’m looking forward to my prep class and test later this year. good luck!

[klickitat_jim]

Good job.

[ranchovillabrew]

Great job!

[leejoreilly]

I decided to take Alewyfe’s suggestion to heart.  It’s definitely easier to throw stones than it is step in and attempt to make a difference.  I took and passed the BJCP Entrance Exam today.  Now, I just need to pass the BJCP Beer Judging Exam within the next year.

Excellent! “Better to light a single candle…”

[AmandaK]

@hopfenundmalz

I decided to take Alewyfe’s suggestion to heart.  It’s definitely easier to throw stones than it is step in and attempt to make a difference.  I took and passed the BJCP Entrance Exam today.  Now, I just need to pass the BJCP Beer Judging Exam within the next year.

Excellent. +1

Woooot!!! 

[duboman]

Way to go!

[jonhughey_att.net (HoosierBrew)]

Nice job !