I’m preparing to take the BJCP judge certification exam and in doing so I’m reading the BJCP Exam Study Guide, on page 14 it states “when the other judges have finished scoring the beer, discuss the techincal and stylistic merits of the beer, and arrive at a consensus score. Be prepared to adjust your score to make them fall within 7 points of the other judges at your table”.?
I don’t understand why a judge would change the original score to make it fit within the other judges’ scores, within 7 points? ???
I think it’s because judging beer is very subjective. And in the case “group think” may well be better than individual opinions. Hopefully others will respond. I’m just kind of guessing.
I think your right, judging is subjective, or maybe one should be objective and subjective when judging. I don’t know why that is that’s why I’m asking.
HA, Borg :D… There is no “I” individuality when judging in a BJCP sanctioned event (I don’t know), It appears to be a “We think approach” according to the guidelines, and so when a BJCP certified judge is evaluating a beer it represents the BJCP certification program. I think.
The other way to think about it is, if the scores are too far off then the entrant is less likely to pay attention to what the judges are actually saying in the comments.
I see what you’re saying beechum, this practice is in place so that the brewer can better receive the judge’s comments. However, in my inexperienced opinion this appears to be a method of manipulation, of fixing and adjusting score sheets so that the score sheets and comments fit nicely in accordance with the rest of the judge’s comments, just does not seem right to me, it seems non-ethical. Maybe one judge has a better sense of smell or taste than another judge, therefore, missing or tasting something more apparent than the other judge, but even then I still would say let it be (because if one judge has to point some flavor or smell out then it probably was not that apparent in the first place). I’m new to this so I’m just trying to understand.
Say you got one score sheet with a 29, the high end of “good” and one score sheet with 40, pretty much in the “excellent” range. This would mean that the two judges could not come to a consensus on the overall quality of the entry, which would be confusing to the brewer.
After the notes are written by the judges, there needs to be some discussion to come to an agreement on this point and then the scores, if needed, are made to conform with this consensus. Sometimes one judge is able to convince the other to raise his score to match the other and sometimes they meet toward the middle or come down to within five points. If there is an impasse, one of the judges may call over a more experienced judge to get a third opinion.
I hope this helps.
I see it as a way of establishing a “grading curve” so that (as noted above) the scores are meaningful to the brewer. I would also think it ensures that any judging session would have some kind of internal consistency and logic not only for a particular beer, but across beers as they are judged.
I agree. There tends to be a “regression to the mean” between two vastly disparate scores. A lot of it depends on the perceived senority/beer cred of one judge relative to the other. For example, a recognized BJCP judge with more brewing experience and lots of medals in previous comps might still be able to sway a certified BJCP judge with less experience. A lot of it is dependent on the personality of the judge. Some judges, frankly, are rudely dismissive of opinions other than their own, rather than being open-minded and engaged in educational discussion with their fellow judge(s). I don’t find it fun to judge with them.
I’ve been on both sides of the situation (i.e., judging beers, and having my beers judged by others). Yes, it is rather subjective, although following the guidelines seems to help inject some objectivity to the evaluation. The problem is that not everyone can detect or appreciate every aroma, flavor, flaw mentioned, nor give the same relative weight when judging. To be fair, the judges don’t have your recipe sheet nor were they with you during the brew process. We’re going off of your entry #101 being labelled as a “Pale Ale” or whatever. We’re strictly going off the BJCP guidelines and our senses.
I also occasionally encounter those who just won’t give a score higher than 41 or 42, even if it is really good, because they perceive that a score higher than that just doesn’t get awarded, and, if it does, then it pretty much guarantees that the beer they’ve awarded a score of 41+ is going to win the comp. This is more of an issue during intra-club comps with just one round, and some judges may have entered a beer as well–in other words, they have “skin in the game.” Even though they aren’t judging their own entry, they’re still judging in that style–something that wouldn’t happen in a real BJCP competition.
I sympathize with you; it is frustrating to have beers judged by those with little experience and/or no BJCP certification saying giving a lower than expected score and very little in the way of constructive feedback, and then getting a vastly different score at a different competition (along with a medal). I’d suggest having your beers judged outside the homebrew club at one or two competitions by those who have no “skin in the game” before you make any conclusions about your beer. JMO.
If there didn’t need to be a consensus score and a set gap, judges wouldn’t discuss the beer at all. If a table has the exact same score for a beer, 9 times out of 10, they simply go on to the next. I worry about those because they may agree on a score but may be giving conflicting advice.
What I don’t like is when the consensus score is adjusted because of a BoS result. I’ve had 38 point beers changed to mid 40s when it was sincerely a 38 point beer because a beer at another table in queued judging was seemingly scored too high. I mean, it’s nice to have a 44 point score on the cover sheet, but when there are minor flaws and it gets that, there’s a problem.
Someone else stated it, the judges need to come to consensus and IMO that means 3 (THREE) points, not seven. At times one judge may be overly harsh and another overly lenient and the two need to come together. Generally by the second beer the kinks are worked out.
I agree to NOT change scores in a mini-BOS round. So what if the winner with a 36 bests a beer the other group gave a 42. What that says to me is the 42 was overly generous and the 36 might have been a little heavy handed, but in the end the best beer to advance was advanced. Scores don’t matter in the final BOS round!
I’d suggest you submit this question to the email forum so you can grab the attention of Gordon Stong and David Houseman…High level judges that run the program will give you much better insight.
Somehow I knew you were going to ask that! For the life of my I can’t find the link, doesn’t even look like they advertise it on the homebrewing site anymore… I’m sure someone else will have it handy, but I keep my email pretty clean and I don’t have any
emails from them right now, if no one comes up with it, I’ll respond on Monday.
Not everyone has a perfect palate. I am very sensitive to diacetyl and fusel alcohols. Not so sensitive to oxidation or acetaldehyde. So if another judge tastes those, I can adjust my scoresheet accordingly.
I am a Recognized BJCP judge, and I for one ABHOR the idea of being virtually forced to change a score based on an arbitrary requirement to be within 7 or 5 or 3 points of another judge, who may have a lot more or a lot less experience – doesn’t matter. Everyone should be entitled to their own opinion, unadulterated, then let the receiver of the scoresheets decide for himself/herself which judge is right on and which he/she does not agree with. Better yet – get a third and even fourth opinion if there is a broad difference of opinion that cannot be resolved.
I feel it is most wise to score each beer on your own without discussion, then spend a few minutes comparing notes to see if there is something you missed or to see if maybe you scored a little too low or too high. That much is reasonable. But also, don’t let anyone convince you that you scored incorrectly and that “surely you can taste that diacetyl” or DMS or whatever, even if you know in your heart that you can’t pick it up in the slightest. Be honest! Don’t let anyone tell you that YOUR score is wrong! You own it! You sign your name to it – no one else does! Any changes must be because YOU realized that you want to change your score, NOT because of some arbitrary range rule. To be coerced into changing a score because others tell you that their opinion matters more than your own constitutes LYING in my book if you do change that score.
I believe this very firmly. I am a man of integrity. And like most judges, I am well educated on beer styles and have tasted more than my share. This is a case where sometimes judges just need to agree to disagree.
For the life of my I can’t find the link, doesn’t even look like they advertise it on the homebrewing site anymore… I’m sure someone else will have it handy, but I keep my email pretty clean and I don’t have any