AHA First Round

General Homebrew Discussion
21

Good points tntjr, I probably could have phrased what I meant better.  Obviously the whole point of BJCP training is to teach people to do a better job tasting and judging beers, and one of the main goals is to give good feedback to entrants.  But there is a level of subjectivity to judging that is less than ideal.  When Jeff points out (correctly) that “You win some and you lose some - different judge different day” it makes it difficult to always take the comments of the judges to heart.  If it was more consistent that would be great, but that’s not the way it works.

For example, let’s say that a judge says your IPA isn’t bitter enough.  Do you immediately change your recipe and up the IBUs?  What if that was their 8th IPA of the flight, and the one immediately before was too bitter and thin?  What if it was the first beer and they just finished their coffee?  What if that judge just prefers an IPA with higher IBUs?  Training can help eliminate some of these problems, but you can’t eliminate order bias in our judging format.  Winning or losing one competition doesn’t mean anything, winning consistently does.  So getting feedback from one set of judges is of limited use in my opinion, but getting similar feedback from several sets is very useful.

As for those specific scoresheets, the judge mentions malt, toast, and roast.  The description doesn’t mention toast, it mentions sweetness and caramelization.  Maybe that is what the judge thought was missing. :-\

22

Bingo. People think that people who enter are just trying to win a bunch of medals. I enter a bunch of competitions with a batch so I get enough feedback that I can interpret it and make a change. The only awards displayed in our home are BoS and regional circuit type stuff. The medals/ribbons are in a pile somewhere. Fun to win but I don’t really want the medals (unless they are super awesome like the Hoppy Halloween ones).

23

Are you kidding?!  I wear my one second place medal from KCBM around every day.  People look at you kind of funny but who cares.  When they ask (and mostly they don’t) I sometimes even lie and tell them its a first place medal, because it could have been first place if not for that other stupid tasty beer.

If you’re a club member, you can always seek the opinions of the other members on your brew.  Seems like that would be a simpler way to garner more feedback, albeit possibly less trained.  If we are criticizing the level of training of the average BJCP judge anyway, then what the heck.

24

And to be clear…John is not referring to Nashville, our beer judging was completed on the 26th. Ciders held us up for a few days until I got the proper pallets to judge that style. They were done on the morning of the 30th, the DB was emailed in the afternoon to the AHA. BJCP reports has already been submitted too

25

I understand there are subjective factors to the scores and I expect that, but I shouldn’t have to guess what the judge’s criteria, subjective or objective were. Tell me exactly how you feel this beer fell short of your ideal 50 point beer.

I may not change my recipe based on one set of scores but if I start to see a trend then I’ll have to acknowledge that my interpretation of the style differs from the consensus. I can then change the recipe if I want to score high and win contests or I can keep brewing what I have been and accept that my goals for this style differ from the consensus.

Tom

26

I can see how you would read it that way, I definitely meant Saratoga.

27

One of my favorite recent judge comments was (on a Belgian Dark Strong Ale): “Though the style guideline provides for the level of sweetness you have in this beer, I prefer the drier examples of this style.”  I was blown away that he basically acknowledged that one aspect of my beer fell within the guidelines but outside his personal taste preferences,and then dinged me for it… ???

28

Wow did you email him or the comp organizer? If BJCP I would let them know

29

+1

That’s a clear violation of BJCP ethics. I suggest you cantact the organizer a.s.a.p.

30

Nah, judges are human beings and as such they will always bring their own prejudices to the table.  BJCP training is excellent, but just because someone “has” the knowledge to be impartial and set their own tastes aside, it doesn’t mean they will.  His was a glaring example because he was dumb enough to write that down, but this sort of stuff is endemic and just comes with the territory unfortunately.

31

I don’t think there is much wrong with the judge’s comment and that goes to the heart of what I feel is difficult to express on the judging forms…the beer is good and meets the style guidelines, but I don’t really prefer it.  Its that subjectivity that is hard to put into words.   I’ve judged plenty of beers that didn’t do ANYTHING wrong, but that just didn’t have the nuances, perceptions, and enjoyment that would put the beer on top.  In some ways, the Best of Show judging is a perfect case in point.  Its not that the beer in front of you is well brewed and to style, it also has to have those special touches.  Unfortunately, I can’t define exactly what those touches are, but I know them when presented with a flight of beers that are compared directly.  (this harks back to the line: ‘can you define pornography…no, but I know it when I see it’)

32

I agree completely with what mabrungard is saying when it comes down to picking a best of show beer or even which of the top 3 in a flight will win the flight.  What bothers me is this:  I brew a lot of medium to strong Belgian ales (7%-12% abv) that are bottle conditioned and meant to survive and improve during cellaring.  For the beer in question I received a score of 42 in a competition one month, and 6 weeks later (an insignificant amount of time for a beer like this) I get what I described in my earlier post and a score of 27.  Wow.

33

I can sympathize with your frustration from my own experiences entering competitions.  Sometimes you get a pair of judges that just don’t give you a fair shake.  It’s the most irritating when they don’t seem to have put in any effort.

I’ve judged in 5 or 6 competitions and recently took the exam.  Do you have any judging experience?  If not, I would just like to add to your contemplation of these divergent scores that factors like fatigue, the quality of the beers surrounding yours in the flight, and judge preferences can play a major role.  I don’t think there’s really any such thing as purely objective evaluation.

What I’ve seen is that the majority of beers that are well made and basically fit the style score in the low-mid 30’s, say 31-35.  That area is kind of the generic “good beer zone”.  It might get you a 3rd in a good size competition.  Trying to get out of that range is a major hump in competition.  I agree that your scores are more divergent than you’d expect them to be for that kind of beer in that time frame.  If the judges didn’t list a notable technical or stylistic flaw, I don’t think you should have scored below 30.  However, a 42 is a very good score, about as high as anyone ever gets in reality, and could reflect the preferences of the judges for a beer like the one you entered or the number of poor entries in your flight.  If you had gotten a 42 and then a 33 I would not have been too suprised.  Leaving the scores aside, did the judges who gave you the 42 demostrate a higher level of interest and ability in their comments?  If so, you’re probably closer to that score in reality.  Entering it again somewhere else will give you a clearer picture.

As a new judge, I really try to focus on writing quality comments and talking with the other judge/s to make sure that the beer ends up in the right general score range of fair, good, very good, and excellent listed on the BJCP score sheet.  It’s just so easy to lose the forest for the trees when scoring.

34

That might not have anything to do with your beer, it could be bottle to bottle or judge to judge variation.  I’ve had the same beer score 45 and 24 in two different comps.  It happens.  Like we talked about in the other thread, there is order bias as well.  The smells in the room.  Too many variables to control really, but the judges are doing their best.

Assuming no off flavors you shouldn’t get a 27 for it being a little too sweet, especially if it is otherwise within the guidelines.  I think the comment is ok in the Overall section, because that is where the big picture comes in.  A beer might fit the guidelines but still be lacking, and as Martin points out, it can be hard to spell out exactly what is missing.  For a 27 you should be able to spell it out though.

Then again, sometimes I’ll write down comments and not actually take points off for something.  I’m just writing down what I think of the beer.  If I say it is within the guidelines but state a preference, that doesn’t mean I deducted anything, I’m just saying what I think would make it better.  For whatever that’s worth.

35

A few years back, I was judging in a competition with a judge and I said essentially this exact same thing… now granted, I’m not in the BJCP, but I’ve entered/judged my fair of competitions and traveled to a few beer destinations… I said that in my mind, all things being equal in regards to the general gist of a beer’s compliance with stylistic guidelines, that drinkability earned a few points when I evaluated a beer… boy did he crucify me… not only that, he wrote a very verbose letter to the organizer and a few BJCP officers informing them that I wasn’t judging according to the guidelines and how dare I "actually say to him something like “this is about what I would enjoy drinking, [not about strict adherence to the artificial guidelines]”.

A couple things happened as a result of that… I lost pretty much all interest in actually taking the BJCP exam, and I lost alot of my motivation to judge at that particular competition.

36

It’s kind of hard to reconcile that attitude with the Beer Scoresheet, specifically the Overall Impression section.

“Comment on overall drinking pleasure associated with entry, give suggestions for improvement”

It seems pretty clear to me.  Pleasure is subjective, and it lets the judge capture the intangibles in the scoring.

37

Maybe they really enjoyed drinking beers that exactly line up with the guidelines… “mmm… 20-28 IBUs never tasted so good.”

38

I try to always offer my personal opinion in the overall impression section, it is about the only section that allows you some freedom to speak your mind and offer some feedback.  My comments in the other sections are purely descriptive of the beer’s characteristics (or lack of characteristics), and how they relate to the style guidelines, and then score accordingly.

There are a lot of beers that are pretty much brewed to style and fit the guidelines, but may not be outstanding or noteworthy in my opinion, yet a different judge under different circumstances may think otherwise.  There are a litany of factors that can influence this.  I know it may break someone’s heart to hear this, but that is the way it goes.  Enter multiple comps, and continue entering those comps where you have been satisified with the quality and detail of the scoresheets.

I would be a lot happier if my beer scored a 27 and had a ton of feedback, vs a score of 42 and poor feedback.  We just finished up the Canadian Region, and I have already been contacted by several entrants who were very happy with their scoresheets, even those that did not win, as they found our judges’ feedback very helpful.

39

Sadly, that’s the toughest sort of feedback to give.  Any judge will do a decent job of detecting obvious faults, the better judges are good at picking up subtle problems and minor style faults. It’s a rare judge that’s both a good enough taster and a  good enough brewer to give you ideas on how to make a good beer great.

If it makes you feel any better, your beer didn’t “lose” 14 points. It probably lost more like 9 points. That’s because most judges mentally reserve scores of 45-50 for hypothetical “perfect” beers. I’ve described these top level beers as:

41-45 - “Angels helped to brew this beer.”
45-49 - “You hear angels singing when you drink this beer.”
50 - "Choirs of angels sing, the clouds open, rays of heavenly light beam down, the hand of God points down and a booming celestial voice says, “I’LL HAVE THAT ONE!”

Whether it’s fair or not, many judges don’t want to get too far away from that magic 7 point range needed for consensus judging, so they hedge their bets a bit. On their own, a judge might say, “that’s a 39-40 point beer”, but they might not want to go out on a limb to defend their decision to the other judge, so they score the beer a little closer to the middle of the “Very Good” range at 36. If your beer was at the beginning of the flight, the judges might also want to be a bit cautious with their scoring to leave a bit more room at the top of the range for a better beer later in the flight.

Judges scores are also biased by other factors - rebound effect/halo effect from a particularly good/bad beer tasted just previously, upward/downward bias in scores over the course of the flight (possibly influenced by the fact that the beers later in the flight have more time to warm), judges’ like/dislike of the style, judges’ familiarity with the style and time of day (Really! Tasters go for maltier beers when they’re hungry). I also think that season and temperature in the judging hall plays a big role.

Finally, while it’s grossly unfair, I think that there’s something of a bias against certain styles of beer. It’s easier to detect all the qualities that make a great Barleywine, American IPA or Belgian Strong great because they’re bigger, more emphatic, showier beers. By contrast, it’s harder to detect greatness in more balanced, lightly-flavored “table beers” like Irish Red or Belgian Pale Ale.

My guess is that, at the level you’re at, you’ve got no process or technique faults. You’re using the right ingredients and tools and you know your equipment well. What you need is more experience with subtle tweaks and recipe formulations. In those respects, you’re probably ahead of me in brewing skill.

When I judge a beer at the upper end of “very good” range (35-39 score), I’m saying it’s comparable to most commercial craft beer. What I’d want at that level to make it even better is:

  1. Big complex aroma - if appropriate for the style. Even for styles where big aroma is a fault, I still want something interesting going on there. It should be complex, interesting and inviting.

  2. Aroma carries into flavor. Basically, the aroma is the promise of what you get when you taste the beer. It sucks when your expectations are let down by flavor which doesn’t match the aroma to some degree.

  3. A flavor profile that “peaks”. All foods have flavor profile with a beginning, a middle, a finish and an aftertaste. A really great food has a profile where the beginning quickly “bursts” into a massive, complex unified middle, then fades away pleasantly in the finish and aftertaste. I think of it as being like a firework - shell goes up, BOOM! lots of light and sound, sparks and squiggles of light shower down.

If I could create recipes that do this on a regular basis, I’d be brewing beer by the hectoliter and getting paid for it, so I’m not the guy to ask how to do it. All I can say is that it takes a LOT of recipe tweaking. You’re wanting your ingredients to stand out, but also to work together, and you’re wanting the flavors from all those ingredients to all peak at once.

  1. Drinkability. Is this a beer I could drink all night, or would I choose something else? Are there little, subtle problems that turn me off? A bit stale? A bit solventy? A bit to harsh in the finish? Again, more recipe tweaking and lots of frustrating equipment and process tweaks.
40

I should have said that you should rebrew beers like a German wheat that does not stay at its peak for too long.

Many others styles are not so time sensitive.  I have 3 going on.  Only one I would consider rebrewing, and it is holding up well.  One of my friends with 3 national medals always says - “It is a crapshoot”.  We had one that we really liked, and was not scored well.  Whatever.