Please Help - Specialty Style Data Collection Project

I’ve started a project to track what sort of beers are entered into the Belgian Specialty (16E) and Specialty (23A) categories in competition.

I intend to collect data from BJCP-sanctioned competitions for a set amount of time (4-12 months, depending on response rate and volume of data), try to sort the entries by “style” (e.g., Black IPA, Hoppy hefeweizen, etc.) and present the information to the BJCP Style Guidelines Committee for consideration when they next revise the guidelines. Currently, the BJCP doesn’t track this information, so it’s hard to figure out what sort of “demand” there is for new style categories.

I’m hoping to get a large data sample (~10k entries?) from competitions held across the U.S. and Canada, so that I can pick up regional variations. Once I get the data sorted, I’ll release my results, along with the raw data so that the real statisticians can sort through it. I’m anticipating a timeline of at least 6 months, with preliminary results first appearing in August.

Here’s how you can help.

  1. Contact the competition organizer of you local HB competition and have them send me results of past competitions. I’d especially welcome data from big competitions like the Bluebonnet Brew-Off, Dixie Cup, Upper Mississippi Mashout, Indiana State Fair, etc.

  2. If you are an organizer, registrar, or other competition official, please send me data.

Send me a private message, I’ll send you an email link.

Here’s the data I need:

  1. Name of the competition

  2. Date it was held.

  3. City/Region where the contest was held (e.g., Houston, TX, Peoria, IL).

  4. ANONYMIZED special ingredients and base style information for all beers entered in categories 16E and 23A. That is, no names, addresses or scores, just info like “Belgian Tripel made with Cascade hops,” “Midas Touch clone” or “Black IPA.” Obviously, the more complete this information is, the more useful it will be.

  5. Total number of entries in the competition. (So I can get a percentage of specialty beers vs. total entries.)

  6. Data can be in any electronic form, although I’ll probably end up using Excel.

  7. Other info that you think might be helpful. The more info the better.

  8. Helpful comments and suggestions.

Nice idea.  Thoughts:

You should be able to dig up a lot of data just by visiting all the competitions’ websites.  You’ll at least get the top 3 entries for each style, and you can usually go back for a few years to see what might have been winning then as well, IF you’re interested in past data.  I mean, why not – I wouldn’t limit new style possibilities to fads that are only popular in 2011 but where the interest will fade in a couple of years.  We need to find the styles with several years’ stay-time, IMHO.

One previously undefined style that sticks out in my mind way more than the Black IPA fad is the “Hoppy Orange Lager”, a la Sam Adams Boston Lager, Lakefront Riverwest Stein Beer… okay maybe those are the only two examples with similarity, but in my mind, it should be stylized as those beers are pretty dang popular but don’t fit into any other style.  These are sort of like Vienna lagers but no Vienna malt, more crystal, and more hoppy.  Speaking of which…

Another idea is to separate American Oktoberfest from German-style Oktoberfest.  The Americans don’t do it right, and tend to make something more like… Hoppy Orange Lagers!  But I digress…

The only problem with competition websites is that they’re sometimes hard to find and they usually don’t give total number of entries, total entries in a particular category, or special ingredients for the beers. It’s easier to contact the webmaster or competition organizer and beg for data.

I agree that data from past competitions would be valuable. I’m trying to get as much as I can.

I can think of all sorts of styles which “ought to be” in the BJCP Guidelines, but I’m trying to figure out what “styles” people are actually entering in competition, since that’s what the BJCP Guidelines are for.

If there was demand for it, I’d be perfectly happy to see the BJCP Style Committee come up with guidelines for “American Amber Lager.” There are a number of Americanized lager styles out there; either survivors of historical American lager brewing traditions (e.g., American-style Oktoberfests and Bocks) or creations of the craft beer scene (e.g., Imperial Pilsner). Sadly, that’s not what homebrewers appear to be brewing. My initial data indicates lots of interest in APA and AIPA variants - black IPA, rye or wheat IPA and APA (i.e., too hoppy to be an American Wheat or Rye), Belgian-style IPA.

This is an excellent idea.

You might consider setting up a google spreadsheet with form input so you have less data entry to do in the end.  After you collect it all you could export to excel or whatever

This is a good idea. I will look into it.

Have you checked with Janis to see if she has the data from nationals in an easy form?  She may even have the last few years.

Also, if you’re looking for regional variations I think you should include the location of the brewer, at least the state.  Competition location doesn’t necessarily tell you the region it was made.

I’ve pinged a few of the local comp organizers to see if they have the data.

She was one of the first people I contacted. I’ve gotten an informal commitment from her to provide data from this year’s 1st round competitions, as well as previous years, but only after this years’ competition winds down in late June.

You’ve got a really good point, but it would be difficult to get such data without either getting personal data or asking my contributors to do a whole lot of editing, neither of which I want. I’m very sensitive to the fact that I’m basically spamming people in order to get my data. I want to make it absolutely clear that I’m not some sort of cybercrook doing some kind of phishing attack and also make it easy for people to give me data.

I’m just going to have to trust that most competitions attract entries from the local area and work harder to get data from smaller competitions which aren’t as well publicized and which don’t get as many “out of town” entries.

I try to enter an American Lager as often as possible just for this reason, so hopefully it will get a subcategory.  Having an entire category named ‘European Amber Lager’ and then only having two substyles makes that category seem incomplete.  Get rid of the ‘European’ moniker and at least add ‘American Amber Lager’ (with Boston Lager and Brooklyn Lager to name two examples) as a substyle.  IMO. :slight_smile:

It depends on how the data are stored.  If there is a separate column for the state then it is easy to include.  If the address is all in one column then yeah, way too much editing to ask of anyone.

So how many entries have you gotten so far?  Are they just rolling in?  One of the organizers I contacted said he was already planning to send you the data this week, so there’s that.  Thanks for taking this project on, it will be interesting to see the results.

After much angst and several attacks of Life, I have preliminary data on this project.

My apologies for the long delay in getting results out.

I received data from 27 competitions (23 unique competitions), scattered across the U.S. and Canada, which were mostly held in Spring or early Summer of 2011, although some folks gave me competition info for competitions as far back as 2008.

Competition location seems to be reasonably representative of BJCP contest activity as a whole, although I didn’t get any data from competitions held in the Southwest (e.g., Texas or Arizona) or Southern California.

Since I don’t have the competition schedule from the period under review, I can’t say what percentage of contest organizers responded, nor whether my data samples are truly representative. (The BJCP Competition schedule has a No Robots policy, so cached versions of the page aren’t available on Archive.org or Google.)

Number of entries ranged from ~100 entries on the low end, to ~800 entries on the high end.

Total number of entries is unknown because some contest organizers failed to give me total number of entries, but the total number is at least 6,000

There were a total of 276 entries in the Belgian Specialty (16E) and 587 entries in Specialty Beer Category, for a total of 864 items.

Within the sample, up to 123 entries were mis-categorized (i.e., base style defined and ingredients listed which would throw the beer in to categories 20-22 rather than 16E or 23A), about 16% of the total. This leaves about 741 entries which truly fall into Specialty or Belgian Specialty.

Within the properly sorted entries, identifiable new styles are as follows:

American Sour 34* 4.59%
Belgian Artisanal Amber Ale 5 0.67%
Belgian Artisanal Dark Ale 9 1%
Belgian IPA 28 3.78%
Belgian Singel 4 0.54%
Belgian Sour 4* 0.54%
Biere de Noel 6 0.6%
Black IPA/Strong Black IPA 104 14.04%
Brown IPA/Strong Brown IPA 22 2.97%
Dark Saison 7 0.94%
Gose 5 0.67%
Heather/Gruit Beer 9 1.21%
Orval Clone 10 1.35%
Rye Beer, Other 8 1.08%
Rye IPA 32 4.32%
Sticke Alt 3 0.40%
Strong Amber Ale 11 1.48%
Strong Brown Ale 4 0.54%
Strong Hefeweizen 1 0.13%
Strong Pilsner 8 1.08%
Strong Robust Porter 7 0.94%
Strong Saison/Strong Dark Saison 20 2.70%
Strong Witbier 4 0.54%
Wheat Wine/Strong Wheat Beer 10 1.35%

*This number could be much higher because many brett or sour beers declare a base style and/or other ingredients, so I haven’t been able to pull out the exact number yet because of the way my data is formatted.

** This number could be much higher because many rye beers declare a base style and/or other ingredients.

Slicing things up a bit more broadly, it appears that:

IPA variants (including Strong Saison) respresent about 27% of all Belgian Specialty and Specialty beers.
Strong variants of existing styles account for about 10% of all Belgian Specialty and Specialty beers.
Sour beers (including Brett beers) account for at least 5% of all Belgian Specialty and Specialty beers, and possibly more.
Rye beers account for at least 5% of all Belgian Specialty and Specialty beers, and possibly more.

Suggestions for the BCJCP Style Committee

Based on preliminary number crunching, it seems logical to add more IPA sub-styles to the BJCP guidelines. At the very least Black IPA should be included as a new sub-style, but thought should be given to at least discussing Rye IPA, Brown/Red IPA (AKA Texas Brown Ale) and Belgian IPA as other IPA variants. Arguably, these new styles should be put in a new category called “Specialty IPA” so they don’t compete directly with the already popular IPA category.

A weaker case could be made for adding an “Open Strong Ale” category which would subsume strong variants of other beer styles. This could be a sub-style of Category 23.

A weak case could be made for adding an “Open Sour Ale” category which would cover all manner of brett beers, sour beers and other wild fermented creations which don’t fall into Category 17 or other categories. This could also be a new sub-style within Category 23.

A weak case could also be made be made for adding an “Unusual Grains” category (again possibly to Category 23) which would cover beers which are out of style due to use of grains such as rye, buckwheat, oats or wheat.

Finally, while it isn’t listed in the stats, it might also make sense to have an “Unusual Sugars” category (possibly within Category 23) which would cover beer which are out of style due to use of unusual sugars such as honey, molasses, agave or lactose. I have sorted beers listing these ingredients yet, but there seem to be a number of them.

Status of the Project

At least on the surface, the data seems to be clearer and more complete than I’d hoped. Still, I’d hoped to have the project run longer and capture more data (especially data for the NHC first round competitions, which was promised to me, but never delivered). In retrospect, it was also a mistake to not ask for ALL entry data. Had I done so, I could have compared percentage of entries in any particular sub-style to the overall total, and I could have also looked at trends within categories 20-22 and 24-29.

I’ve been archiving the competition calendar since January 1st with the intention of restarting the project. The fact that I’m a National judge now might give me a itty tiny bit more influence in getting competition organizers to cough up the data. Still, I’d love it if I could work with a group, preferably led by a GM judge. It would also be great if I got official sanction from the Powers-that-Be within the BJCP organization to continue or expand the project.

In addition to working with a group, it would also be helpful if I could get archived versions of the BJCP competition web page, and if my request for data went out as part of the official information when competition organizers register their competitions. At the very least, it will keep my ISP from thinking I’m a spammer!

Raw data is available as an Excel spreadsheet. Send me a PM if you’re interested.

Nice data so far.  A few ramblings and opinions from a Recognized rank judge and certifiable beer nerd (snob?) (me):

I’m intrigued and delighted about the proposed American Sour style, although I don’t know how you would define it.  What sorts of entries would fit into that style?  Basically any Brett or other sour beer that doesn’t fit into the other 17A to 17F subcategories?  Yeah, that’s probably a good idea.  Pretty broad style though.  And then Orval and similar might fit into the same subcategory.

I’d love to see Gruit Ale/Beer get the respect it deserves, as I recently entered one and debated whether to enter it as 17B, 21A, or 23.  Probably would have scored fine in any of the three, but I didn’t know which was most appropriate so I put it in 23.  If there had been a 21C for Gruit Beer then it would have been a no-brainer.  I have a feeling we’ll be seeing more gruits in the next ~5 years as people become more curious, and more tired of hops (is it possible!?).

I don’t know if Rye IPA or Rye Anything deserves any further recognition.  I think what needs to happen is the American IPA description should be expanded to allow a lot of rye in the recipe.  This would accommodate the rash of Rye IPAs.  Then any other rye beers go to existing American Rye or Roggenbier or Specialty 23.

Most of the Strong Ales you have listed should either be considered Barleywines (already have categories for that) or else should remain as Specialty 23.

And I’m not sure exactly how far you’ll be taking this, but I feel I must beg you and/or the BJCP to please take this opportunity to kill the term “Black IPA” right here and right now, and rename it instead to American Black Ale.  “Black IPA” is a contradiction in terms to refer to something as both black and pale at the same time, and India has nothing to do with it whatsoever anymore.  If people like the style, fine, but call it something that makes an iota of logical sense.  I would also recommend taking very great care to compare and contrast the new style description versus existing styles Robust Porter and American Stout.  The lines between these three styles are very blurry and there is opportunity for a lot of overlap.  Assuming that these are indeed three different styles, I might propose even going so far as to put all three under one new main category called American Dark Ale, and then consider combining all the other existing styles of porters and stouts under something broader such as English Dark Ale.

Sorry to see my Hoppy Orange Lager category didn’t make the cut.  Trust me on that American Oktoberfest thing, though… there’s just no fair comparison between the Americans and Germans.

Things for you and/or the BJCP to think about when it comes time.  8)

Dave: I agree on having a American Dark Ale category. The only thing I hate more than a Black IPA is a Cascadian Dark Ale. I don’t understand why Black IPA would need its own category when the robust porter category allows pronounced hop character, and the American stout category also allows “bolder roasted malt flavors and hopping than other traditional stouts.”

One of the beers listed on the BJCP as an example of a Robust Porter is the Avery New World Porter, which on the Avery website is described as a Black IPA. So, huh?

Seriously though, if someone can explain why a Black IPA wouldn’t fit into either of those two categories (either as those categories currently exist, or as they could be slightly tweaked), I’d appreciate it.

CDA doesn’t fit into any porter or stout category because it lacks the roasted character you find in those beers.  They are not chocolatey or burnt and have no coffee aroma.  Those are some of the signature flavors and aromas in porters and stouts, how can you have one without it?  I also think they are lighter bodied in general.

During my data collection project I tried to stay as neutral as possible. While I did have to make some editorial decisions as to which “style” a specialty beer fell into, I tried to use the brewers’ own descriptions as much as possible.

For the “American IPA + a bit of dark malt” “style” most brewers just called it Black IPA or Dark IPA, while a few (especially from the Northwest) called it Cascadian Dark Ale. A smaller number just described the ingredients or described a base style (American IPA or Double IPA) and noted that debittered black malt had been added. One or two tried to describe what the beer wasn’t - i.e., “lacking the roast character and body of a Russian Imperial Stout.”

So, while “Black IPA” is an oxymoronic name, it’s what most brewers were entering it as. If it makes you feel any better, the Brewers Association style guidelines, used for the GABF competition, call the style “American Dark Ale.”

I also tried to be very cautious in my suggestions for new categories or styles and didn’t try to define what any particular proposed style should be.

My idea for an Open Sour category would cover any beer with significant lactic, acetic, brett or wild-fermented character which doesn’t fall into category 17 - Sour Ales. It would represent both Belgian “specialty sour” beers based on beers in category 17, as well as sour beers based on other styles or no style at all. I’d be reluctant to call it an “American Sour” category, since it could conceivably include traditional European beers such as vatted porter or strong Berlinerweisse.

Ultimately, the BJCP Style Committee will make the final call, based on a number of factors. My survey is just one data point.

Currently, the existing guidelines work pretty well as a teaching tool and competition framework. Serious alterations to the guidelines could have unintended consequences. I think that the least risky changes would be to add more sub-styles to categories 20-23.

Why does the AHA have a different set of beer categories than the Brewer’s Association? Aren’t they the same organization?

It’s the BJCP’s guidelines vs. the BA’s GABF guidelines. So 3 different organizations. I don’t know anyone using the GABF guidelines in the homebrewing world mostly because they’d be crazy impractical and they are very tuned to the needs of that competition and audience. (e.g. they need to respond quickly to trends in the commercial market to allow brewers to enter and be rewarded)

That’s one of the dirty little secrets of the whole guidelines business, they’re not just an exercise in taxonomical purity. Ultimately, whatever guidelines you use need to be balanced for the entry patterns you see to ease judging overload.

One of the more obnoxious trends in the Falcons is that we’ve always maintained our own guidelines. i haven’t put the newest revision online yet, but here is the last one - 2010 Falcons Style Guidelines One of the things I was happy to see is that we already cover a number of Thomas’s findings in the guidelines. (Ok, I needed to do it - so here are the 2012 Guidelines with its one change)

Belgian Brut
Belgian Blonde Ale
Belgian IPA
Belgian Quadruple
Black IPA
Belgian-Style “Wild Ales” (this is our catch-all for things like Temptation, etc)
Imperial Pilsner
Two Saison categories
Wheatwine

And like Thomas, we discovered most of these via watching our entry patterns. Years ago, we had to slide our categories around to make things make more sense competition wise - e.g. APA became it’s own category since we had a huge overload of APA’s. We moved DIPA into the Barleywine / Old Ale category to put it on even footing instead of in position to blow away regular IPAs.

We still have to split up our specialty categories better to reduce burden and to also allow for more entry configurations for the Doug King Memorial Competition, but small changes are good.

This can’t be said often enough. Much of the reason that many people hate the BJCP is because they hate the BJCP guidelines, without realizing that those guidelines are designed to provide clear categories for homebrew competitions. In some ways, they’re as artificial as breed standards for dog shows.

Do your competitions really get a sufficient density of these styles that they merit their own categories?

Some of them seem very obscure (e.g., Belgian Brut beer, Wheatwine).

I do like the idea of separating beers by their SG ranges/ABV.
You’re right about the big beers always seem to blow away the smaller beers within the same category.
A good APA or XPA never stands a chance against an AIPA that’s brewed to the high end of it’s range (or even higher, like a DIPA).

Plus, beers that emphasize hop flavors and aromas, e.g. APA or XPA, practically deserve their own class separate from beers that emphasize hop bitterness, e.g. IPA’s.

Some of them are very obscure! The Brut one is a sop to the guys in the club (aka me and Fletch) who brew champagne beers. We’ve had a few Wheatwines over the years, but here’s where I think I break off from a lot of folks in terms of philosophy:

  • subclasses, particularly small ones, are free. The only cost associated with a new subclass is a little more paper for printing the guidelines. Eventually as subclasses get bigger, then you may have to add a judge flight

The place where the rubber meets the road in terms of making more work is when you create whole classes. Then you have additional judge overhead, medals, etc… Classes cost money. :slight_smile:

Bigger beers have an inherent advantage against smaller beers in that they can have more of what you want in a beer - malt, alcohol, hops - and they also hold up better over time (mostly).

A number of guidelines have small beers and big beers (i.e., 7%+) broken out into their own categories for this reason.