The Importance Of Being Same

General Homebrew Discussion
1

Probably since the beginning of brewing there has been discision regarding differences. At least I’ve noticed it since I started way back in 2012. For the sake of discussion,  spurred by something I read on the forum today, I thought it might be interesting to air out and debate ways of determining differences and how important it is or isn’t.

Let’s pick on Denny for a minute. He is a big proponent for the blind triangle test. Two of one, one of another, pick which one is different. If enough testers with decent palates can pick the odd ball, then there is a difference. If they cant pick it, there’s not a difference. Thats how I understand it anyway.

But then recently Denny posted results of his Noti Brown brewed using Mark’s starter method. He didn’t use the triangle, nor did he use any other tasters other than hisself. He said that it was his best Noti Brown ever. Judged from memory only.

Also recently, there was some debate over possible differences in an experiment Marshall did with old hops. He caught a little scrutiny flak for having two different worts. Same grain bill, water, etc… they were just two seperate batches. Seems to me those two worts were probably different, but only minimally,  and only in theory since they weren’t charted on a GC/MS. So its possible they actually were the same, though a long shot.

Anywho…
When does difference matter so much that it requires charting on a GC/MS?
When does it matter enough to do a blind triangle with BJCP GMs? Masters? Nationals? Certified? Pro brewers? Novices?
When does difference matter enough that brewer’s/taster’s memory is good enough?

I have some thoughts, but I’ll leave it to you guys for now.

2

I think if you’re telling people categorically that one way works better than another way, being able to provide some data like a triangle done with what you consider good palates is more solid than just anecdotal info from your brew days. I don’t think a set rank has to be achieved to do a triangle, just good palates. OTOH, if I’m offering up what I’ve observed in my brewing as only shared anecdotal info, then I don’t think I need to have done a triangle to share it. That’s why I at least try hard not to be categorical, because we all know there are multiple ways to do most of this stuff.

EDIT-  And even a single triangle test doesn’t necessarily categorically prove anything. It’s just a good data point to consider.

3

Data is (or “are” for you snooty grammar folks!) data.  Some data is better than others.  It is up to each individual to decide which data he or she likes and which he or she does not.  The human tendency to be biased/prejudiced dictates that we will tend to gravitate towards data that agrees with our own hypotheses, and ignore or attempt to refute that which does not.  Also, we just love to argue, for no sake other than we just love to argue.

“Differences”/discrepancies/anomalies matter when they matter to several people.  When they matter only to few or one, they don’t seem to matter.  Still, it doesn’t prevent us from arguing about it, because we so love to argue just for the fun of it.

4

A lot depends on your personality type and what sort of brewer you are. I don’t brew for competitions. My goal is to brew beer that subjectively tastes good to me in a way that is fun and takes a reasonable amount of time. If people are making starters in an easy way that works I’ll try it and see if it works for me. No triangle tests, double blind studies, blah blah blah. Someone else might find that aspect of brewing fun but brewing is just one of the things I do so…yeah.

5

I’ll be honest… when I hear about things like the exbeeriments or blind tasting, I take that information and file it away in case I ever need it.  Otherwise, I generally don’t pay much attention to it.  I say this because of all the variables that go into this type of thing.  I almost equate it to a movie reviewer… he liked it or didn’t like it but that’s not me, it’s him.  I follow what I think works the best in my experience.  I take detailed notes on each batch and when I make that recipe again I look and see what I thought should be changed or stay the same.  I mentioned in our FB group that I really like the character I get from FWHing a beer.  One of Marshall’s exbeeriments (I think…) was comparing FWH’d beers to non-FWH’d beers.  The results were that no one could tell the difference.  I find that interesting but anytime I’m drinking a beer and think it’s exceptional and then I go check the recipe, it contained a FWH.  Nothing scientific about my findings, just consistently good results as far as my tastebuds are concerned.  I should also state for the record that I REALLY like when our Super-Brewers conduct these experiments and share their findings with the rest of us.  It makes us all better brewers and shines a light into some dark corners of the brewing world.

6

good comments so far. I too appreciate the information shared by others, and evaluate if its absolute or something different that provided good results. I don’t need a triangle test (although sometimes it helps my evaluation) - just some basic information about what they did and their results. From there, I decide if it has any value for me, and if its worth  my time to give it a shot.

I ran a test on pure pitch without a starter- didn’t make any claims or statements indicating my results were based upon anything other than my past experiences and sensory observations. I made no claims that it’s the new or only way to proceed, just shared my findings and what I thought…the rest is up to others.

that’s what I love about this forum and the people on it.

7

I love the triangle tests/multiple tasters Brulosophy style! I agree that everyones tastes things differently so if something works for you (especially something as effortless as FWH) no need to change it from a random test but that being said, that sort of data does help me make decisions.

There seems to be a lot of misinformation out there that’s accumulated over the years so it’s great to see people test theories out and be able to back it up with some taste tests. In my opinion the proving or disproving or homebrew dogma is the most exciting thing happening in homebrewing at the moment.

8

I also find that subtle nuances in various beers get swept away by variables.  The taste of a single 5-gallon batch of beer changes over a short time.  Our tastebuds are different every day based on what we have eaten, what we drank yesterday or already today, whether we’re sick or on medication, etc.  If I’m making a beer that I would rate 97 out of 100, I’m probably not overly interested in making it better.  If I’m making a beer I would rate a 60 out of 100, I am tirelessly looking for an answer because I’m making beer I don’t want to drink.  Always good to bang these questions around with good brewers who are good people too… echoing Ken’s words.

9

I feel that I can’t even trust my own taster’s memory.

I can’t think of how often I’ve purchased a commercial beer that I’ve had many times in the past and been surprised by how it actually tastes on the particular day I am drinking it.

10

I agree.  I have had bad commercial beers that I know are usually better than that.  Or a draft version of a beer that is WAY better than the bottled versions I have had.  So.  Many.  Variables.

11

I think it depends on what you’re trying to test.

12

I love Marshall’s experiments. I think they’re both highly informative and entertaining but it’s also a single brew day. While he has multiple taste testers, his N still equals 1 regarding the experiment itself. BUT, he never has claimed his experiments to be the final verdict on a topic - only fodder for further discussion and exploration. That caveat is the nature of good scientific process.

In some respects I actually trust Denny’s recent experiment with the shaken not stirred starter method more than I would a single Brulosophy experiment. Why? Because Denny has brewed that particular beer A LOT of times and probably has a well-calibrated sense of how it tastes. If the only thing that has changed is his methodology in preparing the starter and he finds a significant taste difference - well, that’s a lot of data points suggesting that starter preparation method matters. I’d put up the red flag of confirmation bias in this example except Denny was so insistent that the stir plate was the way to go before committing to the experiment.

Replication and repeatability of results is something that is starting to hit home, in a big way, with a lot of pharmaceutical and biochemical research now. So much so that some high-end journals publishing this research are beginning to flesh out protocols regarding repeatability of the experiments. If we are interested in the effect of home-brew techniques on beer taste, replication and repeatability should be on the mind.

13

^^This^^

All day.

14

Jim, it simply depends on what kind of results you’re looking for.  In my case, I wasn’t trying to empirically prove anything.  It was just a trial and I posted my impressions.  I hope I made that clear in my blog post.  But due to those impressions, I intend to do a more thorough test.  At this point, I won’t try to convince anyone of the validity of the validity of my impressions.  They simply are what they are.  If I do a real test, and can get others to repeat it and verify the conclusion, that will be another matter.

Does that all make sense?

15

I think we wonder about differences from ingredients and differences from process. For the sake of simplification, I’m narrowing it to just plain difference, regardless of cause. Its important to think about wether or not the difference is important before choosing a way to determine difference.

In a criminal trial, things must be proven beyond reasonable doubt. If there is no doubt for which there is a reason, the thing is considered proven. In a civil trial, the thing must only be proven by a preponderance of evidence. A weighing out of evidence showing that more likely than not, it happened. If there is 1 reason to doubt but 2 reasons not to doubt, then you have a preponderance of evidence it happened. Whereas in the criminal trial there must be evidence it happened and there can be no reasons to doubt.

So what? Well, sometimes we hold trial on presented ingredients issues or process issues. What level of proof do we require? Is that level of proof in line with how much difference is made and how important that difference actually is? If, for a home brewing process, we require greater level of proof than is needed to put a Murderer in prison… maybe that’s going too far.

For a minute, imagine you’re trying to repeat a beer. Make it the same as the last one. Is that possible? The person who would require absolute evidence would say No. Its impossible to rebrew a pefectly identical beer. I would say Yes. As far as it is only going to be tested in my mouth and compared to my memory, yes, I can rebrew beers that are the same according to my level of proof.

So, if I am looking to make a change, will that change be discernable in my mouth as compared to my memory? And is the discernible difference desirable? Then, is the ingredient or process change worth it to obtain the difference?

That thought process shows the importance of personally experiencing changes with home brewing, rather than putting too much weight on the credibility of the claims of others. Ultimately, beer is a personal experience. It doesn’t matter that a scientific analysys proves a difference, or if some expert says so. If its not there, for you, its not there.

Case in point. I brewed a beer with US05 sprinkled dry. Then I read an expert who claimed that 50% of them die because of osmotic preasure. So I rehydrated. Then I read that liquid yeast was better so I bought Wy1056 and followed the instructions. Then I read that it wasn’t enough yeast, so I made a starter. Then i read that repitching was way better, but i didnt repitch enough, then i was repitching way too much. Then I read that my starter needed to be on a stirplate. Then I read that it needed to not be on a stirplate, but shaken instead. Most recently, I chose to hit it with O2 because I’m too lazy to shake. And pitch at high krausen, whereas I used to chill and decant because I was told that the spent wort was bad. Now unspent high krausen wort seems to be the thing.

So in that last paragraph theres a lot about what I had read or been told but nothing about what I experienced from those different methods. All the expert stuff is just academic. A good starting point if you dont know what to do. The only thing that really matters is what you think of your results with your beer. And when comparing changes, memory is all that really matters, because its awfully hard to preserve a perfect sample from last year’s beer in order to do a triangle test.

Some times some of us need to prove things to others, but all of us need to prove it to ourselves.

16

Good discussion - from legal standards of proof to philosophy about introspection and memory.  Who’d a thunk it from homebrewers?  As Denny might say: Good on ya!

17

I read it all, consider it all, and then compare it to my own experience. Every brew I do, I either incorporate what I’ve read–much of it from this forum–or heard with what I know from my own experience.

With that said, I can’t try everything, since I only brew one or two times a month, so I find the things that I read, whether it is old–I read Charlie Papazian for years–or the newest Brulosopher experiment, or the latest post I read of Denny or Martin, food for thought or possibly the impetus to change the way I do things.

I guess what I’m saying is that I have an open mind. It has served me well in the sense that I, my wife, and my friends love my beer. That and the relaxation I get from the hobby and the validation that I get from producing something that improves/enhances my immediate society reinforces many of the ideas of the homebrew community on the whole.

Being on this forum, I am a man among giants. So, I say keep it coming: triangle tests, educated opinions, empirical evidence, even anecdotal evidence. I’ll run it up the flagpole, see how it looks and maybe even try it myself. But, above all, keep it coming!

18

Hear hear

19

+2.  Well said, Frank.

20

Jim,

Have you been drinking?

I was actually thinking about this this morning.  Complete consistency in a repeat is not possible at the homebrew level.  For a 5 gallon batch, the margin of error in our measurements must be unbelieveable compared to someone brewing a 700bbl batch.  However, I’ve been brewing several of the same recipes for years, and they are as consistent from year to year as I need them to be.

So the answer is still “it depends.”  As it usually is.

Now, on to the NL Wild Card.  Pirates suck.  Go Cubs!