.jpg)
Perception influences reality to a huge extent. If a slice of the same yellow cake is colored green, testers will report detecting a vague mint flavor. Similarly, if it is brown, they will sweat there is chocolate present. While our sense of sight can deceive our taste buds, so to can our preconceived notions regarding a given product. Is Weller Full Proof better than Maker's Mark Cask strength? Do you have a friend that has really hyped up everything Buffalo Trace? If you have a favorable opinion or nostalgia for either brand, it could influence your ranking beyond that of the actual tasting experience. Blind tasting, simply where the taster does not know for sure what is in the glass, helps us mitigate for these external factors to better empirically assess a product without confirmation bias or suggestion bias (placebo).
Background on the Practice of Blind Tasting
Blind tastings have been used throughout history, though the popularity of the practice in Beer, Wine, & Spirits can be largely traced back to Judgement of Paris in 1976. The wine competition, organized in Paris to commemorate the US Bicentennial, pitted the best of French and California wines against one another in a set of two blind tastings. Though the contest was judged by a panel of French oenophiles in addition to the two organizers (one American and one Brit), there was no home-field advantage. It shocked the world when wines from Napa County won both the white and the red tasting (both from Stag's Leap Vineyard). Previously, French wine was commonly held to be the best in the world, but this contest paved the way for expanded investment in New World wines, tripling the number of California wineries over the next decade. In customary bombastic fashion, the French media completely ignored the event. You can bet that if the judges had known which wine was which the results would have been far different!
Since then, blind tasting have become a staple across gastronomy and many industry certification exams include a blind tasting component. Most literature tends to focus on wine and coffee, as those industry bodies have more thoroughly pursued rank judging, but the practice continues to grow both in the field and at home. As you think about rating whiskies, it is often both prudent and fun to blind various bottles against one another. The results may surprise you and will only serve to improve the relative integrity of your ratings and value of your purchases. To prove the point, the Journal of Wine Economics published a large study finding that "individuals who are unaware of the price do not, on average, derive more enjoyment from more expensive wine. In fact, unless they are experts, they
enjoy more expensive wines slightly less," a finding not possible without the extensive use of blind tasting techniques.
.png "Paris Tasting")
Types of Blind Tastings
There are two main types of blind tastings, driven by different goals:
- Ordinal Tasting - In an ordinal tasting, as you may assume, the goal is to rank or order all of the participating products by quality. This is the most common format used in competitions and by consumers at home and is intuitive to understand.
- Difference/Discrimination Tasting - Instead of saying that one thing is better than another, there are times when you may wish to determine whether or not two things are in fact the same. A difference test, much like the null hypothesis of a t-test in statistics, exists to evaluate whether or not there is a perceptible difference in the quality of two or more products. This type of test is commonly used to evaluate changes in recipes for foods. If a brewery is evaluating using a cheaper barley variety versus their current more expensive one, they may brew two batches, keeping all factors other than the barley the same. When tested, if the difference tasting panel is unable to tell the difference, then the brewer can safely move forward with the cheaper option, having saved money while not perceptibly harming the product. Difference tasting therefore typically arises when making optimization decisions.
On the subject of statistics, all blind tastings are subject to the same sample size rules as other experiments. For a finding to be significant, you'll likely need to have multiple tasters or rounds of tasting to be conclusive. The proper method for calculating significance will depend on the tasting method, but the main point is to say that a test-of-one proves nothing. Invite some friends!
Methods of Blind Tasting
Numerous methods for conducting blind tastings have emerged over the years, so here are some the most common. Note that a tasting approach may combine multiple of the below techniques.
Single Blind Test
Description: In a single blind test, tasters know that they are tasting different samples but do not know the identify of the samples. The proctor may know which samples are which. Samples are typically scored or alternatively may be ranked if the number of samples is small. An average of scores or rankings of each sample will be taken across all tasters.
Type: Test Design - potentially applicable to all methods
Double Blind Test
Description: Neither the participants nor the person administering the test know the identity of the samples until after the tasting is completed. This method is used to control for both participant bias and potential unconscious cues from the test proctor. Typically this will be accomplished by using some sort of semi-random obfuscation or by having a third party arrange the samples.
Type: Test Design - potentially applicable to all methods
Sequential Monadic Test
Description: Samples are presented one at a time in sequence with tasters evaluating each sample independently before moving to the next. This is the most simple test and what you would likely do without prompting. Ratings can be simple using a variety of scoring systems. Rating items using the numbers 1-4 is common. This type of tasting scales the best, so if you have more than five samples I would highly recommend rating them individually and then doing a ranking in aggregate.
Type: Ordinal Tasting
Ranking Test
Description: Multiple samples are given, and the taster ranks them from best to worst or according to a specific criteria (e.g. from most to least aromatic). I recommend small groups of samples, limited to between 3-5, for this exercise to keep things manageable. Average the rank across tasters to determine the winner. Personally, I like to build a set of similar whiskies and then rank them.
Type: Ordinal Tasting
Bracket Tasting
Description: After samples are blind labeled, setup a bracket of 1 versus 1 tastings, where the taster selects their favorite from each group of two and tastes the winners sequentially until only one remains. This is complicated by bracket seeding and requires a minimum of eight samples. Ideally brackets will be seeded randomly for multiple tasters so that the average placement is more reflective of reality. High level of effort.
Type: Ordinal Tasting
Triangle Test
Description: In a triangle test, groups of three samples are presented to a taster where two of the samples are the same and the taster's goal is to pick the odd one out. Commonly two groups will be presented where group A has two of one item while the other group has the third as the odd one out. This "Double Triangle Test" is done to increase the power of the test. If a taster correctly identifies the odd one out in both groups, it lends additional credence to disproving the null hypothesis. It also makes it easier to prepare samples as an individual as three of each option are presented. Simply switch on sample of each group and employ a randomization approach.
Type: Difference Test
Paired Comparison Test
Description: Given a list of attributes (sweetness, bitterness, smoothness, thickness), the taster is presented with two samples. They will chose one or the other as having more of a given attribute. These may then be scored via a scorecard into rankings or to build comparative profiles between products.
Type: Difference Test
Duo - Trio Test
Description: Akin to a triangle test, a Duo-Trio Test involves presenting a reference sample first, followed by two test samples. The taster tries to identify which test sample matches the reference. If the error rate approaches 50%, then it can be assumed that the difference is imperceptible. This type of exercise is better used as a training mechanism in my opinion, but is great for building your confidence in feeling out differences between whiskies. This is a great place to start before tying to do full paired triangle test, but is difficult to do individually since you must pour a different number of samples of each product.
Type: Difference Test
.png "Blind Tasting Man")
Conducting Your Blind Tasting
The tasting experience begins with a goal: what do you want to learn? After that, you can choose and combine different methods to best answer that question while removing bias. Many of the same preparation and hosting steps apply as discussed in our guide to hosting whiskey tastings. You'll need to make decisions on each of the following areas.
1. Selecting Products for Tasting
In a blind tasting, groups of products should be relatively similar, otherwise there is not much purpose to doing any of the blind elements as the taster can easily determine which is which. If I was doing a tasting of an IPA, a Lager, and a Stout, it would be trivially easy to know which was which. This is why most major competitions are done in categories of similarly styled products or with multiple representatives of each style present in the group. Unless you are doing Sequential Monadic testing, I recommend keeping groups of products to five or less (thankfully the Triangle and Duo-Trio test are 1:1 so that saves us some complexity). It is very difficult to wrap the brain around many more items than that at a time and palate fatigue is a real issue. Evan with a monadic format, you'll want to take ample time between each tasting, more than a minute and ideally around five. Why rush the experience?
Examples:
🔴Bad - Redemption Rye, Maker's Mark, and Jack Daniel's - Very Different flavor profiles
🟡Better - Maker's Mark, Larceny, Buffalo Trace, and Wild Turkey - Multiple styles present
🟡Okay - Maker's Mark, Larceny Barrel Proof , Weller Special Reserve, Bernheim Wheat - All are wheat bourbons but one is much higher proof.
✅Good- Woodford Reserve Double Oaked, Jim Beam Holiday 2024, Elijah Craig Toasted Barrel - all are finished bourbons with similar proof
If you're worried about building a tasting from scratch, bonded bourbons make great tastings since they are all the same proof and most are around the same age.
2. Blind Method
In practice, you seldom really need to chose between single and double blind tests. If you have a friend or partner who doesn't know much about whiskey, it is easiest to just have them pour and mark the samples. For truly large tastings or real competitions, have a third party label it all and be done with it. You can buy pre-made blind tasting kits online, some of which include the liquid already labeled with a cipher for decoding later.
Marking methods
You will need some way of keeping track of which product is which for the final scoring and validation. To do this while maintaining the blind, you will need a way to record the information and then hide it during the tasting. Here are a few easy methods:
- Paper/Tape/Sticky Note - This can be done by folding over a post-it note or folding over and taping a piece of paper. Once the samples are marked, you should not be able to see the labeling. After randomization, Identify the outside-facing part of each sample as "A", "B", "C" etc. for tracking notes/ratings during the tasting.
- Coasters - Depending on your randomization method, you may be able to record the name of each product on the underside of a coaster. Flip the coaster so the real name is hidden, randomize the samples and give them a new identifier.
Randomization Methods
Alternatively, there are a few individual randomization methods that work well as double blinds. These can be done for larger quantities so that you only have to randomize the products once. If you want to randomize tasting order across tasters, you can roll a dice to determine the placement for each person.
- The "Box" - Pour Samples into small vials (you can buy a set of 20 on Amazon very cheaply or maybe you already have some lying around from Flaviar tasting kits). Mark the vials to keep track of what is in each. Put all the samples in a box and gently shake it around or mix the vials without looking. Draw the vials back out one at a time and mark each with a letter to keep track of your scoring/rating. If you are doing a larger tasting, I've done up to full water bottle sized containers in this manner in order to pour out 20ish samples.
- Lazy Susan - Arrange samples in a somewhat symmetric shape, close your eyes, and give the Lazy Susan a good spin. Keep spinning it for around 30 seconds. Label samples with a letter for tracking.
- Shell Game - Place each sample container underneath a cup. Add one additional cup with an empty/water sample in it. Rotate the cups for around a minute until you can't remember the order. If you have trouble not following the cup, try doing it while watching TV. Label with a letter for tracking. If you have a lot of samples individually, you can do this with whole boxes. Since they're sealed, I sometimes just flip a few boxes every time I walk by for a few days until they're good and scrambled.
3. The Tasting
At this point, you should have some randomized samples with labels. If you're doing the tasting solo, I wouldn't bother with opaque glassware as you'll be able to see the samples while pouring. On the other hand, if you have a facilitator and the cash, opaque glassware can be a fun investment to remove any visual bias. Arrange the samples for tasting and you're off to the races. A few things to keep in mind:
- Give yourself time between samples - 1 to 5 minutes to cleanse the palate and avoid fatigue.
- Water, Water, Water - Cleanse your palate with water and a lightly salted cracker to reset
- Don't Feel like you need to finish it all - Servings should be of limited volume, perhaps around .75oz. If they are larger, don't feel pressured e to drink everything if you already have a good handle on your notes/rating. You can always polish them off after rating the other samples, and muddling your senses increases unnecessary variance.
Record and discuss your results with your fellow tasters. Apply any statistical significance calculations as you see fit.
.png "Whiskey Tasting Science")
Conclusion
Blind tasting is a hugely valuable tool when it comes to better understanding a product. At Castle & Cairn, we will typically conduct at least one comparative tasting, often blind, before releasing a rating. Beyond adding a bit of science and procedure, blind tasting is fun! Check out this example blind tasting comparison of Kirkland Bottled-in-Bond and Wolcott Bottled-in-Bond, two white label bourbons from Barton 1792. Cheers.