Music Generates Taste of Sour, Bitter, Sweet and Salty

Can you taste music? According to some scientists and musicians, the answer is “Yes.” Not only they have found reliable associations between taste perception and music but they have also developed an algorithm that can “flavor” your favorite song! Want a salty version of “With you or without you”? Or how about a bitter version of “Rolling in the deep”?

The research project, led by Mariano Sigman (physicist), Bruno Mesz (pianist and mathematician) and Marcos Trevisan, has the objective of investigating the relation between music and other aspects of semantics and sensory perception. In this first phase, they investigated a narrow and self-contained domain of semantics and sensation: taste words.

“We found that taste words were mapped to consistent directions in musical space” says Mariano Sigman. Bitter is low-pitched and legato (continuum of notes), salty is staccato (sharply separated notes), sour is high pitched, dissonant and fast and sweet is consonant, slow and soft.

Below you can play the different “flavored” versions of the song "Les sons et les parfums tournent dans l'air du soir" by Debussy (Préludes, Book I) and experience this by yourself! We will soon allow our Facebook followers to submit their favorite song to be "flavorized" by the algorithm into four versions representing each of the four taste sensations.

Unflavored Original

Sour: high pitch, short duration, high dissonance

Bitter: low pitch, low articulation (legato)

Sweet: long duration, low dissonance, low articulation, and soft (low loudness)

Salty: short duration and high articulation (staccato)

The group is currently working on a “Concert of Music and Taste” that will be staged by the end of 2011 in Buenos Aires, Argentina, with the contribution of molecular gastronomy Chef Alejandro Digilio. Upon entrance, each spectator will receive a tray with canapés prepared following notions of molecular gastronomy on such a way that the perception of taste will be partially camouflaged. A series of short piano pieces composed with the algorithm described in this article will be played in a classic piano. Musical pieces and canapés will be in sequential correspondence.

“It will be a concert of two ‘voices’ - one will be auditive and the other will be taste - that could be in correspondence or not during different moments of the performance.” says Mariano Sigman. “At some point you could be eating and listening to something sour and in other instances you could be eating something sweet and listening to sour music.“ (you can read more about this performance below)

There is considerable research that has been published to date that shows that auditory cues influence our eating/dining experiences. Researchers have found that what we hear influences everything from what we choose to eat, the rate at which we eat it, the perceived pleasantness of the food in our mouth and the flavor profile of wine and beer.

Molecular gastronomy chefs are working closely with scientists and applying some of the findings in their restaurants to create a rich multisensory eating experience. Charles Spence, head of the Crossmodal Research Laboratory at Oxford University, has worked with molecular gastronomy Chef Heston Blumenthal for years and their research results led to the introduction of the signature seafood dish ‘Sound of the Sea’ on the tasting menu of The Fat Duck restaurant.

With this dish, diners are presented with a plate of food that is reminiscent of a beach thanks to some foam, seaweed and sand. The diners are also given a sea-shell with an iPod hidden inside and just the headphones poking out. The diners are encouraged to put on the headphones (whereupon they hear the ‘sounds of the sea’ soundtrack) before starting to eat the dish.

Charles Spence is now researching how the shapes of foods, plates, cutlery and packaging influence our eating experience. More about his research coming soon at MolecularRecipes.com.

Bridge Between Sensory Modalities

There are several types of associations between physical phenomenons that can be sensed.

Colors and temperature: warm colors and cold colors (e.g. blue is cold, red is hot).

Language and sound: consonants are acute and vocals are soft.

Space and time: past is behind and future is ahead.

“This type of bridge between sensations is called synesthesia” says Mariano Sigman. “It is the capacity of relating things like vision with taste and sound with color; like a direct metaphor”

Wikipedia says synesthesia is a neurologically based condition in which stimulation of one sensory or cognitive pathway leads to automatic, involuntary experiences in a second sensory or cognitive pathway. Over 60 types of synesthesia have been reported, but only a fraction have been evaluated by scientific research.

So this smart group of scientists and musicians asked themselves: is there a similar association between music and taste?

The Research

These mappings between directions in musical and semantic space were found with mappings in both directions: musicians produce music in concordance with this mapping when asked to improvise based on taste words and listeners perceive the corresponding taste words when they listen to music with these properties.

The group performed two experiments to investigate how taste words elicited specific representations in musical space.

In the first experiment, they asked highly trained musicians to improvise on a MIDI keyboard upon the presentation of the four canonical taste words: sweet, sour, bitter and salty. The group then analyzed the musical improvisations and found emerging patterns for each taste word. They could easily decode at high performances (> 80%) the taste word that had triggered the improvisation.

“We then developed and implemented an algorithm that extracts motives from a large corpus of classic and popular songs. The algorithm minimizes the distance between the fragments in the corpus and the region in musical space evoked by taste-words.” says Sigman.

In the second experiment, they played back musical pieces corresponding to each taste word produced by their algorithm to listeners with no musical training. The participants were asked to assign one taste word to each piece after listening for 15 seconds. Results showed that participants could decode well above chance the taste word of the composition (>70%).

Concert of Music and Taste

As mentioned before, the group is currently working on a performance of music and taste in which pianist Bruno Mesz and molecular gastronomy Chef Alejandro Digilio will compose the musical menu.

The performance will take place in a room with very dim lights. People (small groups for each performance session) will sit on cushions or wander through the room. Upon entrance each spectator will receive a tray with canapés prepared following notions of molecular gastronomy on such a way that the perception of taste will be partially camouflaged.

A series of short piano pieces composed with the algorithm described here will be played in a classic piano. Musical pieces and canapés will be in sequential correspondence. In addition, the color of each canapé, which will be determined by color-taste associations, will be matched to lighting conditions of the room providing perceptual coherence between music, taste and color. Several projectors will flash words which were evoked by musicians during the improvisations and by listeners in response to these improvisations. Words will be presented rapidly, as transient events in an otherwise continuous and smooth experience.

Participants of this experience will be asked to work on their introspection and informed that the results of their experience will feed back to construct algorithmically successive versions of the performance. Participants will fuel the installation by 1) providing new words which they evoked during the experience which will be projected to other participants and 2) by reporting (using provided technology) aspects of their introspective experience.

The Future

The next step in the project? Studying the association between moral concepts and music. They have given the seven capital sins to musicians…..!