Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now


In the mood

In this week's feature, Professor Rumsey talks emotion and finds out whether an audio signal might have the potential to give us the tingle factor.

There’s a common understanding that certain music makes us feel sad or happy, or contains the so-called “tingle factor” that sends shivers down the spine. Some of the factors that give rise to these emotions are intensely personal and others seem to be more universal. For example there may be a particular piece that has sad connotations for you alone because you were listening to it at a moment in your life when something bad happened. On the other hand, some pieces come up regularly in the top ten list of those that arouse the tingle factor in a wide range of people. In music analysis a certain amount of research has been done to find out what gives rise to emotional responses, and there is a growing understanding of the dependency on rhythmic complexity, harmonic tension, delayed gratification and so forth. How about audio, though? Is there a similar effect on the emotions that depends on features that can be extracted from audio signals? In some cases these might be related closely to the musical content and in others they might be specific mixing characteristics such as the bass level, spatial positioning, frequency balance or the amount of compression. At NHK Science and Technology Research Labs in Japan, Satoshi Oode has been conducting experiments to find out how to evaluate sound in terms of “Kandoh”. This is so that NHK can make programmes and develop sound production techniques to increase the emotional involvement of listeners. Kandoh is difficult to translate, but it amounts to something like “goose bumps”, “stirring” and “strong emotion”, according to Oode, which seems distinctly like the tingle factor to me. They’ve been trying to find words that express these Kandoh feelings and getting participants to rate programme material on verbal scales, with words such as “nostalgic” and “romantic” coming out strongly. A recent AES paper by Panda and Paiva, from the University of Coimbra in Portugal, describes a means of automatic mood tracking for audio music. One of the main reasons for doing this, based on features of the audio signal, is to enable new computer search methods based on the emotional characteristics of recorded material. This is a branch of music information retrieval (MIR). As they say, mood does not stay constant throughout a song for example, but most research so far has not taken this into account, only attempting to find a single mood label for a song. This automatic mood tracking work is based on a simple model of mood developed by Thayer, which gives rise to two dimensions: “arousal” and “valence”. Two MIR software toolkits were used to extract a collection of audio features from 189 song clips, including spectral measures, sound quality metrics, dynamic and musical characteristics. Trained using mood annotations from real listeners, the automatic system was able to achieve a little over 50% accuracy in predicting these. Part of the problem was that listeners agreed more about which were happy clips, and how happy they were, but less about which ones had negative emotions. Also the song clips were not balanced in their emotional range, but it’s a good start. Just imagine having a plug-in to your favourite workstation software that would allow you to tweak the mix for maximum happiness…