Dr. Roberto Dillon has written this article about his Music-Emotion Driven Game Engine. I think this idea to really have potential.

Music-Emotion Driven Game Engine

In the fast developing world of videogames, sound has traditionally played a less important role than other features, such as graphics. Nonetheless, recently, due to the latest technological advances, both developers and audiences alike have understood how much sound and music can do to enhance gaming experiences and increase the overall realism and emotional impact of virtual worlds.

Today, many music games are being released with great acclaim on different formats but many of these games tend to be in the rhythm/action category where the player is actually “reacting” to music while a real and natural “interaction” with this medium is still lacking.

This aspect is currently being addressed by a team of researchers and developers at Nanyang Polytechnic in Singapore, led by Italian computer engineer Dr. Roberto Dillon, with a project codenamed M-EDGE (”Music-Emotion Driven Game Engine”) which is fully supported by the National Research Foundation of Singapore.

The underlying idea of the project is to assess and further investigate music related cognitive science topics and integrate the results into games by designing and developing an interactive and emotionally-aware musical game engine that enables players to experiment with a completely new gaming interface.

In fact, by using this engine as a development tool, game developers will be able to build content on top of it where players are not limited to control their actions in the virtual world by using standard interfaces, like a mouse or keyboard, but also to directly affect the game progress by expressing emotional content through the playing of a real musical instrument.

To accomplish this, M-EDGE will be developed to recognize different basic emotions (like happiness, sadness and anger) as expressed by players in their musical playing on an instrument of their choice like flute, guitar, violin or drums.

This information can then be used to control the game accordingly by allowing the player’s in-game character to perform particular tasks or by developing the game story in particular directions. For example, one of the games currently being designed by Dr. Dillon to test the system capabilities is based on the well known legend of the Pied Piper of Hamelin. In the game the player will impersonate the Piper himself by actually playing a flute and go through the storyline, including its multiple endings as described by different versions of the original legend, according to the emotions he will be able to express through his very own playing and hence decide the fate of Hamelin young people.

Obviously, this kind of games would be a great tool for players who have previous musical knowledge and know how to play a musical instrument, nonetheless special attention will also be dedicated to give a chance of successful playing to people who never played music before. This will be achieved by allowing prospective players to experiment with simple percussive instruments so as to give a chance and an incentive to get into the fascinating world of music making through games to as many people as possible.

The Singaporean team believes the concept of an emotion-based music game engine like this would represent a leap forward within the music games genre and could also have a strong potential for commercial exploitations, besides having a valuable educational side. In fact, as suggested, it will actually encourage people to learn and practice music through a challenging and fun tool where all players, from beginners to professionals, will be given a chance to freely express themselves through music, something no other game has ever done before.

Besides these, more serious and medical applications could also be possible: the final framework could, in fact, be a valuable tool in assisting emotionally impaired people to practice and experiment with their feelings under supervision of a physician/psychologist.

The project, though still in its early stages, is raising a good interest in the gaming community and has already been presented at the Game Convention Asia Conference in Singapore last September. For those interested, slides of the presentation can be downloaded from Dr. Dillon’s website.

This entry was posted on Friday, November 16th, 2007 at 1:33 pm and is filed under Arts, Games, Ideas. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.