Report in Jornal da UNICAMP

In a small studio at Unicamp, a woman dances delicately. Her arms and legs flow with elegance, creating a harmony with the music and colorful images that come alive on the walls. With each gesture of the dancer, the environment transforms: when she raises her arms and shifts her shoulders, the images burst into a vibrant fusion of colors. When she reins in her movements, the figures calm, returning to a serene light-yellow hue.

In this performance, filmed at the University’s Interdisciplinary Sound Communication Center (NICS), the dancer demonstrates how BehCreative works—a digital cave developed by musicologist and postdoctoral researcher Elena Partesotti. The technology, whose name derives from the phrase “behave creatively,” is an extended digital musical instrument (EDMI) combining virtual and augmented reality, where users manipulate images and sounds through interactive tools.

Using a computer, a motion tracker, projectors, and speakers, BehCreative generates visual and auditory responses based on the user’s movements—each tied to different sounds and color patterns. “The idea is that your body becomes a musical instrument. There’s no physical object. You produce sounds and images with your movements—something aligned with the embodied cognition paradigm,” explains Partesotti, referring to the philosophical theory that human cognitive experiences are shaped by the body’s interaction with its environment.

A prof. Gabriela Castellano investiga como o cérebro se reorganiza. Fotografia: Antonio Scarpinetti, Jornal da UNICAMP

Originally created to explore how sound and movement combine in immersive settings, this EDMI aims to stimulate artistic and creative processes within music therapy. Studies have shown that musical interventions provide important tools for promoting patients’ physical and psychological well-being. These tools trigger the release of hormones such as oxytocin, dopamine, norepinephrine, and adrenaline—linked to emotions like love and pleasure—and melatonin, which regulates biological processes such as sleep and wakefulness. As a result, they can reduce anxiety and depression, alleviate pain, and improve quality of life for people with cognitive deficits, including Alzheimer’s disease.

More recent research has also highlighted the positive effects of music therapy on brain neuroplasticity—the central nervous system’s ability to adapt to new environmental stimuli—and on the rehabilitation of stroke survivors. For this reason, one of the music therapist’s goals is to evaluate BehCreative’s ability to rehabilitate motor function and cognitive capacity in these patients, a project conducted under the guidance of Professor Gabriela Castellano from the Gleb Wataghin Institute of Physics (IFGW).

According to Castellano, the musical instrument is still in prototype phase, but researchers hope that, one day, patients will be able to use it in their own homes as a complementary physiotherapy treatment.

“We know that debilitating diseases are becoming more common due to sedentary lifestyles and an aging population. So there are more and more people with neurological sequelae from various health conditions. And nowadays clinics and physiotherapists can’t keep up with the growing demand from these patients,” observes the professor.

Cognitive Rehabilitation

Castellano is one of the principal researchers at Unicamp’s Brazilian Institute of Neuroscience and Neurotechnology (BRAINN), which investigates the mechanisms behind epilepsy and strokes and the damage they cause. Before working with Partesotti, Castellano had already been developing extended-reality applications for stroke therapy in partnership with postdoctoral researcher Alexandre Brandão. “We’ve been evaluating what brain changes result from using these applications, aiming to understand how the brain reorganizes afterward. For that, we use neuroimaging techniques such as functional MRI and electroencephalography,” she explains.

However, those devices don’t use musical elements and, being focused on motor rehabilitation, require patients to make specific, predetermined movements—limiting user creativity and, consequently, potential cognitive response. That’s why the researchers now plan to apply the same neuroimaging techniques to BehCreative users, to see if engaging in a more creative activity leads to cognitive improvement.

Preliminary tests using functional MRI have already shown that healthy individuals exhibit altered connectivity in emotion-related brain regions after using BehCreative, affecting emotional regulation and boosting users’ creative expression. These findings are slated for publication in the Nordic Journal of Music Therapy, one of the world’s leading music therapy journals. But due to COVID-19 restrictions, the initial tests involved only five people, making it difficult to generalize the results. More recently, the study was repeated with 15 healthy users, though that data is still under analysis.

Currently, Partesotti and Castellano are beginning experiments with BehCreative in stroke patients and recruiting new candidates for the research. Over ten days of testing at Unicamp, participants will use the equipment for up to ten minutes, during which they’ll undergo an EEG to record any brain-function changes and receive care from a physiotherapist. “The idea is that people decide how to express themselves using this equipment. We call this creative empowerment—providing tools so they can incorporate what they love into their rehabilitation, aiming for both motor and cognitive improvement,” says the researcher.