Report in Jornal da UNICAMP

In a small studio at Unicamp, a woman dances delicately. Her arms and legs flow with elegance, creating a harmony with the music and colorful images that come alive on the walls. With each gesture of the dancer, the environment transforms: when she raises her arms and shifts her shoulders, the images burst into a vibrant fusion of colors. When she reins in her movements, the figures calm, returning to a serene light-yellow hue.

In this performance, filmed at the University’s Interdisciplinary Sound Communication Center (NICS), the dancer demonstrates how BehCreative works—a digital cave developed by musicologist and postdoctoral researcher Elena Partesotti. The technology, whose name derives from the phrase “behave creatively,” is an extended digital musical instrument (EDMI) combining virtual and augmented reality, where users manipulate images and sounds through interactive tools.

Using a computer, a motion tracker, projectors, and speakers, BehCreative generates visual and auditory responses based on the user’s movements—each tied to different sounds and color patterns. “The idea is that your body becomes a musical instrument. There’s no physical object. You produce sounds and images with your movements—something aligned with the embodied cognition paradigm,” explains Partesotti, referring to the philosophical theory that human cognitive experiences are shaped by the body’s interaction with its environment.

A prof. Gabriela Castellano investiga como o cérebro se reorganiza. Fotografia: Antonio Scarpinetti, Jornal da UNICAMP

Originally created to explore how sound and movement combine in immersive settings, this EDMI aims to stimulate artistic and creative processes within music therapy. Studies have shown that musical interventions provide important tools for promoting patients’ physical and psychological well-being. These tools trigger the release of hormones such as oxytocin, dopamine, norepinephrine, and adrenaline—linked to emotions like love and pleasure—and melatonin, which regulates biological processes such as sleep and wakefulness. As a result, they can reduce anxiety and depression, alleviate pain, and improve quality of life for people with cognitive deficits, including Alzheimer’s disease.

More recent research has also highlighted the positive effects of music therapy on brain neuroplasticity—the central nervous system’s ability to adapt to new environmental stimuli—and on the rehabilitation of stroke survivors. For this reason, one of the music therapist’s goals is to evaluate BehCreative’s ability to rehabilitate motor function and cognitive capacity in these patients, a project conducted under the guidance of Professor Gabriela Castellano from the Gleb Wataghin Institute of Physics (IFGW).

According to Castellano, the musical instrument is still in prototype phase, but researchers hope that, one day, patients will be able to use it in their own homes as a complementary physiotherapy treatment.

“We know that debilitating diseases are becoming more common due to sedentary lifestyles and an aging population. So there are more and more people with neurological sequelae from various health conditions. And nowadays clinics and physiotherapists can’t keep up with the growing demand from these patients,” observes the professor.

Cognitive Rehabilitation

Castellano is one of the principal researchers at Unicamp’s Brazilian Institute of Neuroscience and Neurotechnology (BRAINN), which investigates the mechanisms behind epilepsy and strokes and the damage they cause. Before working with Partesotti, Castellano had already been developing extended-reality applications for stroke therapy in partnership with postdoctoral researcher Alexandre Brandão. “We’ve been evaluating what brain changes result from using these applications, aiming to understand how the brain reorganizes afterward. For that, we use neuroimaging techniques such as functional MRI and electroencephalography,” she explains.

However, those devices don’t use musical elements and, being focused on motor rehabilitation, require patients to make specific, predetermined movements—limiting user creativity and, consequently, potential cognitive response. That’s why the researchers now plan to apply the same neuroimaging techniques to BehCreative users, to see if engaging in a more creative activity leads to cognitive improvement.

Preliminary tests using functional MRI have already shown that healthy individuals exhibit altered connectivity in emotion-related brain regions after using BehCreative, affecting emotional regulation and boosting users’ creative expression. These findings are slated for publication in the Nordic Journal of Music Therapy, one of the world’s leading music therapy journals. But due to COVID-19 restrictions, the initial tests involved only five people, making it difficult to generalize the results. More recently, the study was repeated with 15 healthy users, though that data is still under analysis.

Currently, Partesotti and Castellano are beginning experiments with BehCreative in stroke patients and recruiting new candidates for the research. Over ten days of testing at Unicamp, participants will use the equipment for up to ten minutes, during which they’ll undergo an EEG to record any brain-function changes and receive care from a physiotherapist. “The idea is that people decide how to express themselves using this equipment. We call this creative empowerment—providing tools so they can incorporate what they love into their rehabilitation, aiming for both motor and cognitive improvement,” says the researcher.

In a report about entrepreneurship within Universities, the Unicamp Journal highlighted the successes of two daughter companies of CEPID BRAINN. Bioxthica develops solutions in the area of ​​neuroreabilitation and Hoobox operates in the area of ​​robotics and artificial intelligence. Check out below an exclusive translation of article and follow the link at the end of the text to read the full report (in Portuguese).

The following excerpt was originally published on September 18, 2018 in the Jornal da Unicamp.

Preincubation

Incamp [Unicamp´s new businesses incubator] also offers a pre-incubation program. “For a period of one year, it’s a stage to validate an idea or a project, to check if it’s feasible to assemble a prototype or if the business is indeed promising,” explains Mariana Zanatta. Bioxthica is one of the companies that has gone through this stage. Today, already incubated, the company develops solutions in the area of ​​neurorehabilitation based on virtual reality. “When a patient suffers a stroke, there are injuries to the brain that compromise the person’s movement. Our expectation with this technology is to stimulate new areas of the brain to relearn that lost movement”, explains Alexandre Brandão, founding partner of Bioxthica.

The company is a spin-off of one of the Centers for Research, Innovation and Diffusion (Cepid), of FAPESP, the Research Institute on Neuroscience and Neurotechnology (Brainn). In one of the applications of the technology now in development in the company, sensors fixed in the legs of a patient send signals to a controller card when it makes a walking movement. The board, then, connects to a virtual reality environment through special glasses. “The user visualizes an environment that simulates a city with buildings, streets, vehicles, street lanes, etc. and that responds to the actions of this user. This allows a limited movement of a patient to be reproduced as a complete movement. In the virtual environment, the patient visualizes the perfect movement and not limited one”, explains Brandão. “Interactivity in the virtual environment can accelerate patient recovery, stimulating new areas of the brain to take charge of these movements,” he adds.

According to Brandão, the idea is that the new product will be in the market in the medium term, targeting the professionals who works with rehabilitation. “Our challenge now is to come up with an affordable product. I strongly believe in the potential of having a safe and non-invasive method [to help patients in motor rehabilitation], which can have an important impact both in the recovery of patients with impaired mobility and also for the caregivers, “says Brandão.

In addition to Incamp, the company received funds from Sebrae and the Brazilian Enterprise for Research and Industrial Innovation (Emprapii), from CPqD (which is a unit of Emprapii). Bioxthica is also investing in a new design of a wearable device to collect vital signs of patients, in partnership with the company Monitora Soluções Tecnológicas.

Artificial intelligence

Another company that emerged in Brainn was Hoobox, which operates in the area of ​​robotics and artificial intelligence. Incubated since last year [at Unicamp´s Incamp], the startup has created Wheelie, the world’s first computer program capable of translating facial expressions into commands to control a wheelchair and computers. “The technology eliminates the use of body sensors and any type of training”, said engineer Claudio Pinheiro, one of the founders of the company. “I’ve always wanted my research at the University to be a product that impacts people’s lives. For me, this is a way of rewarding the investment that society makes in the public university”, reveals Pinheiro.

Hoobox combines the founders’ entrepreneurial profile with leading-edge research conducted at Unicamp. “We have a very close relationship with the University. Our collaborators are masters and doctoral students who have the experience of research and, at the same time, the experience of working in a startup. We also maintain close contact with teachers because they are the bridge between academia and knowledge, which is the basis of our business”, says Pinheiro. According to him, it was the training at Incamp that enabled the company to elect its market of action: American war veterans for wheelchair mobility technology.

Hoobox already has an office in Houston, Texas, in partnership with Johnson & Johnson’s – the US pharmaceuticals and personal care giant – Laboratory of Innovation. The company’s potential also caught the eye of Albert Einstein Hospital, the largest hospital in Latin America, which has become an investor partner. “Those are environments that bring important benefits to our development: at Einstein we have contact with the hospital environment and with doctors that enable crucial tests in our area of ​​operation. In the United States, we have expanded our knowledge about the health market “, says Pinheiro.