• Medicine
  • Spirituality
  • Astronomy
  • Sociology
  • Technology
  • History
  • Open Access
  • News

Emotion Recognition Using A Transformer-Based Model

A Transformer-based approach for processing electrocardiograms is practical for boosting emotion recognition ability.

The Transformer's attention processes may be utilized to create contextualized representations of a signal, assigning greater weight to pertinent sections.

A fully connected network can then look at these representations to determine how people feel.

A team of researchers headed by Juan Vazquez-Rodriguez and Grégoire Lefebvre from the University of Grenoble Alpes in France collected numerous ECG datasets with no emotion labels to pre-train a model they subsequently fine-tuned for emotion identification using the AMIGOS dataset.

They use self-supervised learning to overcome the relatively modest number of datasets containing emotional descriptors.

Their technique achieved state-of-the-art performance for emotion identification using ECG data on AMIGOS.

The research demonstrated that transformers and pre-training are potential ways of recognizing emotions using physiological information.

Binary Emotion Recognition

A downstream task of binary emotion detection (high/low levels of arousal and valence) was tested on ECG signals.

The datasets utilized were presented, preprocesses were applied, and two phases of pre-training and fine-tuning were parametrized.

In this dataset, 40 respondents saw films specifically designed to elicit feelings.

Following each video, they do a self-assessment of their emotional condition.

Subjects assessed their degrees of arousal and valence on a scale of 1 to 9.

37 of the 40 individuals viewed 20 videos, while the other three subjects only watched 16 films.

ECG data were collected on both the left and right arms throughout each experiment.

To fine-tune our model, we exclusively utilized data from the left arm.

AMIGOS contains a preprocessed version of the data that has been down-sampled to 128Hz and filtered using a low-pass filter with a cut-off frequency of 60Hz.

These pre-processed data sets were utilized in studies, including the pre-training phase.

The ECG data utilized for fine-tuning consists of around 65 hours of recordings.

Signal Pre-processing, Pre-training, And Fine-tuning

Signals were filtered using the 8th order Butterworth band-pass filter with 0.8Hz low and 50Hz high cut-off frequencies.

Pre-training the model on many datasets should reduce overfitting when fine-tuning one.

Moreover, the model without pretraining tended to overfit soon, but the pre-trained model did not.

These findings showed the value of pretraining Transformers.

The pre-trained model outperforms the untrained model inaccuracy and F1 score.

Instead of employing a pre-trained signal encoder, The researchers evaluated architecture by fine-tuning our model on AMIGOS with all parameters randomly initialized.

Since these evaluations yield arousal and valence ratings from 1 to 9, we utilize the average as a low or high threshold.

The dataset's emotional self-assessments were utilized as labels.

The model was fine-tuned using the AMIGOS dataset for emotion identification.

The receptive field size was chosen because it is equivalent to the usual ECG signal interval between peaks, 0.6-1s.

Thus, the receiver field is 113 input values or 0.88s.

The first layer and encoder output apply layer normalization.

The input encoder is a 3D-CNN with ReLU activation.

Both models have 83401 10-second segments pre-trained.

When testing our model using AMIGOS segments, we fine-tune the pre-trained model with the other half of AMIGOS.

Then two models were pre-trained, one utilizing half of the AMIGOS data and the other half.

Because AMIGOS was utilized for fine-tuning, the same segments could not be used for pretraining and evaluation.

For pretraining, they employ ASCERTAIN, DREAMER, PsPM-FR, PsPM-RRM1-2, PsPM-VIS, and AMIGOg.

Segments of 10 seconds are ultimately added.

Each subject's signal is normalized to have zero mean and unit variance.

ECG display
ECG display

Future Of Using Transformers For Emotion Recognition From ECG Signals

This study yielded a variety of viewpoints.

New pre-training tasks may be investigated; alternative approaches, such as contrastive loss or triplet loss, may be more efficient in ECG signal specificities than masked points prediction, which was employed in this study.

Extending this study to additional input modalities (EEC, GSR, and even non-physiological inputs like environmental sensors) and, in general, processing multimodal scenarios might be beneficial for boosting emotion identification ability.

Finally, bigger stmowith more data from different places will help us better understand how this method works.

Final Words

The researchers looked at using transformers to detect arousal and valence in ECG signals.

This technique employed self-supervised learning for pre-training using unlabeled data, followed by fine-tuning with labeled data.

The trials showed that the model generates robust features for predicting arousal and valence on the AMIGOS dataset and outperforms current state-of-the-art techniques.

This study showed that self-supervision and attention-based models, such as Transformers, may be effectively employed in emotion recognition research.

About The Authors

Suleman Shah

Suleman Shah - Suleman Shah is a researcher and freelance writer. As a researcher, he has worked with MNS University of Agriculture, Multan (Pakistan) and Texas A & M University (USA). He regularly writes science articles and blogs for science news website immersse.com and open access publishers OA Publishing London and Scientific Times. He loves to keep himself updated on scientific developments and convert these developments into everyday language to update the readers about the developments in the scientific era. His primary research focus is Plant sciences, and he contributed to this field by publishing his research in scientific journals and presenting his work at many Conferences. Shah graduated from the University of Agriculture Faisalabad (Pakistan) and started his professional carrier with Jaffer Agro Services and later with the Agriculture Department of the Government of Pakistan. His research interest compelled and attracted him to proceed with his carrier in Plant sciences research. So, he started his Ph.D. in Soil Science at MNS University of Agriculture Multan (Pakistan). Later, he started working as a visiting scholar with Texas A&M University (USA). Shah’s experience with big Open Excess publishers like Springers, Frontiers, MDPI, etc., testified to his belief in Open Access as a barrier-removing mechanism between researchers and the readers of their research. Shah believes that Open Access is revolutionizing the publication process and benefitting research in all fields.

Recent Articles

  • Stone Age Humans Conducted Surgical Amputation, A New Study Finds Evidence

    Stone Age Humans Conducted Surgical Amputation, A New Study Finds Evidence

    Scientific investigation in Borneo has unearthed the world's first documented instance that stone age humans conducted surgical amputation. This discovery represents a significant achievement in the annals of human prehistory.

  • 180 Tips - Best Website For Football Prediction And Tips

    180 Tips - Best Website For Football Prediction And Tips

    How can you beat the best odds at the most well-known online bookmakers when betting on soccer matches? Today's 180 Tips forecasts from Betwinner360 include a record of winning methods. 180 tips provides in-depth research of more than 50 different football leagues, betting tactics organized by market, and expert predictions for each match.

  • Virgo And Sagittarius Compatibility - Great Conversations

    Virgo And Sagittarius Compatibility - Great Conversations

    Virgo and Sagittarius's compatibility is such that they are attracted to one another right away. They base their first interactions on how well they get along in conversation, which comes naturally to them both. These changeable signs will enjoy every minute of talking to each other to death! They engage in frenzied arguments, lengthy philosophical discussions, and fast banter.

  • Pisces And Aquarius Compatibility - Creative Brilliance

    Pisces And Aquarius Compatibility - Creative Brilliance

    Pisces and Aquarius compatibility is such that these two are masters of manifestation while working together. When Aquarians let go of their rigid views, they become natural alchemists who are capable of creating with their creative brilliance. Because of their more flexible and understanding nature, Pisces may assist them in becoming less stuck and releasing their boundless creative potential.

  • Pisces And Scorpio Compatibility - Best Match For Love

    Pisces And Scorpio Compatibility - Best Match For Love

    Pisces and Scorpio compatibility is that they have an innate knowledge of one another, even though they are frequently mysterious to others. These two are irresistibly attracted to each other because they feel like they understand each other but don't say it out loud. They seem to have a psychic connection because they can easily tune into one another's brains.

  • A Massive 7.6 Earthquake Rocks Papua New Guinea

    A Massive 7.6 Earthquake Rocks Papua New Guinea

    On Sunday, a massive 7.6 earthquake rocks Papua New Guinea, causing landslides, cracking roads, and damaging houses. It is currently unknown how extensive the damage is; however, there is debris scattered over the highways and fissures popping up in the roads. According to the United States Geological Survey, the epicenter of the earthquake was located at a depth of 90 kilometers, or around 56 miles, and was close to the town of Kainantu, which has a population of approximately 8,500 people.

  • Most Frequent Angel Numbers You Keep Seeing Is Luck Or Unlucky

    Most Frequent Angel Numbers You Keep Seeing Is Luck Or Unlucky

    The most frequent angel numbers you keep seeing are distinct from other forms of esoterica in that they have no connection to the date that you were born. In numerology, every number has a meaning. Nevertheless, certain number sequences are more prevalent than others. Because all of us have come across them, three-digit angel numbers are widespread.

  • Pisces And Leo Compatibility Is Quite Strong

    Pisces And Leo Compatibility Is Quite Strong

    Leo will inevitably rule Pisces in some aspect of life, Pisces and Leo compatibility whether it is in terms of work, friendship, or love. However, as Leos are kind and loving rulers who have no malice in their hearts, and since the fish don't mind being ruled if it's done with warmth and love, this isn't always a terrible thing.

  • Angel Number 6666 Meaning - A Sign That Your Heart Is Full Of Love

    Angel Number 6666 Meaning - A Sign That Your Heart Is Full Of Love

    If you keep seeing the same number in your life, it's not a coincidence. The angels use Angel Number 6666 meaning to send you an important message. If you can clearly interpret the message of angel number 6666, it will serve you well. This number appears in your life directly from the divine realm.