Affective virtual Environments

Affective Virtual Environments

The development of an Affective Virtual Environment requires at least two components:

1) An emotion recognition system.
2) A virtual environment generator.

Emotion Recognition System

Semantic Model

We built a long-short term memory (LSTM) network architecture to capture the information’s context. The algorithm was trained with two databases containing labeled poetry and film reviews.

Acoustic Model

We propose a convolutional neural network (CNN) trained with three different augmented datasets.

Virtual Environment Generation

SER 2 Audio

To transfer speech emotion to music, we use a computational model to create chord progressions based on consonance and harmonic dispersion in the Tonal Interval Space.

SER 2 Visual

The graphical environment is built using a text-to-image API. This image is scaled and used as a style source transferred over an equirectangular projection image chosen for each emotional category.