AVE
Affective virtual Environments
Affective Virtual Environments
The development of an Affective Virtual Environment requires at least two components:
1) An emotion recognition system.
2) A virtual environment generator.
Emotion Recognition System
Semantic Model
We built a long-short term memory (LSTM) network architecture to capture the information’s context. The algorithm was trained with two databases containing labeled poetry and film reviews.
Acoustic Model
We propose a convolutional neural network (CNN) trained with three different augmented datasets.
Virtual Environment Generation
SER 2 Audio
To transfer speech emotion to music, we use a computational model to create chord progressions based on consonance and harmonic dispersion in the Tonal Interval Space.
The graphical environment is built using a text-to-image API. This image is scaled and used as a style source transferred over an equirectangular projection image chosen for each emotional category.
Emotional Machines 