Emotion-based Cross-Modal Retrieval Demonstration

 

Task

Generated MVs

Query (ID of Original MV)

Top 1 (ID of Original MV)

Audio-to-Video

Demo1

The Audio of (1)

The Video of (2)

Audio-to-Video

Demo2

The Audio of (3)

The Video of (4)

Video-to-Audio

Demo1

The Video of (5)

The Audio of (6)

Video-to-Audio

Demo2

The Video of (7)

The Audio of (8)

 

Information of the Original MVs

ID

Artist Name

Song Title

1

Michael Franti & Spearhead

Say Hey (I Love You)

2

Of Montreal

Wraith Pinned To The Mist And Other Games

3

Napalm Death

Procrastination On The Empty Vessel

4

Rage Against The Machine

Bombtrack

5

Sigur Ros

Von

6

Radiohead

No Surprises

7

Mika

Love Today

8

Alphabeat

Fascination

 

 

All the multimedia sources are obtained from the DEAP dataset.

http://www.eecs.qmul.ac.uk/mmv/datasets/deap/

Koelstra, S., Muhl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., 
and Patras, I., “DEAP: A database for emotion analysis using physiological signals,” 
IEEE Transactions on Affective Computing, 3(1), pp. 18-31, 2012.