Note:
This bibliographic page is archived and will no longer be updated.
For an up-to-date list of publications from the Music Technology Group see the
Publications list
.
Multimodal Deep Learning for Music Genre Classification
| Title | Multimodal Deep Learning for Music Genre Classification |
| Publication Type | Journal Article |
| Year of Publication | 2018 |
| Authors | Oramas, S. , Barbieri F. , Nieto O. , & Serra X. |
| Journal Title | Transactions of the International Society for Music Information Retrieval |
| Volume | 1 |
| Issue | 1 |
| Pages | 4-21 |
| Abstract | Music genre labels are useful to organize songs, albums, and artists into broader groups that share similar musical characteristics. In this work, an approach to learn and combine multimodal data representations for music genre classification is proposed. Intermediate representations of deep neural networks are learned from audio tracks, text reviews, and cover art images, and further combined for classification. Experiments on single and multi-label genre classification are then carried out, evaluating the effect of the different learned representations and their combinations. Results on both experiments show how the aggregation of learned representations from different modalities improves the accuracy of the classification, suggesting that different modalities embed complementary information. In addition, the learning of a multimodal feature space increases the performance of pure audio representations, which may be specially relevant when the other modalities are available for training, but not at prediction time. Moreover, a proposed approach for dimensionality reduction of target labels yields major improvements in multi-label classification not only in terms of accuracy, but also in terms of the diversity of the predicted genres, which implies a more fine-grained categorization. Finally, a qualitative analysis of the results sheds some light on the behavior of the different modalities on the classification task. |
| preprint/postprint document | http://hdl.handle.net/10230/35647 |
| Final publication | http://doi.org/10.5334/tismir.10 |
