The Basics of Spotify Music Recommendations

That Arthur Headphone meme

I am always very interested in finding new music. Always. In college I would frequently go to the multimedia library, go to the librarian in charge of the audio CDs, and ask them to go and pick a random one. Several of the librarians were confused. Some could see what I was up to and would smirk. Others looked at me like I was an idiot.

I would get all kinds of stuff, obviously. An album of sweet polish songs, some violin concerto, the soundtrack from Jurassic Park, another violin concerto. At best it was a quick dose of spontaneity in my life and at it’s worst I would forget I checked it out and get a $2 fine.

Nowadays well over half of listeners consume music online. Spotify, YouTube, and other streaming services are responsible for showing users new music. With the popularity of features like Spotify’s Discover Weekly, I’ve been curious what goes into programmatically recommending music to listeners.

How Spotify Recommends Music

Collaborative Filtering at Spotify

The first and most widespread method is Collaborative Filtering. An early example is Netflix allowing users to rate programs in order to compare and contrast different users ratings, or consumption patterns. These ratings “determine the user’s preferences” and create suggestions based on other users’ preferences.

Spotify has since gotten rid of ratings so in their case user patterns are created from usage data, pressing play. Unfortunately because of this Collaborative Filtering is somewhat of a popularity contest as songs that are the most played have accrued the most data and are thus more likely to get played again.

In this recommendation model, a recommendation service scours the web for any words or phrases in close proximity to references of its catalogue. Say the song in question is “Life is a Highway” by Rascal Flatts. In order to get a better idea of who would like this song, Spotify looks for any reference to this song on the internet and finds all related terms. Among that data it will weigh the most frequently used terms heavier. Much like Collaborative Filtering, it cross-references this information with user data to find the strongest correlations.

What?

The past two methods are also very similar in that they use data “about” music. Play counts, or reviews aren’t the same as actually listening to music. The third way that Spotify creates recommendations for users is through convolutional neural networks and deep learning.

There is a large gap between the audio data in music and the attributes of music that affect listener preferences. Qualities like genre, or instruments used in a piece of music are fairly easy to extract from audio data. Other aspects like mood or the year that the music was created in are much more difficult.

In spite of the challenges, making inferences based the actual sound of music is helpful because it circumvents the popularity bias presented in the first two methods and it goes without saying that the actual sound definitely plays a part in determining if you’ll like a piece of music.

Cover for DJ Shadow’s “Endtroducing”

The ways in which music recommendation and aesthetic modeling has evolved over the years is fascinating. I for one am inspired by new tools to discover and connect with beautiful music.

But there’s one thing I can’t recreate with Spotify right now. I can’t go up to the “librarian” and ask for a random pick. Maybe some day…

Recommending music on Spotify with deep learning

How Does Spotify Know You So Well?