Music recommenders Established Last.fm (CF > HYBRID) Pandora (CBF > HYBRID) Genius (CF)... Upcoming Foafing the music / BMAT (HYBRID) Echonest (HYBRID) Mufin (CBF) Hypemachine (SOCIAL)... Dr. S. Baumann DFKI Recommender systems 1
Last.fm (CF) Music service For free AcQve community: 21 Mio User / month More than music: Radio, Videos, Events, Charts, RecommendaQons Founded 2002 East London Sold 2007 to CBS, 140 Mio GBP [Source: E. Pampalk: Forschungsteam Last.fm, Stand 2009] Dr. S. Baumann DFKI Recommender systems 2
Last.fm s RecommendaQons Several Mio Items and several Mio User Types of recommendaqons Music (ArQsts, Events, Videos, Albums, Tracks) Neighbours Human to human RecommendaQons Modi Lean back and Lean forward Not only for PC Mobiles (personalised radio, m.last.fm) Internet enabled hardware in general [Source: E. Pampalk: Forschungsteam Last.fm, Stand 2009] Dr. S. Baumann DFKI Recommender systems 3
Data & RecommendaQons Huge data sources as input Last.fm data sources Scrobbles( Listening behaviour quanqtaqve) <user, track, Qme> 20 Mrd., 800 Mio. per month Social tags <user, item, tag> 50 Mio., 2.5 Mio. per month Feedback sources ( love, ban, etc) Individual profiles and user cluster Open System: Last.fm API [Source: E. Pampalk: Forschungsteam Last.fm, Stand 2009] Dr. S. Baumann DFKI Recommender systems 4
Social tags [Source: E. Pampalk: Forschungsteam Last.fm, Stand 2009] Dr. S. Baumann DFKI Recommender systems 5
How do people tag? [Source: E. Pampalk: Forschungsteam Last.fm, Stand 2009] Dr. S. Baumann DFKI Recommender systems 6
Tagging behaviour [Source: E. Pampalk: Forschungsteam Last.fm, Stand 2009] Dr. S. Baumann DFKI Recommender systems 7
Challenges Sparsity Coldstart Nichy songs and arqsts ( Long Tail ) Bias Scrobbles Popularity Bias Feedback Loops Top listeners Tags Social vandalism / Paris Hilton Effekt Synonyms hyper acqve Labels and arqsts Dr. S. Baumann DFKI Recommender systems 8
Social Vandalism [Source: E. Pampalk: Forschungsteam Last.fm, Stand 2009] Dr. S. Baumann DFKI Recommender systems 9
Pandora (CBF) Origin: Music Genome Project, Tim Westergren, 2000 Facts 2007: 35,000 ArQsts 500,000 Songs 15,000 Songs / Month are analysed 400 Features Detailed value ranges of the features 8 Mio. registered users Dr. S. Baumann DFKI Recommender systems 10
A Pandora: Features * Abstract Lyrics * Accordion Playing * AcousQ Lectric Sonority * AcousQ SyntheQc Sonority * AcousQc Bass Solo * AcousQc Drum Samples * AcousQc Guitar Accompaniment * AcousQc Guitar Layering * AcousQc Piano Accompaniment * AcousQc Rhythm Guitars * AcousQc Rhythm Piano * AcousQc Rock InstrumentaQon W * AcousQc Sonority * Afro Cuban Influences * Wah Wah Guitar * Aggressive Drumming * Well ArQculated AcousQc Guitar Solo * Well ArQculated Alto Sax Solo * Aggressive Female Vocalist * Well ArQculated Electric Guitar Solo * Aggressive Male Vocalist * Well ArQculated Piano Solo * Altered Female Vocal * Well ArQculated Tenor Sax Solo * Altered Male Vocal * Well ArQculated Trombone Solo * Altered Piano Timbres * Well ArQculated Trumpet Solo * Altered Vocal Sound * West Coast Rap Roots * Ambient Soundscapes * Western Classical Influences * Ambiguous Lyrics * Wet Recording Sound * Angry Lyrics * Wet Snare * Angular Melodies * World Music Influences * Atmospheric ProducQon Dr. S. Baumann DFKI Recommender systems 11 * Avant garde Leanings
ExplanaQons [Source: O.Celma & P.Lamere: Music RecommendaQon Tutorial, ISMIR2007] Dr. S. Baumann DFKI Recommender systems 12
Manual analysis by experts Pandora approach 4 songs / album of an arqst SelecQon of representaqve songs Musical outliers are considered, too 400 Features 10 point scale: [0 1 2 3 4 5 6 7 8 9 10] Back beat prominence [0 1 2 3 4 5 6 7 8 9 10] Electric Guitar Wah Wah [0 1 2 3 4 5 6 7 8 9 10] light or breathy vocals Concrete features are confidenqal [Source: O.Celma & P.Lamere: Music RecommendaQon Tutorial, ISMIR2007] Dr. S. Baumann DFKI Recommender systems 13
Example Dolly Parton Stairway to heaven country influences bluegrass influences folk influences a subtle use of vocal harmony mild rhythmic syncopation acoustic sonority demanding instrumental part writing intricate melodic phrasing thru composed melodic style a clear focus on recording studio production minor key tonality melodic songwriting a heavy twang in the vocals acoustic rhythm guitars [Source: O.Celma & P.Lamere: Music RecommendaQon Tutorial, ISMIR2007] Dr. S. Baumann DFKI Recommender systems 14
CBF details Pandora RecommendaQon 400 Features for a song, Vector Space Model Genre dependent weights for the features Cross genre recommendaqons are difficult SelecQon of songs based on nearest neighbor, filtering including: Rights of licensing Prominence, novelty, surprise ArQst similarity is based on 1 representaqve song! [Source: O.Celma & P.Lamere: Music RecommendaQon Tutorial, ISMIR2007] Dr. S. Baumann DFKI Recommender systems 15
Pandora goes social Pandora : CBF+CF Cultural effects are considered now Thumbs down' Feedback 'Thumbs up data' Feedback Mainstream bias is used for filtering Result: Playlists are massively improved [Source: O.Celma & P.Lamere: Music RecommendaQon Tutorial, ISMIR2007] Dr. S. Baumann DFKI Recommender systems 16
Pandora: CBF+CF=HYBRID The New Pandora Radio StaQon instead of recommender system Precision is important not Recall Impact of hybrid approach More user saqsfacqon CBF is resolving problems with coldstart and feedback loops [Source: O.Celma & P.Lamere: Music RecommendaQon Tutorial, ISMIR2007] Dr. S. Baumann DFKI Recommender systems 17
Pandora challenges AutomaQc feature selecqon Social Tags Open issues How to add new genres? How to handle cross genre arqsts? ( shakira problem ) How to add new musical features? (Update of the old database) [Source: O.Celma & P.Lamere: Music RecommendaQon Tutorial, ISMIR2007] Dr. S. Baumann DFKI Recommender systems 18
New recommender systems Foafing the music & BMAT Echonest, Mufin Hypemachine HORST (DFKI) Dr. S. Baumann DFKI Recommender systems 19
Foafing the music / BMAT (HYBRID) Research project of O.Celma (2005) Ph.D theses at UPF/MTG with P. Cano Spinoff BMAT 2006 as startup for Music & Audio Technologies 2009: O. Celma is Chief CreaQve Officer at BMAT HYBRID = CBF + CF + SOCIAL Dr. S. Baumann DFKI Recommender systems 20
Foafing the music (HYBRID) Dr. S. Baumann DFKI Recommender systems 21
Foafing the music Dr. S. Baumann DFKI Recommender systems 22
Foafing the music Dr. S. Baumann DFKI Recommender systems 23
BMAT (HYBRID) CBF: Musical features are detected automaqcally Beispiel: Jamiroquai, Canned Heat Mood: upbeat energetic. Rhythm: 120bpm, no rubato, high percusiveness. Harmony: Dm. Instrumentation: no electronic, singing voice Dr. S. Baumann DFKI Recommender systems 24
BMAT (HYBRID) CF: User Feedback Buying history Playlists Listening behaviour AcQviQes in music networks Dr. S. Baumann DFKI Recommender systems 25
BMAT (HYBRID) SOCIAL: Web Mining of context Jamiroquai - Canned Heat: Country: UK Labels: Acid Jazz, Sony BMG, Columbia Genres: Funk, Disco, Acid Jazz, Jazz Fusion, Pop-rock Years active: 1992 - present Associated acts: Brand New Heavies, Guru, Julian Perretta Dr. S. Baumann DFKI Recommender systems 26
Echonest (HYBRID) Founded by B.Whitman & T. Jehan 2006 CBF + CF + SOCIAL SOCIAL: 1. Reads about music, constantly analyzing millions of blog posts, reviews, playlists and discussion forums to understand how the online world describes every ar<st, album and song. Dr. S. Baumann DFKI Recommender systems 27
Echonest (HYBRID) Founded by B.Whitman & T. Jehan 2006 CBF + CF + SOCIAL CBF: 2. Listens to music, with technology actually listens to audio files, extrac<ng musical aaributes, such as tempo, instrumenta<on, key, <me signature, energy, harmonic and <mbral structures, to understand every song in similar ways a musician would describe it (e.g. "heavy beat, swing groove, fast tempo, 4/4 <me, key of B flat, mezzo piano"). Dr. S. Baumann DFKI Recommender systems 28
Echonest (HYBRID) Founded by B.Whitman & T. Jehan 2006 CBF + CF + SOCIAL CF: 3. Learns about music trends, analyzing the en<re world of online music behavior who's talking about which ar<sts this week, what songs are being streamed and downloaded, etc. to understand the latest trends, buzz and fan opinion. Dr. S. Baumann DFKI Recommender systems 29
Echonest: Recommender Dr. S. Baumann DFKI Recommender systems 30
Echonest: Analysis Dr. S. Baumann DFKI Recommender systems 31
Echonest: Promobot Dr. S. Baumann DFKI Recommender systems 32
Mufin (CBF) Founded: 2007, Fraunhofer Technologie Pure CBF Dr. S. Baumann DFKI Recommender systems 34
Hypemachine (SOCIAL) Founded: 2005 by Anthony Volodkin SOCIAL > HYBRID=SOCIAL+CF(Last.fm) Usage of Music Weblogs as sources of data Memetracking: AutomaQc tracking of arqsts, songs and links Now: HYBRID, Last.fm Audioscrobbler is adding listening behaviour to Hypemachine Dr. S. Baumann DFKI Recommender systems 35
Hypemachine (SOCIAL) Dr. S. Baumann DFKI Recommender systems 36
Hypemachine + Last.fm (HYBRID) Dr. S. Baumann DFKI Recommender systems 37
HORST (SemanQc)
SemanQc Data
ArQsts, Bands, Directors, Songs, Movies,...
hzp://horst.d{i.de
References hzp://www.last.fm hzp://www.pandora.com hzp://foafing the music.iua.upf.edu/ hzp://www.bmat.com/ hzp://the.echonest.com/ hzp://www.mufin.com/ hzp://hypem.com/ Dr. S. Baumann DFKI Recommender systems 46