An Analysis of Rate Your Music Ratings Is Today's Music Really Worse? Aaron Levine October 2016

Transcription

1 An Analysis of Rate Your Music Ratings Is Today's Music Really Worse? Aaron Levine October

2 Introduction Rate Your Music (RYM) is one of the largest online databases of music It's also the largest database that contains user submitted reviews and ratings [1] Top albums from the most recent decade are rated lower than top albums from prior decades Is music just getting worse? Hypothesis: we know which albums from 60's and 70's are classics and rate them appropriately. Albums that were added after RYM went online are fiercely debated and the average rating declines accordingly This will be supported if average rating declines after RYM launched Let's use data from RYM to quantify this effect! [1] 2

3 Data Gathering RYM has no API Data gathered using HTML scraping and parsing Time consuming, limited amount of data gathered for this small project Focus on top albums Worst albums from prior decades are forgotten but more recent worst albums are ridiculed on the site and receive many ratings and reviews Entirely different effect, for a future project! Initial goal was to analyze all albums between 1990 and 2014 with over 3.5/5 rating and over 500 reviews RYM's ratings are strict: 3.5/5 actually is quite good for the site Scraped data from top of year charts until all albums with over 3.5/5 and over 500 reviews were collected 3

4 Data Selection RYM ranks albums by a proprietary algorithm that takes into account number of ratings and votes An album with 3.4/5 but 1000 ratings could be ranked above an album with 3.5/5 and 600 ratings After scraping enough HTML from the RYM charts to get all albums with over 3.5/5 and over 500 ratings for the year, a Gaussian distribution was created when the rating requirement was disregarded Reason: tail on the lower end of the rating scale consisting of albums with more votes Ultimately, it doesn't make sense to apply a universal ratings cutoff when the whole point of this project was to determine how much the ratings varied from year to year! Final selection: Scraped HTML for top 680 albums per year, from 1990 to 2014, disregarded all albums with less than 500 votes 4

5 RYM Ratings by Year Scroll to see changes! 5

6 1990 6

7 1991 7

8 1992 8

9 1993 9

10

11

12

13

14

15

16

17 2001 RYM Launched First year of RYM: Mean lower than all years but 1992,

18 2002 Second year of RYM: Lowest mean so far 18

19

20 2004 Mean keeps declining... 20

21

22

23

24 2008 [1] wikipedia.org/wiki/rate_your_music User Reviews added [1] Immediately a 0.05 decline in mean from 2007! Largest previous change was

25

26

27

28

29

30

31 1990 vs

32 Three Distinct RYM Ratings Eras Pre RYM ( Up to 2000): Top albums discussed in pop culture, users have solidified opinions before rating Transitionary period ( ): RYM online, but still in early period. Internet less universally prevalent, lack of reviews Modern RYM (2008-Present) Reviews added, average rating of top albums drops an unprecedented 0.05 in first year of modern era, continues declining throughout era Increased awareness of good but not classic albums shrinks width of top 680 distribution 32

33 Modern RYM Era: A Closer Look Mean declines linearly throughout era Apply a linear regression to quantify the effect Result for year > 2007: f(x) = x Strong linear relationship R 2 = Residual standard error:

34 Corrected RYM Ratings Correct Modern RYM ratings to account for drop in mean For each rating, use: Corrected rating = (initial rating mean(year)) X σ(prerym)/σ(year)) + mean(prerym) mean(year) is from linear regression from mean(prerym) = mean( ) = 3.66 σ(prerym) = σ( ) = 0.16 σ(year) = σ for each year of the modern RYM era This simple method produces a distribution from with the same mean and σ and the distribution Simplest way of correcting for this effect. Many more sophisticated ways to correct can be implemented in future 34

35 Corrected ModernRYM Vs. PreRYM Corrected 35

36 Top 10: PreRYM and ModernRYM Before Corrections Nothing from ModernRYM is in top 10 Rating Year Artist Album Radiohead Ok Computer My Bloody Valentine Loveless Wu Tang Clan Enter the Wu Tang Nas Illmatic Radiohead Kid A GZA Liquid Swords Godspeed You Black Emperor Portishead Dummy Lift Your Skinny Fists Like Antennas to Heaven DJ Shadow Endtroducing Godspeed You Black Emperor F A 36

37 Top 10: PreRYM and ModernRYM After Corrections Mixture of ModernRYM and PreRYM in top 10 Rating Year Artist Album Kendrick Lamar Good Kid Maad City Radiohead Ok Computer My Bloody Valentine Loveless Kanye West My Beautiful Dark Twisted Fantasy D'Angelo Black Messiah Wu Tang Clan Enter the Wu Tang Swans The Seer Nas Illmatic Radiohead Kid A Vektor Black Future 37

38 Results and Conclusions Data supports hypothesis: ratings decline after creation of RYM Simple corrections to ModernRYM to account for ratings declines produces a top album list with an equal mixture of albums from both eras More sophisticated corrections taking into account number of votes and genre could be added as part of a larger future project Future work could also examine the volatile transition period from More difficult to examine because of a relatively rapidly changing and increasing user base Apparently I really need to listen to Kendrick Lamar's 2012 album! 38