For future researchers: limitations, caveats and lessons learned October 2014 As a follow up to the meeting of researchers in Vienna on October 4, 2014, the Artist Revenue Streams team wanted to articulate some of the limitations that are inherent in the Artist Revenue Streams structure. These are issues that we (a) discussed and debated at length while designing the project, (b) tested and rejected during our building process, or (c) that we learned the along the way, but are not explicitly stated in the protocol documents. 1. Building a representative sample is tough 2. It s hard to ask musicians about the dollar value of their activity 3. Gross versus net 4. Stakeholders will only care about certain types of musicians 5. Collective societies are different in the UK/EU 6. Government subsidies: absent in US but present in many other places We think a review of these six issues will be helpful to any future projects. 1. It s difficult to draw from a representative sample As much as we would have loved to conduct research that drew from a representative sample, it was practically impossible to do with the US musician population. First, we do not know the total size of the US musician population the denominator in the equation. We have written about the difficulties in calculating its size here. Second, there is no one organization that represents the majority of musicians in the US, thus making it impossible to randomize a request for participation. To compensate for this, we did our best to engage as many musicians working in diverse fields. With our interview work, we used snowball sampling to move beyond familiar musicians to those with which we had no prior contact. And with our survey, we reached to out to over 150 music-related organizations and associations, asking each to encourage their members to participate. This was in addition to partnerships, paid advertising, earned media and social media marketing efforts. And this coordinated marketing effort paid off; over 5,300 US-based musicians and composers completed this survey, which is
a huge response rate, and sufficient for us to examine the details of various subpopulations of respondents. We are also able to compare our survey data set to some other data sets. For instance, our Jazz Musicians and Money from Music report compares jazz respondents from our survey with jazz respondents from earlier work by Joan Jeffri. And, in other cases, we are able to compare our survey data to some data collected by the US Census about musicians and composers. These are not perfect comparisons, but these larger sets collected by the government over time do give us some helpful reference points. Representativeness should always be a goal, and it may be possible to randomize requests for participation if there are organizations in your country/region that represent the majority of musicians and/or composers. Either way, be clear about how you will achieve this, or how you will compensate for a lack of representativeness. 2. It s really hard to ask musicians how much they make in dollars/euros on any particular activity. At Q12 of the Money from Music survey, we asked musicians to allocate their music income amongst eight possible categories using percentages. The survey was programmed to only accept answers that added up the eight categories to 100%. We did not ask musicians to input dollar amounts into these eight categories. We chose not to ask them to about dollar amounts because: during interviews, musicians were hesitant to talk with us about the actual dollar value of their activities, even with guaranteed anonymity some musicians had no idea how much they made from various activities we also found that survey beta testers struggled with any questions that required some mental math, such as adding up income that came from disparate sources. The also struggled when there were too many choices presented. During survey construction, we tested another version of Q12 that had approximately 40 input boxes instead of 8. We knew that this would reduce the problems of survey respondents having to do math in their heads because there would be specific input boxes for different income sources, but the layout of a question with 40 line items which required onscreen scrolling was impossible for survey respondents to navigate and comprehend. It was too much information, too many boxes. The strategy we adopted struck a balance between data collection and respondent capacity. First, we kept the number of input categories to 8 thus not overwhelming the survey taker with input boxes. Second, we asked for percentage amounts instead of dollar 2
amounts, which was something musicians had an easier time answering. Third, the use of percentages meant the question wasn t too invasive. Clearly, these choices made it less possible to calculate the dollar value of certain activities, but this question structure was sufficient for gathering information about the allocation of revenue amongst key revenue streams. This challenge was also a reason that we conducted the financial audits, which involved us reviewing the actual income and expenses of a handful of musicians over time. This was the most accurate way for us to measure income and expense over time, but it was extremely difficult to get musicians to trust us enough to hand over personal financial records, and it is a very resource-intensive data collection process that is difficult to scale. 3. Gross versus net If you ask a musician how much money they made on t-shirts/merchandise in the past 12 months, they might do some math in their head and say: 100 shirts x $20 each = $2,000 They might have grossed $2,000, but what about the costs of making or buying those shirts? After costs are deducted, the net income from those shirts might be $1,400. Now, ask a similar question about how much money they made from digital downloads in the past 12 months. A musician might think about their CD Baby statement and say: 1,000 single downloads x 60 each = $600 Again, this is what the musician remembers, and it s what the musician sees in his or her bank account. But this digital download answer is actually an adjusted gross number, because itunes and CD Baby have already taken their cuts. And this doesn t even account for the origination costs to make the music. The correct answer is probably that they netted $0, if the musician considered any costs associated with recording the music or marketing the project. On the surveys and in our interviews, we asked about gross income or revenue before costs. First, we needed some consistency in our requests and, second, we needed to make it as simple as possible. Though expenses are really important to acknowledge, asking musicians to calculate the net income of any activity on a survey or during an interview was too much. This issue underscores the value of using multiple methods. The financial case studies which looked at real income and expenses gave us a way to overcome gross versus net issues, and calculate musicians net income. 3
4. Some stakeholders/readers will only care about certain musician populations, or certain revenue streams In 2012, we presented some data about sound recording income at a music industry conference in the US. The audience seemed bored and detached, even though this was the first time this data had ever been presented. After the panel, an audience member bluntly said that our data wasn t interesting to him because the sample had too many classical musicians in it a genre that he perceived to be of little value in the commercial music industry landscape that he worked in. Point taken. We went back and amended that data memo to provide some meaningful information to those whose focus is on the commercial value of sound recordings. But this exchange also served as an important reminder. Everyone who reads or evaluates your research will have a certain interest. Some will only care about the data on professional musicians. Some will only care about commercial musicians. Some will only care about certain genres, like rock musicians, or specific roles, like composers. Some will really only want to know how technology has impacted musicians earning capacity. For instance, journalists who get in touch with us often want to know how much money musicians (en masse) earn from specific activities like streaming music on Spotify, or selling t-shirts. There are three ways to prepare to meet different constituencies expectations: Think about the sub-populations of musicians that stakeholders especially government officials, the media, and partners will want to know about. What characteristics would define a professional musician or a commercial musician, or any other sub-populations that you intend to study. The ARS team has a list of our self-made groups and their defining characteristics Ask the demographic/descriptive questions that will allow you to create the appropriate sub-categories of respondents for filtered analysis. The ARS has a list of demographic/descriptive questions we asked on the survey. This is a critical step, and makes any survey data you collect much more valuable. Manage expectations in advance. Be clear about what kind of data you will collect, and what it will be able to tell stakeholders. 5. Collective societies are different in the UK/EU The Artist Revenue Streams project is a reflection of how the US music industry operates. This is not only seen in our list of 42 streams, but also in how we collected data. But it 4
will be different in other countries, which means there may be other ways to access information. In the US we have three performing rights societies (ASCAP, BMI, SESAC), which means there is no single source for royalty data. In addition, these PROs collect from many licensors, but based on fewer rights. For instance, we have no performance royalty for sound recordings, so the PROs don t collect any money for performers. And, in the US, mechanicals are not handled by an autonomous agency, but instead paid to the record label, which then pays the publisher. It will be different in other countries. In some cases, your performance rights and mechanical rights societies handle much more payment data, for a much greater majority of the musician population. If you are able to strike up good relationships with your regional societies, there may be some fantastic longitudinal data that could be collected to supplement or inform your work. 6. Government subsidies: absent in our work, but present in the EU The Artist Revenue Streams project included questions about grants and commissions, but the US has essentially no direct government support, tax breaks or subsidies for musicians (or artists of any discipline). We know that some EU countries offer subsidies or government support, to the point that it can become a significant source of money for some artists. Researchers in the EU will need to take these structures into consideration and include questions that capture musicians interactions with these systems. We encourage interested researchers to review the entire post-project protocol to understand how we structured the research initiative. We are also happy to discuss strategies that can improve the collection of comparative data. Kristin Thomson kristin@futureofmusic.org Jean Cook jean@futureofmusic.org 5