Scientific and technical foundation for altmetrics in the US

Transcription

1 Scientific and technical foundation for altmetrics in the US William Gunn, Ph.D. Head of Academic Outreach

2 Why altmetrics? data repositories 19 funder policies 16 data journals New forms of scholarship need new metrics.

3 Problems with Impact Factor

4 Problems with Impact Factor Country Documents Citable documents Citations Self- Citations Citations per Document H index United States 7,063,329 6,672, ,540,193 62,480, ,380 China 2,680,395 2,655,272 11,253,119 6,127, United Kingdom 1,918,650 1,763,766 31,393,290 7,513, Germany 1,782,920 1,704,566 25,848,738 6,852, Japan 1,776,473 1,734,289 20,347,377 6,073, France 1,283,370 1,229,376 17,870,597 4,151, Canada 993, ,493 15,696,168 3,050, Italy 959, ,701 12,719,572 2,976, Spain 759, ,452 8,688,942 2,212, India 750, ,232 4,528,302 1,585,

5 Problems with Impact Factor During discussions with Thomson Scientific over which article types in PLoS Medicine the company deems as citable, it became clear that the process of determining a journal's impact factor is unscientific and arbitrary.

6 There is no correlation between the number a citations an article receives and the impact factor of the journal. The higher the impact factor, the more likely the research is to be retracted, partly due to intense competition. What matters is who is reading your work!

7 Adams, Jonathan. "Collaborations: the fourth age of research." Nature (2013):

8 King, Christopher (2012) Thomson Reuters Annual Report

9 What are altmetrics? Research impacts more than authors

10

11 Citations are slow

12 Research is fast

13 Readership vs. citations it comes with a payload of metadata it accrues faster it illuminates previously hidden impact

14 Install Mendeley Desktop...and aggregates research data in the cloud Mendeley extracts research data Collecting rich signals from domain experts.

15 Defining readership Each document addition is a read stamped with metadata describing the context of the read event a read is like a citation, but faster and captures more

16

17 Professors on Mendeley tend to be in applied math, stats, and physics. Graduate students on Mendeley tend to be in engineering disciplines.

18 Cell Biology and Neuroscience are highly active disciplines, relative to their output. Social sciences are highly active relative to their citations/paper

19 altmetrics show broader impact

20 Issues To Be Addressed Identity Privacy Attribution Gaming standards/ best practice Filtration

21 Consistency is key

22 NISO Altmetrics Standards Alfred P. Sloan Foundation American Library Association California Institute of Technology Center for Research Libraries EBSCO Elsevier Harvard Internet Archive Wiley Library and Information Technology Association Library of Congress Los Alamos National Laboratory National Institutes of Health National Library of Medicine OCLC Princeton Columbia Smithsonian Stanford

23 NISO Altmetrics Standards Types of sources: Mendeley, Twitter, Views, Downloads, Github Quality of sources: collection, reporting, aggregation methods; provenance; availability Use cases: discovery and assessment (of people and objects)

24 Types of sources Most altmetrics providers use the following: Page views or downloads Mendeley readers (articles only for now) Tweets Comments:Blog posts, Pubmed Commons Github

25 Quality of sources Collection methods vary & counts are inconsistent. Further study is needed. For reporting, transparency is key. Show raw data, not just a derived number. Aggregation of raw data is generally done by the recipient (institution, funder, publisher, author, etc) according to their need, instead of using one central source.

26 Quality of sources Understanding and open reporting of provenance is important for community buy-in and long term stability. Raw data should be available under open license, via API, with identifiers. Identifiers include DOI:object, ORCID:person, ISNI/Ringgold: institution Ex. This person(orcid), at this institution (ISNI), released this object (DOI).

27 Use cases Two main use cases exist: Discovery and Evaluation Because the data sources remain variable, discovery can be done now. Accuracy of numbers matters less in recommendation than in assessment. Precision important for both.

28 Next Steps NISO white paper will be in public comment period soon. Working Groups will be established to develop best practices and standards. Pending approval, NISO will issue recommended practice or published standard. NISO to develop training to implement and adopt any recommended standards.

29 There is no gold standard Amgen: 47 of 53 landmark oncology publications could not be reproduced Bayer: 43 of 67 oncology & cardiovascular projects were based on contradictory results Dr. John Ioannidis: 432 publications purporting sex differences in hypertension, multiple sclerosis, or lung cancer. Only one data set was reproducible

30

31 Cloud Library Home Work Mobile

32 Shared Folder

33 Mendeley Research Catalog

34 Mendeley Research Catalog

35 Read papers + keep track of notes 470M documents

36 Taking some misery out of writing

37 Information Extraction

38 We are publishing this data to the LOD cloud