“No one can read everything. We rely on filters to make sense of the scholarly literature, but the narrow, traditional filters are being swamped. However, the growth of new, online scholarly tools allows us to make new filters; these altmetrics reflect the broad, rapid impact of scholarship in this burgeoning ecosystem. We call for more tools and research based on altmetrics.”
Research metrics are nothing if not controversial, none more so than the “alternative” variety currently in vogue from Plum Analytics, ImpactStory and the donut people themselves altmetric.com. For the most part, the twitterati are buying into them in a big way, not to mention Big Publishers and Open Access advocates, though some influential voices are dismissive. At best. See this title from David Colquhoun and Andrew Plested “Why you should ignore altmetrics and other bibliometric nightmares“. Or even read it – it’s hard to deny that most folk who retweet the monster-altmetric articles probably don’t (personally I eschew the RT without having read an article, whether academic or pop-cultural trivia. Maybe that’s just me*)
* I’ll admit occasionally to having retweeted *before* reading but only if I trust the source.
Though a mere bibliometric tinkerer myself, I rather take umbrage on behalf of professional bibliometricians everywhere who are somewhat maligned in this article: “The mistake made by all bibliometricians is that they fail to consider the content of papers, because they have no desire to understand research.” This arguably needs a citation and at least one cohort of bibliometricians to take an IQ test to establish if bibliometrics really are “for people who aren’t prepared to take the time (or lack the mental capacity) to evaluate research by reading about it.” I can’t necessarily back up with a citeable source myself but I’m willing to bet that the very first thing one learns as a bibliometrician is the strengths and, often profound, weaknesses of the various metrics, whether JIF, h-index or alternative. To dismiss their value entirely seems to rather throw the baby out with the bathwater.
Not that I don’t have reservations – the colour coded donut of altmetric.com surrounding a number arguably lacks clarity and I’m not entirely certain what the number actually refers to, though it seems to correlate vaguely with tweets (coloured blue in the donut) which is invariably the highest “metric” – and arguably the least valuable – there is presumably some weighting towards potentially more valuable, less ephemeral activity on blogs (yellow) and in mainstream news outlets (dark red). I’m not convinced Mendeley and CiteULike necessarily reflect representative networks across all disciplines (dark red and light blue respectively) or Facebook for that matter (dark blue)…however, this is surely the point, the network, and while altmetrics do perhaps pander to a human propensity to quantify with raw numbers – though no more than traditional citations – their main strength is undoubtedly to visualise and leverage the network; to connect with peers online, or with interested laypersons, to foster community and promote research activity and impact beyond the walls of the ivory tower.
N.B. I’ve just picked up this article from Twitter – Numbers behind Numbers: The Altmetric Score and Sources Explained…but haven’t read it yet!
Donuts can be embedded easily using code provided at http://api.altmetric.com/embeds.html and the API can be accessed free of charge, with various paid options for more sophisticated functionality, though with a little knowhow you can derive useful data without a subscription to the premium service. There is also a plugin available from the EPrints Bazaar
Below are the top 10 articles by altmetric for repository downloads in April 2015 as recorded by IRUS-UK and derived from 84 UK repositories and a total of more than 87,000 unique items. Data is presented with a link to the primary repository and highlights numbers of total downloads* from this, and any other repositories where the paper is archived (*i.e. all downloads from each repository, not just April). There is a DOI lookup link and a note whether the article is available to download from the publisher’s site (N.B. only 3 of these 10 are restricted access with 5 under CC-By and 2 freely accessible albeit © All Rights Reserved).
No further analysis is offered other than to say that there does not appear to be any obvious correlation between altmetrics and repository downloads – number 1 is obviously related to a major story in the media about a certain King found in a Leicester carpark – but, it bears reiterating, that is not necessarily the point; rather repositories are an embedded, institutionally managed element of researchers’ online network with tools like IRUS and altmetrics enabling data to be visualised and leveraged across an increasingly coherent infrastructure.
N.B. Information on how this data was derived using the export function from IRUS-UK and a Google Sheet will follow in a subsequent post.
- King, T.E., Fortes, G.G., Balaresque, P., Thomas, M.G., Balding, D., Delser, P.M., Neumann, R., Parson, W., Knapp, M., Walsh, S., Tonasso, L., Holt, J., Kayser, M., Appleby, J., Forster, P., Ekserdjian, D., Hofreiter, M., Schürer, K. (2014) Identification of the remains of King Richard III. Nature Communications, 5, art. no. 5631
DOI lookup (Available OA from publisher: CC-BY)
Total downloads from Leicester Research Archive to April 2015: 132 - Sumner, Petroc, Vivian-Griffiths, Solveiga, Boivin, Jacky, Williams, Andrew James, Venetis, C. A., Davies, Aimee, Ogden, Jack, Whelan, Leanne, Hughes, Bethan, Dalton, Bethan, Boy, Frederic and Chambers, Christopher D. 2014. The association between exaggeration in health related science news and academic press releases: retrospective observational study. BMJ 349, g7015
DOI lookup (Available OA from publisher: CC-BY)
Total downloads from Online Research @ Cardiff (ORCA) to April 2015: 42 - ATLAS collaboration (2012) Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC. Physics Letters B, Volume 716 (Number 1). pp. 1-29. ISSN 0370-2693
DOI lookup (Available OA from publisher: CC-BY)
Total downloads from Lancaster EPrints to April 2015: 29
Total downloads from WRAP: Warwick Research Archive Portal to April 2015: 16 - RTS,S Clinical Trials Partnership; (2011) First results of phase 3 trial of RTS,S/AS01 malaria vaccine in African children. The New England journal of medicine, 365 (20). pp. 1863-75. ISSN 0028-4793
DOI lookup (Freely available from publisher: © Massachusetts Medical Society)
Total downloads from LSHTM Research Online to April 2015: 695 - Smith, O., Momber, G., Bates, R., Garwood, P., Fitch, S., Pallen, M., Gaffney, V., Allaby, RG. (2015) Sedimentary DNA from a submerged site reveals wheat in the British Isles 8000 years ago. Science 347 (6225) pp. 998-1001
DOI lookup (Access restricted from publisher site)
Total downloads from Research@St Andrews to April 2015: 11 - Hobaiter, C., & Byrne, R.W., (2014) The meanings of chimpanzee gestures Current Biology 24(14) pp. 1596-1600
DOI lookup (Access restricted from publisher site)
Total downloads from Research@St Andrews to April 2015: 245 - Darby, SC, Ewertz, M, McGale, P, Bennet, AM, Blom-Goldman, U, Bronnum, D, Correa, C, Cutter, D, Gagliardi, G, Gigante, B, Jensen, M-B, Nisbet, A, Peto, R, Rahimi, K, Taylor, C and Hall, P (2013) Risk of Ischemic Heart Disease in Women after Radiotherapy for Breast Cancer NEW ENGLAND JOURNAL OF MEDICINE, 368 (11). 987 – 998. ISSN 0028-4793
DOI lookup (Freely available from publisher Copyright © 2013 Massachusetts Medical Society. All rights reserved.)
Total downloads from Surrey Research Insight Open Access to April 2015: 368 - Otto, A, Otto, FEL, Allen, MR, Boucher, O, Church, J, Hegerl, G, Forster, PM, Gillett, NP, Gregory, J, Johnson, GC, Knutti, R, Lohmann, U, Lewis, N, Marotzke, J, Stevens, B, Myhre, G and Shindell, D (2013) Energy budget constraints on climate response. Nature Geoscience 6 (6). 415 – 416. ISSN 1752-0894
DOI lookup (Access restricted from publisher site)
Total downloads from White Rose Research Online to April 2015: 517 - Costa, Marta D, Pereira, Joana B, Pala, Maria, Fernandes, Veronica, Olivien, Anna, Achilli, Alessandro, Perego, Ugo A., Rychkov, Sergei, Naumova, Oksana, Hatina, Jiri, Woodward, Scott R., Eng, Ken Khong, Macaulay, Vincent, Carr, Martin, Soares, Pedro, Pereira, Luísa and Richards, Martin B. (2013) A substantial prehistoric European ancestry amongst Ashkenazi maternal lineages. Nature Communications 4 (2543). ISSN 2041-1723
DOI lookup (Available OA from publisher: CC-BY)
Total downloads from The University of Huddersfield Repository to April 2015: 104 - Wood, Michael J. and Douglas, Karen (2013) “What about building 7?” A social psychological study of online discussion of 9/11 conspiracy theories. Frontiers in Psychology 4 (N/A). p. 409. ISSN 1664-1078.
DOI lookup (Available OA from publisher: CC-BY)
Total downloads from Kent Academic Repository (KAR) to April 2015: 1244
1 thought on “Network effects: on alternative metrics”