A bit of context first. This letter arose as a result of discussions regarding citations practices in meta-analyses on Twitter several weeks ago. We all agreed citations of work within meta-analyses should be counted, but noted that many modern citation counting services disregard any citations not in the main body of a paper. Appendix citations are dropped, however. And due to a variety of restrictions, we often only cite the papers we use as data in appendices, and so are not counted. Our discussion resulted in the following letter that sixty-eight scientists around the globe have signed. I look forward to further fruitful discussion with the organizations above (who have been contacted) and hope that this situation is corrected.

Update 3: Despite also hearing from SCOPUS that this was being forwarded on to their product manager, no word back. Also, no word from GS, despite this being sent directly to members of their dev team by common contacts.

Dear Staff & Support Teams,

We are writing as a community of researchers in ecology, evolution and conservation biology who rely on the citation metrics you provide. We love your service, and, frankly, depend on it for a wide variety of reasons (e.g., identifying research trends, finding relevant papers, evaluating the influence of papers/researchers). However, in one respect your counts are becoming increasingly inaccurate. We have come to realize that citations – sometimes hundreds of them – that we have made in the appendices of our own published papers are not being counted by your service. We felt obligated to bring this issue to your attention, as it affects the accuracy and reliability of your statistics.

We are a group of researchers who conduct meta-analyses: analyses of published data, typically gathered by extracting data from published scientific papers. The standard practice in our and many other fields is to include citations to works that we use for data – but in the case of meta-analyses in particular, these citations do not necessarily appear in the main reference section of our papers. As meta-analyses regularly get their data from 50+ papers, and journals often have strict space limits and limits on the number of references, the citations are often included in supplementary material, most of which are online only (SOMs). These appendices are subject to the same level of peer review as the core article.

As far as we understand, citations in SOMs are not recorded by you. Hence, the papers we cite do not receive the proper citation credit, and the citation lists you provide for our own papers are incomplete. The failure to consider references in SOMs – particularly in the case of meta-analyses – severely reduces the accuracy of the citation counts you present for each paper. It is also poses a huge ethical problem for us because it means that we do not give proper credit, in the form of a citation, to the many superb scientists whose high-quality work is the jumping-off point for our own analyses. In turn, these colleagues of ours may be less inclined to be charitable when we come asking for data contributions to future meta-analyses.

We ask that you consider including citations garnered from supplementary online materials in academic journals.

If there is any way that we can facilitate this by contacting journals and editors in our own fields, we are more than happy to help. We feel very strongly about correcting this problem, and hope it can lead to some tangible benefits for both you and countless numbers of scientists.

Thank you, and we look forward to engaging in a productive dialogue about this issue.

Signed,

Jarrett Byrnes
Department of Biology
University of Massachusetts Boston
Boston, MA 02125

Ross Mounce
Department of Biology & Biochemistry
University of Bath
Bath, BA2 7AY
UK

Alexander Bond
Department of Biology
University of Saskatchewan
Saskatoon, Saskatchewan, S7N 5E2
Canada

John Griffin
Department of Biosciences
Swansea University
Swansea, Wales, SA28PP
UK

Mark Anthony Browne
National Center for Ecological Analysis & Synthesis
University of California, Santa Barbara
735 State Street, Suite 300
Santa Barbara, CA 93101-3351

Gavin Simpson
Institute of Environmental Change and Society
University of Regina
3737 Wascana Parkway,
Regina, SK S4S 0A2
Canada

Michael P. Taylor
Department of Earth Sciences
University of Bristol
Bristol BS8 1RJ
UK

Julia Stewart
National Center for Ecological Analysis & Synthesis
University of California, Santa Barbara
735 State Street, Suite 300
Santa Barbara, CA 93101-3351

Mary Hunsicker
National Center for Ecological Analysis & Synthesis
University of California, Santa Barbara
735 State Street, Suite 300
Santa Barbara, CA 93101-3351

Robert Lanfear
Ecology, Evolution, and Genetics
Australian National University
116 Daley Road, Acton
Canberra, ACT 2602
Australia

Ethan White
Department of Biology and The Ecology Center
Utah State University
5305 Old Main Hill
Logan, UT 84341
United States

Morgan Ernest
Department of Biology and The Ecology Center
Utah State University
5305 Old Main Hill
Logan, UT 84341
United States

Noam Ross
Department of Environmental Science and Policy
University of California at Davis
1 Shields Avenue
Davis, CA 95616
twitter: @noamross

Trevor A. Branch
School of Aquatic and Fishery Sciences
University of Washington
Seattle, WA, 98195
United States
twitter: @TrevorABranch

Philip Martin
Centre for Ecology & Hydrology,
Wallingford,
Oxfordshire.
OX10 8BB.
UK.
twitter:@_PhilMartin

Kyle Edwards
Kellogg Biological Station
Michigan State University
3700 E. Gull Lake Dr.
Hickory Corners, MI 49060

Jonathan S. Lefcheck
Virginia Institute of Marine Science
The College of William & Mary
Gloucester Point, VA 23062

Terry McGlynn
Department of Biology
California State University Dominguez Hills
1000 E. Victoria St.
Carson, CA 90747
USA

Karthik Ram
Museum of Vertebrate Zoology
University of California, Berkeley
Berkeley, CA 94720
USA

Daniel Falster
Biological Sciences,
Macquarie University NSW 2109,
Australia

Bradley Cardinale
School of Natural Resources & Environment
University of Michigan
Ann Arbor, MI 48103

Ignasi Bartomeus
Postdoctoral Researcher
Department of Ecology,
Swedish University of Agricultural Sciences,
SE-75007 Uppsala, Sweden
@ibartomeus

Sean Tuck
Department of Plant Sciences
University of Oxford
South Parks Road
OX1 3RB

Helen Phillips
Department of Ecology and Evolution
Imperial College London
Silwood Park Campus
SL5 7PY

Mark Westoby
Dept of Biological Sciences
Macquarie University, Sydney NSW 2109 Australia

Thomas White
Dept of Biological Sciences
Macquarie University, Sydney, NSW 2109
Australia

Gregory Carey
School of Biological and Chemical Sciences
Queen Mary, University of London
London
E1 4NS

Adam Algar
School of Geography
University of Nottingham
Nottingham NG7 2RE
UK

J. Emmett Duffy
Virginia Institute of Marine Science
The College of William and Mary
Gloucester Point, VA 23062-1346

Eduardo S. A. Santos
Departamento de Ecologia
Universidade de São Paulo
Rua do Matão, 321 – Trav. 14, sala 243
Cid. Universitária – São Paulo, SP
05508-090
Brazil

Lauri Laanisto
Institute of Agricultural and Environmental Sciences
Estonian University of Life Sciences
Kreutzwaldi 5, 51014, Tartu
Estonia

Emilio M. Bruna
Department of Wildlife Ecology and Conservation &
Center for Latin American Studies
University of Florida
Gainesville, FL 32611-0430

Julian A. Velasco
Laboratorio de Análisis Espaciales
Instituto de Zoología
Universidad Nacional Autonóma de México
México, D.F. CP. 04510
Phone: (55) 5622-8222 ext: 47880

Timothée E. Poisot
Theoretical Ecology Group
Université du Quebec a Rimouski
Quebec Centre for Biodiversity Sciences
Rimouski, QC, Canada

Steven J. Cooke
Biology Department
Carleton University
Ottawa, ON, K1S 5B6, Canada

Mick Watson
Director of ARK-Genomics
The University of Edinburgh
Edinburgh, UK
EH25 9RG

Xiao Xiao
Department of Biology and The Ecology Center
Utah State University
5305 Old Main Hill
Logan, UT 84341
United States

Joshua King
Biology Department
University of Central Florida
Orlando, FL 32816

Andrew D. Steen
Department of Microbiology
University of Tennessee Knoxville, TN 37996

Chris Harrod
School of Biological Sciences
Queen’s University, Belfast
BT9 7BL
UK

Allen Hurlbert
Department of Biology
University of North Carolina
Chapel Hill, NC 27599-3280

Mike S. Fowler
Department of Biosciences
Swansea University
Swansea, SA2 8PP
UK

Dieter Lukas
Department of Zoology
University of Cambridge
Cambridge, CB23EJ
UK

Kate Boersma
Department of Zoology
Oregon State University
Corvallis, OR 97331

Ramón E. Martínez-Grimaldo
Instituto de Biología
Universidad Nacional Autónoma de México

Fernando T. Maestre
Universidad Rey Juan Carlos
Departamento de Biología y Geología
Escuela Superior de Ciencias Experimentales y Tecnología
C/ Tulipán s/n, Móstoles, 28933
SPAIN

Andrew B. Cooper
School of Resource and Environmental Management
Simon Fraser University
Burnaby, British Columbia, CANADA V5A 1S6

Josef C. Uyeda
Institute for Bioinformatics and Evolutionary Studies
University of Idaho
Moscow, Idaho 83844

Elita Baldridge
Department of Biology and The Ecology Center
Utah State University
5305 Old Main Hill
Logan, UT 84341
United States

Franciska T. de Vries
Faculty of Life Sciences
Michael Smith Building
The University of Manchester
Oxford Road
Manchester
M13 9PT
United Kingdom

Aidan M. Keith
Centre for Ecology & Hydrology
Lancaster Environment Centre
Library Avenue
Bailrigg, Lancaster
LA1 4AP, United Kingdom
@Aidan_M_Keith

Christophe Thebaud
UMR “Evolution & Biological Diversity”
CNRS & Univ Paul Sabatier
F-31062 Toulouse
France

Rafael D. Zenni
Department of Ecology and Evolutionary Biology
University of Tennessee
Knoxville, TN
USA

Natalie Cooper
School of Natural Sciences
Trinity College Dublin
Dublin 2, Ireland

Philippe Desjardins-Proulx
Université du Québec à Montréal
Quebec Centre for Biodiversity Sciences
Rimouski, Québec
Canada

Lars Gamfeldt
Deparment of Biological and Environmental Sciences
University of Gothenburg
Gothenburg, Sweden

Mark Hahnel
figshare
4 Crinan Street
London
N1 9XW
UK

Jessica Couture
National Center for Ecological Analysis & Synthesis
University of California, Santa Barbara
735 State Street, Suite 300
Santa Barbara, CA 93101-3351

Jarrod Cusens
Institute for Applied Ecology New Zealand
Auckland University of Technology
31-33 Symonds Street
Auckland
New Zealand

Steph Borrelle
Institute for Applied Ecology New Zealand
Auckland University of Technology
31-33 Symonds Street
Auckland
New Zealand

Nick Isaac
NERC Centre for Ecology & Hydrology
Wallingford OX10 9QA
UK

Sally Keith
ARC Centre of Excellence for Coral Reef Studies
James Cook University
Townsville, QLD 4811
Australia

Sarah Supp
Department of Ecology and Evolution
Stony Brook University
650 Life Sciences Building
Stony Brook, NY 11794-5245

Jessica Blois
School of Natural Sciences
5200 N. Lake Rd.
Merced, CA 95343
USA

Mike Whitfield
Botany, School of Natural Sciences
Trinity College
College Green
Dublin 2
Ireland

Richard J. Butler
School of Geography, Earth and Environmental Sciences
University of Birmingham
Edgbaston
Birmingham, B15 2TT

Jens Kattge
Max Planck Institute for Biogeochemistry
Hans Knoell Str. 10
07745 Jena
Germany

Scott Chamberlain
Biology Department
Simon Fraser University
Burnaby, BC, Canada V5A 1S6

Blaire Steven
Bioscience Department
Los Alamos National Laboratory
Los Alamos, NM, USA, 87545

Santiago Soliveres
Universidad Rey Juan Carlos
Departamento de Biología y Geología
Escuela Superior de Ciencias Experimentales y Tecnología
C/ Tulipán s/n, Móstoles, 28933
SPAIN

Francis Brearley
School of Science and the Environment
Manchester Metropolitan University
Chester Street
Manchester
M1 5GD
UK

Michiel van Breugel
Smithsonian Tropical research Institute
Av. Rossevelt 401
Balboa, Ancon
Panama, Panama

Dylan James Craven, PhD.
Synthesis Centre for BioDiversity Sciences (sDiv)
German Centre for Integrative Biodiversity Research (iDiv)
Leipzig, Germany

x-posted from openpub

One year ago, one of the more intereting experiments in open access publishing – PeerJ – launched. It’s model of membership rather than paying by the article is still something people are grappling with – it’s just so different, so delightfully disruptive. Not only that, but PeerJ has stepped in to fill the void in providing a biology preprint server (which we have used), as well as coming up with a more intuitive interface for commentary on preprints and published work – along with supplying reviews alongside published work. A number of other great open access journals have tried one or more of these innovations, but few have tried them all *at once*. Not only that, but I have the feeling they’re not going to stop there.

After all, if they’ve tried this many new things in year one, I, for one, want to know what year two is going to hold…

So, I recently submitted a piece regarding the future of scholarly publishing in Ecology & Evolutionary Biology. Simultaneous to posting, I put up a preprint in PeerJ Preprints and also put it on Google Docs for line by line commentary (which you are welcome to give!). I asked in both places that commenters identify themselves, unless they felt deeply uncomfortable.

OMG the experience has been amazing!

At PeerJ you can comment on the main page of the article, and others can rate it – which is fantastic – and I’ve gotten some wonderful feedback there (thanks Lars!)

The Google Doc experience has been even more fascinating, given the ability to put in line by line comments.

One of our reviewers is using the Google Doc for their comments. It has made it easy to see what they are saying, respond to things that I think are relevant (or I’ll just change some of the text in the next draft for bigger changes), and have an interactive experience with the reviewer. It absolutely fabulous.

I’ve been really fascinated by the idea of how collaboration can improve peer review ever since reading Leek et al.’s 2011 piece Cooperation between Referees and Authors Increases Peer Review Accuracy. I’m delighted that one of our reviewers has embraced that ethos, and in so doing, I can see how this will really help with future publications if not just Ross Mounce, but everyone embraced this model. Very cool!

x-post from the OpenPub Project blog

So, we got together, had two working group meetings to discuss the future of scholarly publishing in Ecology, Evolutionary Biology, and the Earth and Ocean Sciences. What were were thinking that entire time?

We’ve just submitted a piece that brings together our broad ideas (some of which have been seen before), but, simultaneous to publication, we’ve also decided to put up a preprint. Why? Simply put, immediate access is one of our four pillars of the future of scholarly publication. Once you feel something is ready for public consumption, put it out there! We’ve been delighted to watch the evolution of PeerJ Preprints, so we’ve placed our piece there.

Byrnes et al. (2013) The four pillars of scholarly publishing: The future and a foundation. PeerJ PrePrints 1:e11 dx.doi.org/10.7287/peerj.preprints.11

This immediate access to the piece goes hand in hand with another of our four pillars. Open Review. We want to know what you think. And now. We hope you give us feedback over at the preprint. Or, if you want to give us more detailed annotated comment, we’ve put it in a comment-open Google doc. Highlight something you disagree with. Argue with us. We welcome it! We’d ask that you put your name with the comment. We want a discussion, as discussion will improve this manuscript and help us shape our argument rather than just one-way commenting. This will also allow *you* to get full recognition for your comments, and we will include this in future acknowledgements.

So, enjoy the piece – our commentary is not a straight experiment-analysis-discussion piece, but rather part of a broader ecosystem of scholarly products that we feel are important to get out there. We look forward to hearing what you think of the piece!

As part of the OpenPub project, we’re soliciting folk to send us videos about their experience with the scholarly publication process. We want to use these to try and crowdfund the development of OpenPub – our preprint server with robust tools for discussion and interaction. Interested? Check out the full request over here and/or email me!

After the fantastic NCEAS working group on the future of scholarly publishing in EEB (big success – stay tuned for more news on that front!), I participated in a really interesting online discussion about Post-Publication peer review with Carl Boettiger, Jamie Ashlander, and Scott Chamberlain. The conversation was great, and I think brought up a lot of interesting points – both that echoed some discussions in our group and went far beyond them.

There’s a lot of grist for the mill here – what is the roll of post-publication peer review, do we have an efficient system to conduct it, can we do better, and if so, how (and how can we do it simply instead of having to build some whole new platform)?

Take a gander, and I’d love to hear more of your thoughts about how we can better accelerate the pace of the scientific discussion in an efficient way.

Friday’s NCEAS discussion about changing the future of scientific publishing was fascinating. We had a wide variety of views about the current system, what it’s adding to our papers, if the problems really are problems, and why we got here. But there was one thing that came up again and again, that even the most skeptical status quo folk in the crowd agreed was where the true value of the current journal system lies –

Editorial peer review.

Not a single person in the audience denied that their papers were not hugely improved by the peer review process, as well as the editor’s comments and revisions that came with it.

And that was it, really. Any other model you please was deemed a non-starter by many if it didn’t include editorial peer review. The one thing which generally we all do as part of service (except at higher EIC levels – usually). There was some other talk of niceties – professional layout, etc., but it was also pointed out that much of that we can now do ourselves on our desktop.

So that would seem to be the gauntlet thrown. If you want to change the system, the new system still has to involve strong peer review – guided by editors or guided by the crowd or whatnot was not discussed (but, interesting to ponder). Without it, solutions will be viewed with skepticism or even derision (gauging by reactions). But with it, there is room for a viable alternative. Food for thought.

Today at The National Center for Ecological Analysis and Synthesis we’re having an open discussion about the future of scholarly publishing. I may post some notes from it later, but, Stephanie Pau and I have compiled a list of thought questions and helpful links to help folk prep. And, as it was an interesting gathering of linkage, I thought it mide be useful more broadly. So, below is the text of my email and the useful links. I realize that I am blogging my email. Is this a new low? Perhaps. Enjoy!

Hello, all! For today’s open house, we’ll be talking about the growing hubub regarding academic publishing and the relationship between scientists and publishers. Is it time for a change? What is the way forward?

This is a big topic, and not all of you may be aware of the things that have gone down over the past few months that are bringing this to a head. As such, Steph and I have put together a list of links that give you an introduction to all of this as well as some questions to bear in mind while perusing them. Don’t worry, they’re all pretty quick reads – short blog posts or an Xtranormal video which is brilliant.

Also exciting, we’re going to have a few guests who are involved in scholarly publishing from campus (and maybe beyond).

Marty Einhorn (KITP) for perspective on arxiv.org and the physics community’s take on this
Josh Schimel (EEMB) who is on the ESA publications committee
Chuck Bazerman (Education) a consultant for hypothes.is/

See you at 4pm in the lounge. And, as the weather has been gorgeous, if we want to continue the conversation after 5, Tony Ray’s?

Questions to Ponder

1. Do we need a change our model of scientific publishing? Why?

2. What needs to change?

3. Are scientists/ecologists ready for a change? Are we too conservative or slow to adopt change in general, open access in particular? I.e., good in theory, doesn’t work in practice? (www.nytimes.com/2012/01/17/science/open-science-challenges-journal-tradition-with-web-collaboration.html?ref=science)

4. What are the differences between commercial (e.g., Elsevier) and non-profit journals (e.g., ESA) that affect the exchange of scientific information (bjoern.brembs.net/comment-n820.html)? There does not appear to be a difference in quality as measured by the number of citations (octavia.zoology.washington.edu/publishing/ecology_citationprice.html)

5. Do commercial journals offer us something that non-profit journals do not? Prestige? What about differences between ESA and PloS (high costs put on author) models?

6. If the exchange of information is better served by open access, should we refuse to review for commercial journals?

7. Is there a difference between exchange of information for the sake of the discipline and personal academic success?

Resources on Open Access
oad.simmons.edu/oadwiki/Social_media_sites_about_OA

The Issue at Hand
An Introduction with some Humor
www.youtube.com/watch?v=GMIY_4t-DR0

The Research Works, H.R. 3699 Act & Responses from Scientists
thomas.loc.gov/cgi-bin/query/z?c112:H.R.3699:

www.michaeleisen.org/blog/?p=807

blogs.scientificamerican.com/evo-eco-lab/2012/01/06/scientists-fight-for-access/

osc.hul.harvard.edu/stp-rfi-response-january-2012

www.monbiot.com/2011/08/29/the-lairds-of-learning/

www.guardian.co.uk/commentisfree/2011/aug/29/academic-publishers-murdoch-socialist

ESA’s statement on Open Access back in Jan (that some on Ecolog-L were not too happy about)
www.esa.org/esablog/ecology-in-policy/esa-policy-news-january-13-2/

A Pledge to Not Publish in Elsevier Journals (e.g., TREE) with a lot of folk signing on
thecostofknowledge.com/

Further Meditations
Academic Spring?
www.slate.com/articles/technology/future_tense/2012/02/federal_research_public_access_act_the_research_works_act_and_the_open_access_movement_.html

Comments from Michael Hochberg
https://sites.google.com/site/perspectivesinpublishing/our-mission

Do publishers add value? Nature says yes.
www.nature.com/nature/journal/v481/n7382/full/481409a.html">

www.nature.com/nature/journal/v481/n7382/full/481409a.html

www.michaeleisen.org/blog/?p=873

Publishers need us more than we need them
dougbelshaw.com/blog/2012/01/21/you-need-us-more-than-we-need-you/#.TyAbbuNSSWU

http:/