GeoLog

peer review

Enmeshed in the gears of publishing – lessons from working as a young editor

Enmeshed in the gears of publishing – lessons from working as a young editor

Editors of scientific journals play an important role in the process research publication. They act as the midpoint between authors and reviewers, and set the direction of a given journal. However, for an early career scientist like me (I only defended my PhD in early December 2016) the intricacies of editorial work remained somewhat mysterious. Many academic journals tend to appoint established, more senior scientists to these roles, and while most scientists interact with editors regularly their role is not commonly taught to more junior researchers. I was fortunate to get the chance to work, short term, as an associate editor at Nature Geoscience in the first 4 months of this year (2017). During that time, I learned a number of lessons about scientific publishing that I felt could be valuable to the community at large.

What does an editor actually do?

The role of the editor is often hidden to readers; in both paywalled and open-access journals the notes and thoughts editors make on submitted manuscripts are generally kept private. One of the first things to appreciate is that editors judge whether a manuscript meets a set of editorial thresholds that would make it appropriate for the journal in question, rather than whether the study is correctly designed or the results are robust. I’d argue most editors are looking for a balance of an advance beyond existing literature and the level of interest a manuscript offers for their audience.

At each step of the publication process, from initial submission, through judging referee comments, to making a final decision, the editor is making a judgement whether the manuscript still meets those editorial thresholds.

The vast majority of the papers I got the chance to read were pretty fascinating, but since the journal I was working for is targeted at the whole Earth science community some of these were a bit too esoteric, and as such didn’t fit the thresholds we set to appeal to the journal audience.

I actually found judging papers on the basis of editorial thresholds refreshing – in our capacity as peer reviewers, most scientists are naturally sceptical of methodology and conclusions in other studies, but as an editor in most cases I was able to take the authors conclusions at face-value, and leave the critical assessment to referees.

That’s where the important difference lies; even though editors are generally scientists by training, since they are naturally not experts in every field that they receive papers from, it’s paramount to find reviewers who have the appropriate expertise and to ask them the right set of questions. In journals with academic editors, the editors may have more leeway to make critical comments, but impartiality is key.

Much of this may be already clear to many readers, but perhaps less so to more junior scientists. Many of the editorial decisions are somewhat subjective, like gauging the level of interest to a journal audience.

In the context of open access research journals, I think it’s worth asking whether the editorial decisions should also be made openly readable by authors and referees – this might aid potential authors in deciding how to pitch their articles to a given journal. This feeds into my next point – what are journals looking for?

By which metrics do journals judge studies?
The second big thing I picked up is that the amount of work does not always equate to a paper being appropriate for a given journal. Invariably, authors have clearly worked hard, and it’s often really tricky to explain to authors that their study is not a good fit for the journal you’re working for.

Speaking somewhat cynically, journals run for profit are interested in articles that can sell more copies or subscriptions. Since the audiences are primarily scientists, “scientific significance” will be a dominant consideration, but Nature and subsidiary journals also directly compare the mainstream media coverage of some of their articles with that of Science – that competition is important to their business.

Many other authors have discussed the relative merits of “prestige” journals (including Nobel prize winners – https://www.theguardian.com/science/2013/dec/09/nobel-winner-boycott-science-journals), and all I’ll add here is what strikes me most is that ‘number of grad student hours worked’ is often not related to those articles that would be of a broader interest to the more mainstream media. The majority of articles don’t attract media attention of course, but I’d also argue that “scientific significance” is not strongly linked to the amount of time that goes into each study.

In the long run, high quality science tends to ensure a strong readership of any journal, but in my experience as an editor the quality of science in submitted manuscripts tends to be universally strong – the scientific method is followed, conclusions are robust, but in some cases they’re just pitched at the wrong audience. I’d argue this is why some studies have found in meta-analysis that in the majority of cases, articles that are initially rejected are later accepted in journals of similar ‘prestige’ (Weller et al. 2001, Moore et al. 2017).

As such, it’s imperative that authors tailor their manuscripts to the appropriate audience. Editors from every journal are picking from the same pool of peer reviewers, and so the quality of reviews should also be consistent, which ultimately determines the robustness of a study; so to meet editorial thresholds, prospective authors should think about who is reading the journal.
It’s certainly a fine line to walk – studies that are confirmatory of prior work tend to attract fewer readers, and as such editors may be less inclined to take an interest, but these are nonetheless important for the scientific canon.

In my short time as an editor I certainly didn’t see a way around these problems, but it was eye-opening to see the gears of the publication system – the machine from within, as it were.

Who gets to review?
One of the most time-consuming jobs of an editor is finding referees for manuscripts. It generally takes as long, if not far longer, than reading the manuscript in detail!

The ideal set of referees should first have the required set of expertise to properly assess the paper in question, and then beyond that be representative of the field at large. Moreover, they need to have no conflict of interest with the authors of the paper. There are an awful lot of scientists working in the world at the moment, but in some sub-fields it can be pretty hard to find individuals who fit all these categories.

For example, some studies in smaller research fields with a large number of senior co-authors often unintentionally rule out vast swathes of their colleagues as referees, simply because they have collaborated extensively.

Ironically, working with everyone in your field leaves no-one left to review your work! I have no doubt that the vast majority of scientists would be able to referee a colleagues work impartially, but striving for truly impartial review should be an aim of an editor.

As mentioned above, finding referees who represent the field is also important. More senior scientists have a greater range of experience, but tend to have less time available to review, while junior researchers can often provide more in-depth reviews of specific aspects. Referees from a range of geographic locations help provide diversity of opinion, as well as a fair balance in terms of gender.

It was certainly informative to compare the diversity of authors with the diversity of the referees they recommended, who in general tend to be more male dominated and more US-centric than the authors themselves.

A positive way of looking at this might be that this represents a diversifying Earth science community; recommended referees tend to be more established scientists, so greater author diversity might represent a changing demographic. On the other hand, it’s certainly worth bearing in mind that since reviewing is increasingly becoming a metric by which scientists themselves are judged, recommending referees who are more diverse is a way of encouraging a more varied and open community.

What’s the job like?
Editorial work is definitely rewarding – I certainly felt part of the scientific process, and providing a service to authors and the readership community is the main remit of the job.

I got to read a lot of interesting science from a range of different places, and worked with some highly motivated people. It’s a steep learning curve, and tends to be consistently busy; papers are always coming in, so there’s always a need to keep working.

Perhaps I’m biased, but I’d also suggest that scientists could work as editors at almost any stage in their careers, and it offers a neat place between the world of academia and science communication, which I found fascinating.

By Robert Emberson, freelance science writer

References

Moore, S., Neylon, C., Eve, M. P., O’Donnell, D. P., and Pattinson, D. 2017. “Excellence R Us”: university research and the fetishisation of excellence. Palgrave Communications, 3, 16105

Weller A.C. 2001 Editorial Peer Review: Its Strengths and Weaknesses. Information Today: Medford NJ

All you ever wanted to know about EGU publications

All you ever wanted to know about EGU publications

Did you know that, the EGU, through Copernicus Publications, publishes 17 peer-reviewed open-access journals? The journals cover a range of topics within the Earth, planetary and space sciences: with publications spanning the cryospheric sciences, soil system sciences, through to non-linear processes in geophysics, there is something for everyone. Whatever your area of research, chances are you’ll be represented within the range of EGU publications!

Better still, the EGU is a signatory of the Berlin Declaration. This means we believe that scientific literature should be publicly available and free of charge. Anyone wishing to read, download, copy, distribute, search or print research findings is able to do so without encountering any financial, legal or technical barriers. Authors of research articles are fully protected, too! They retain full copyright for their work via the Creative Commons Attribution License, which requires that full credit for any distribution of the research is given and any changes made to figures and or/data is highlighted, too.

Most EGU Publications also extend the traditional peer-review process by applying the Interactive Public Peer Review system. This means that a manuscript is subjected to two stages of review. The figure below helps to illustrate the process.

Two-stage public peer review as practised in the scientific journal Climate of the Past (CP) and its discussion forum Climate of the Past Discussions (CPD). 1. Submission; 2. Access review; 3. Technical corrections; 4. Publication as Discussion paper; 5. Comments; 6. Final response; 7.Post-discussion editor decision; 8. Revisions; 9. Peer-review completion; 10. Final revised publication.

Two-stage public peer review as practised in the scientific journal Climate of the Past (CP) and its discussion forum Climate of the Past Discussions (CPD). 1. Submission; 2. Access review; 3. Technical corrections; 4. Publication as Discussion paper; 5. Comments; 6. Final response; 7.Post-discussion editor decision; 8. Revisions; 9. Peer-review completion; 10. Final revised publication.

In the first stage, the manuscript undergoes a rapid pre-screening and is immediately published as a ‘discussion paper’, in the journal discussion forum. During the next eight weeks or so, the paper is reviewed by the referees, as well as the scientific community. Referees and other scientists can leave comments which are published alongside the paper. The referee’s comments can be anonymous, or signed, whilst the public comments are always signed. Authors can actively participate in the discussion by clarifying remarks and offering further details to those reading the discussion paper.

The second stage of review follows: if the editor is satisfied with the author’s responses to the comments, the manuscript can be accepted for publication. If the editor still has some concerns about the publication, further revisions will be carried out until a final decision is reached. If necessary, the editor may also consult referees in the same way as during the completion of a traditional peer-review process. In order to increase transparency, some journals also publish a report that documents all changes to the paper since the end of the public discussion.

The system offers advantages to the authors, referees, editors and even the reader. The publication of the ‘discussion paper’ means that research is rapidly disseminated. Added to which, the interactive peer review and discussion means that authors receive feedback directly and can participate in the discussion. The final published research undergoes a full peer-review process, in addition to comments from other scientists, assuring the quality of the research, that is published in EGU journals.

On average, it takes approximately 200 days for a manuscript to complete its journey from submission to publication. However, this time can vary from journal to journal and manuscript to manuscript. This video, produced by our publisher Copernicus, shows the review times for various EGU Journals. Not only that, the average length of time the manuscript spends at each of the stages from submission to publication is broken down, too.

Maybe next time you come to publish your research findings you’ll consider submitting your manuscript to one of the EGU journals. You can learn more about the EGU publications by following this link. To submit your manuscript, head over to the website of any of the EGU journals, and look for the author guidelines and resources for reviewers.

Some food for thought to finish off this post: Have you ever considered the global journey a manuscript goes on after it is submitted? Using an article from Atmospheric Chemistry and Physics, Copernicus produced a video tracking its globetrotting journey: from its birth in Norway and collaborations in eight different countries, to its editor in Switzerland and referees spanning Europe and Asia, the global impact of this manuscript is truly remarkable.

Did you know you can follow many of the EGU journals on Twitter, too? With links to useful journal information, highlight and discussion papers, the social media platform provides a quick way to keep up to speed with the journals. Please follow this link to find out which journals are on Twitter.

Do you have any questions about EGU journals that were not answered in this post? Get in touch through the comments below.

References

Pöschl, U.: Multi-stage open peer review: scientific evaluation integrating the strengths of traditional peer review with the virtues of transparency and self-regulation, Frontiers in Computational Neuroscience, 6, 33, 1-16, doi:10.3389/fncom.2012.00033, 2012.

GeoEd: Under review

In this month’s GeoEd column, Sam Illingworth tells us about how teaching undergraduate students about peer review can help eliminate bad practice.

To anybody other than a researcher, the words peer review might seem like a fancy new age management technique, but to scientists it is either the last bastion of defence against the dark arts or an unnecessary evil that purports to ruin our greatest and most significant works.

According to Wikipedia (itself a fine proponent), peer review is defined as “the evaluation of work by one or more people of similar competence to the producers of the work (peers). It constitutes a form of self-regulation by qualified members of a profession within the relevant field.”

Peer review itself is not a new concept; the first documented description of a peer-review process, being found in the ‘Ethics of the Physician’ by Ishap bin Ali Al Rahwi (854–931), states that the notes of physicians were examined by their contemporaries to assess if treatment had been performed according to the expected standards (you can read more on the history of the peer-review process in this article).

Even the great Carl Sagan found the critique of his work difficult to stomach (Photo credit: NASA JPL, via Wikimedia Commons).

Even the great Carl Sagan found the critique of his work difficult to stomach (Photo credit: NASA JPL, via Wikimedia Commons).

“Why do we put up with it? Do we like to be criticized? No, no scientist enjoys it.” So sayeth American cosmologist and author Carl Sagan about the ‘joys’ of peer review, in his book ‘The Demon-Haunted World: Science as a Candle in the Dark.’ He goes on to say that “Every scientist feels a proprietary affection for his or her ideas and findings. Even so… the hard but just rule is that if the ideas don’t work, you must throw them away.”

Just reading these words brings me out in the kind of cold sweat that I normally associate with seeing the bill from mechanic, after having your car serviced. You know that you are going to have to bite the bullet, but in your heart of hearts you just wish that it weren’t so.

Love it or loathe it the peer-review system is an integral part of being a researcher, and given its prevalence it is strange that for many scientists the whole notion of it is a completely alien concept until they first encounter the publication process during their postgraduate studies.

During the first year of my PhD I remember being aghast at the notion that two, or possibly three, strangers would be wholly responsible for deciding whether or not my research was deemed ‘suitable’ for publication, and despite my otherwise excellent undergraduate education I had nothing to prepare me for the whole ordeal. Thankfully I had a very experienced supervisor who was able to guide me through the whole process and teach me a few tricks of the trade (always respond politely, compliment the reviewer for their suggestions, avoid the urge to break down into tears and instead break the comments down into manageable chunks), but even now I still feel a sense of dread when an email notification appears in my inbox telling me that “the reviewer’s comments have been posted.”

Is this how reviewers are perceived? (Photo credit: deviantArt)

Is this how reviewers are perceived? (Photo credit: deviantArt)

By nature I am quite a defensive person, and have been known to take criticism (fair or otherwise) rather to heart, but my experiences of the peer review system have certainly helped me take a more level–headed and professional approach to the critique of my work. Crucially it has also helped me to become a better reviewer myself.

Constructive criticism is essential in order to help one develop as a researcher, and indeed as an individual, but some of the peer reviews that I have seen (and sadly been subjected to) are nothing more than mean-spirited attempts by the reviewer to assert their own supposed authority on a subject. This kind of analysis is beneficial to absolutely no one, and it should be the responsibility of the editors and administrative staff of the journals and e-zines to help eradicate it. There is always something positive to be said about any piece of research (unless it is utterly nonsensical, in which case again the editor should have stopped it from ever being submitted to a reviewer), and being totally negative in your comments will only serve as fuel for a vicious cycle in which young researchers believe that the purpose of peer review is to find fault in the work of others. Instead, good peer review should be a helpful critique of a fellow colleagues work, which politely points out any shortcomings, makes suggestions for improvements, and praises what is good.

I will now be teaching my own university students about the peer-review system, and will be asking them to mark one another’s work throughout the unit that I teach on Science Communication at Manchester Metropolitan University. I think that most undergraduate courses would benefit from a similar approach, not only to prepare future scientists, but also to help students learn how to respond to criticism and how to critique the work of others in a productive and conducive manner. By educating and encouraging young scientists in this way we can hope to potentially avoid these kinds of reactions in the future.

Teaching about peer review at university can help to eliminate bad practice (Photo credit: Gideon Burton).

Teaching about peer review at university can help to eliminate bad practice (Photo credit: Gideon Burton).

For those of you who are currently reviewing a paper, I set you the challenge of explicitly writing at least one compliment to the author. This could be in regards to the excellence or originality of their research, the structure or fluidity of the article, or indeed the clarity with which they express their ideas. To those of you who are not reviewing a paper, try and find at least one positive thing to say (the colour really brings out your eyes, it’s certainly an affordable mode of transport, these scones are delicious!) the next time that your opinion is required; I guarantee that it will leave everyone feeling just a little bit more capable of themselves and what they can achieve.

 By Sam Illingworth, Lecturer, Manchester Metropolitan University

Read all about it! The latest on EGU journals

The last month has been a big one for the EGU’s publications, with a new journal in the pipeline, another adopting interactive peer review and a new addition to Web of Science. Here’s the latest…

Soil

Say hello to SOIL

We will be launching a new interactive, open access journal at the EGU 2014 General Assembly. SOIL is dedicated to the publication and discussion of high-quality research in the field of soil system sciences. It will open for submissions in May 2014, following the journal’s official launch at EGU 2014.

Find out more about SOIL on the EGU website and take sneak peek at SOIL over at www.soil-journal.net.

 

NPG_cover

 

Nonlinear Processes in Geophysics becomes interactive
Nonlinear Processes in Geophysics (NPG), is transitioning from an open access journal with a traditional review process into an interactive open access journal that uses public peer-review and interactive public discussion. Find out more about this new peer review process here.

 

 

ESD cover

Earth System Dynamics indexed in ISI Web of Science

Last but not least, one of our open access journals, Earth System Dynamics (ESD), is to be included in the Web of Science/ISI listings, following the com­pletion of their assessment of the quality, characteristics, and flow of papers published in the journal since its launch in 2010! This is terrific news and highlights the tremendous work of the editorial board and the scientific community in submitting so many excellent articles to ESD. Over the next few months all ESD papers will be added to the listings.

Stay up-to-date with EGU news at www.egu.eu/news/announcements.