Should early career scientists be judged by their publication record? The new format of Great Debate took place at this year's General Assembly. (Credit: Lena Noack)
The EGU’s General Assemblies have a long tradition of Great Debates – sessions of Union-wide interest which aim to discuss some of the greatest challenges faced by our discipline. Past topics have included exploitation of mineral resources at the sea bed, water security given an ever growing population and climate geoengineering, to name but a few. This year’s meeting saw the first Great Debate aimed, specifically, at an Early Career Scientist (ECS) audience which boasted an innovative format too: Should early career scientists be judged by their publication record? A set of group debates. Today’s post, written by Mathew Stiller-Reeve, a convener of the session, summarises some of the main outcomes of the discussion.
We, early career scientists, are told that we need to become expert writers, presenters, and teachers if we are going to make it in the world of research. Many of us agree such transferrable skills are extremely important. But if we invest time in developing these skills, it sometimes feels like time wasted. All said and done, we only seem to be judged on our publication record and our h-index. How many papers have we published in high impact journals, and how often have they been cited?
Early career scientists seem very clued up on transferrable skills. They want to invest in these skills. Therefore, we wanted to hear from them about whether ‘early career scientists [should] be judged mainly on their publication record?’ And so we put this question to them (and others) at a Great Debate at the EGU’s 2017 General Assembly. We also wanted to test out a new format where the audience had the opportunity to voice their opinions about important issues concerning modern academia. The publication issue affects us all, so we should have a say.
With only 8 people at each table and over 40 minutes to debate, everyone had an opportunity to speak their mind and contribute to developing solutions. The room was buzzing with over 100 early career and more established scientists discussing, agreeing, disagreeing, and finding compromises.
In the end, each table was tasked to debate and boil their thoughts down to one or two policy-type statements. These statements will be presented to the EGU Council to inform them of where EGU early career scientists stand on this matter.
So without further ado, here are the conclusions of the tables:
– We need more criteria. Quality is most important, measured by prizes, PhD results and the incorporation of the community via new media.
-More activities need to be taken into account in a measurable way, but according to scaled categories #notjustanumber.
-The current system is cheap, easy and fast. A person should be judged on the broader contributions to society, to their colleagues, to their disciplines. We should move beyond metrics.
-Because scientists are more than a list of publications, assess them individually. Talk to them and read their output, including publications, blogs and chapter/book contributions.
-We should not be judged on publication record alone. We need a multi-variant set of criteria for assessment for judgment of impact beyond just academic publications.
-One suggestion is a weighted metric depending on the position you’re applying for which considers other factors such as teaching, outreach, conference participation etc.
-No, the h-index should not be the sole number, even though it is not a totally useless number.
-Quality should be judged on more than quantity and the large number of authors on publications devaluates the contributions of early career scientists.
-Publications are the accepted way of communication in science, but there is not any one number describing the quality of the early career scientist, whom in our humble opinion should not only be judged on the quantity of papers but also on their quality as a part of a complete set of research skills, including other contributions such as project development.
-We acknowledge the publication record as a reliable metric, but we suggest an additional step for assessing applications, based on video or audio presentations to emphasize your other outstanding qualities.
-We doubt that we are mainly judged on our publication record and we think that publications should be part of what we are judged on.
-When hiring, follow the example of the Medical Department at Utrecht University: only ask for the 3 papers, teaching or outreach experiences you think are important for the position you are applying for: we are more than numbers.
Should they be adopted? Do you agree? How can we adopt them?
The message in many of the statements from the Early Career Scientists at the European Geosciences Union is quite clear: We are more than numbers! Several suggestions arose from the debate: new metrics, video presentations, and even new application processes. Now the statements from the debate are recorded. This will hopefully inspire us (and others) to find better solutions. At the very least, the discussion has begun. Solutions are impossible if we don’t talk!
By Mathew Stiller-Reeve, co-founder of ClimateSnack and researcher at Bjerknes Centre for Climate Research, Norway
Editor’s note: This is a guest blog post that expresses the opinion of its author and those who participated at the Great Debate during the General Assembly, whose views may differ from those of the European Geosciences Union. We hope the post can serve to generate discussion and a civilised debate amongst our readers.