We asked four peer reviewers to share their experiences of being part of the EPSRC Peer Review College: Jan Evans-Freeman, Mick Steeper, Peter Ward and Professor Iain Thayne.
Jan Evans-Freeman is Pro Vice Chancellor, College of Engineering, University of Canterbury, New Zealand. She is a current member of the EPSRC Peer Review College.
Q: What are the benefits of taking part in the peer review process? And what is the value to you as an academic based in New Zealand?
A: To me, as the overall academic and research leader of an engineering college, the value is in assessing the relevance of the work we do here in New Zealand in relation to leading-edge research in Europe and the UK. Although national needs can differ, global requirements are common to all, and it is essential to me to see how international researchers address common themes.
Every research proposal that comes my way offers insight, either into how research is organised in the UK, or how researchers are addressing hot topics.
Q: Describe the work that you do on the panel.
A: This is connected to my first answer, and has to do with me retaining a global perspective on research, research impacts and research outcomes.
Q: How much time commitment do you put into the college?
A: About three proposals to review per year, each one takes several hours.
Q: What attracted you to becoming a peer reviewer?
A: I am always grateful to receive a well-thought-out and informed review of my own work, and am pleased to try to do that for other people.
Q: Do you have a message for researchers based outside the UK who may be thinking of applying to the call?
A: If you want to understand how another research economy organises itself, what is important to them, and what the best and emerging researchers are working on, be an EPSRC reviewer!
Mick Steeper has been a peer reviewer for over four years, a college member for six years and is currently on the Manufacturing the Future strategic advisory team. During this time he has worked at Siemens and Primetals Technologies in a research and development (R&D) group specialising in metals processing technology, and was responsible for external R&D liaison with universities, research training officers and partner companies.
Q: What are the benefits of taking part in the peer review process? And what is the value to your business?
A: You learn so much, even in the fields you thought you already knew well. Research proposals (the good ones, at least) are inherently novel, and reviewing them will give you a new way of looking at their subject. For the businesses that employ reviewers, they can expect energised employees who’ve been exposed to novel ideas.
Q: What insights have you gained?
A: More than once, my peer review activity has been the source of first awareness of researchers who have gone on to become key partners in my employer’s R&D. I’ve also improved my all-round knowledge of multiple science and engineering fields, and in my own specialist field I have found both previously unknown research activity and early recognition of new concepts and trends.
Q: How much time commitment do you put into the college?
A: For each proposal I consider, I aim to give it at least two hours, though the more intriguing ones sometimes get longer. If my other work commitments allow it, I aim to read a proposal through a couple of days before working intensively on its review. So much of research is synergistic, and it takes time to consider the connections.
Q: What attracted you to being a peer reviewer?
A: My initial motivation was pretty altruistic, the opportunity to contribute something in return to my knowledge community. I didn’t realise before I started how much I would learn. With hindsight, that’s the greatest benefit of all.
Q: Do you have a message for other businesses who may be thinking of applying to the call?
A: The EPSRC’s Peer Review College is an expert community of scientists and engineers, and any of your employees joining it will have the opportunity to access a network of experts in their field, and your business field.
The EPSRC’s domain of pre-competitive research is implicitly collaborative. The peer review college is a door to cooperative research opportunities.
Interview with Peter Ward and Professor Iain Thayne
Peter Ward: My name’s Peter Ward and I come from an industrial background. I’ve been in the semiconductor industry for 30 years, worked for several international companies and, at the moment, I’m an independent consulting engineer.
I’ve been involved with EPSRC Peer Review College for many years actually, I suppose the last stint is about six or seven years but it’s been off and on as I’ve been available in the UK.
Professor Iain Thayne: My name is Professor Iain Thayne. I’m from the University of Glasgow and I lead the Ultra-fast Systems Group in the Electronics Department. I’ve been associated with the college for the last six years.
Peter Ward: Well I’m not what you might call a conventional college member because the majority of them are academics but, for me, I enjoy being a member of the college because it gives me an insight into the academic community. Perhaps it also gives me a chance to put something into the academic community which they wouldn’t have otherwise.
Professor Iain Thayne: I think there’s definitely a policy in EPSRC for devolving some of the decision-making processes to the community so it’s very important to be involved in that decision-making process. As a consequence of that, to be a member of the college gives you that opportunity, partly in reviewing proposals when you see them and, secondly, to actually be part of the decision-making process in panel meetings.
Peter Ward: The referees’ reports are fundamental to the panel because that’s the starting point. Without good referees’ reports, the panel simply can’t function. Apart from anything else, the panel itself can’t hope to cover the huge breadth of subjects which come up in front of it, so we have to rely on the expertise of the referees.
Professor Iain Thayne: The panel decision-making process is quite an interesting one. Basically, when a proposal is submitted to EPSRC, the first thing that happens is that it goes out to external reviewers for assessment. And then, providing that those reviews are sufficiently positive, then those proposals are combined and a panel is formed. In the panel process itself you’re not in a situation where you’re actually re-refereeing the proposal, you’re directed not to do that but rather to take on board the views of the reviewers comments and then to try to make a value judgement on the quality of the proposal based on those assessments. The interesting aspects come when you’re trying to rank the various proposals, because often there are a very, very large number of very, very high quality and fundable applications and the issue is then how you actually go about doing that.
Peter Ward: The single biggest problem with the referees’ reports is when they don’t actually write anything, they just tick the boxes. The key thing in referees’ reports is they really need to explain why they think a proposal is good, or why they think it’s bad. It’s really just not good enough to say excellent, excellent, excellent, outstanding, outstanding, and sign your name at the bottom, because that doesn’t give the panel anything to work with.
Professor Iain Thayne: The job of the panel is to take often disparate proposals from different subject areas and try to come to some ranked conclusion on which areas and which projects should be funded. Regrettably, there’s never enough money to support everything. But you’re working very, very strongly on the comments from the external reviewers in this process. The panel is not at liberty to re-referee the proposal and the key things, the pieces of information that you’re using to make the decision are the reviewers comments, plus the responses of the investigators to the reviewers comments. So actually the responses that you make as a principal investigator, having got your referees comments back is actually very, very important, because it can make a huge difference to whether the proposal ultimately is ranked highly or not.
Peter Ward: The EPSRC website is very helpful and useful. You can find all the information you need. What I tend to do is ring people up, and generally EPSRC people are extremely helpful, extremely obliging, and they help you out if you have a problem with something. So I would advise people just to talk to the guys here in Swindon because they’re very knowledgeable and very helpful.
Professor Iain Thayne: The key thing for referees is that they put a lot of narrative into their reports. It’s no good to just tick the boxes, we need to have lots of words there, so you can actually make a value judgement on what this person really believes, because having a digital response on something being good, adequate, outstanding or whatever, really doesn’t give full justice to the scope of the proposal, so we need very, very strong narrative.
Professor Iain Thayne: Managing college work, it’s part of the job, so it just has to be done. You can go through a number of months where you don’t receive any proposals to review, and then suddenly a number will arrive on your desk at the same time and, often, it can be challenging to find the time to give due opportunity to do justice to various proposals. I would say you probably need between one to two hours to look at each proposal properly to give it a rigorous scrutiny. And then for panel meetings themselves, well you get a lot of documentation, maybe two weeks before the proposal, before the panel review meeting, and I would say on average you probably require between 12 to 15 hours of preparation work, plus the day at Swindon to actually go through the whole process. So in some regards it’s fairly arduous, but as I said at the outset, it’s very, very important that the whole process is properly dealt with, and people do allocate enough time to ensure that fairness is achieved.
Professor Iain Thayne: I think the support and the administrative aspects that EPSRC provides are very, very strong. So the documentation is usually very full and we’re given a good steer and good guidance and so on, so I think actually they have to be complimented on that.
Peter Ward: I think the single piece of advice I’d give to anybody who was thinking of joining the college or is approached to join the college, is to enjoy it. Firstly, because it is a very interesting, challenging and fun thing to do and also because you see stuff that you would never normally look at in a month of Sundays. It really does open your eyes to what’s going on around you, and sometimes it can be quite intellectually challenging to actually have to read up on something that is a million miles away from your speciality.
Professor Iain Thayne: I think for new college members the key aspect is that you should be aware that, although you’re having to devote some time and effort to this, you do actually benefit from it in many ways. It gives you better insight into the way in which you should be writing proposals if you want to be successful. And, at the end of the day, maybe having that inside track if you like, is a very useful thing for your own activities.