I listened to the episode on peer review the other day and felt frustrated through big portions of it because there were some critical points about peer review and research articles not stated. For reference, I’m a PhD candidate at a US university in computer science studying robotics, and a Lecturer in my department as well. I’ve been working in robotics research for the last 6-7 years.
- Research articles are written for other researchers, not for the general public.
A researcher’s job is not to explain things to the general public. Their job is to produce new knowledge in their field. They do this by developing new algorithms, ideas, math, etc related to their field, and then writing up the results. Generally academic papers in STEM fields have two goals: 1) inform other people in the field about the new work, and 2) ensure that other researchers can re-produce the results. That second goal means the audience of these articles are other experts in the fields. Communicating technically to other experts in your field is going to be much different than communicating with the general public. In order to ensure other people can re-produce your results, you have to speak technically.
At some point, one host expressed frustration at how they were trained to use jargon during their doctoral training. This is part of doctoral training because jargon is used to communicate with other experts in your field. Learning how to use jargon is essentially learning how to communicate technically about your field. When I write papers and talk to other roboticists, I use jargon all the time. I use jargon so that my message is communicated as clear as possible for other roboticists. However, my lab has visitors all the time that range from high schoolers to business executives. The visitors don’t care about the details of my work - they are only interested in the high-level ideas so there is no need to speak technically. For instance, in robotics the technical term for a robot arm with a hand attached is a ‘manipulator’. So when I write papers, I say manipulator when I’m referring to a manipulator. However, when I talk to visitors I just say arm. For roboticists, the distinction between a robot arm and a robot manipulator is significant. For non-roboticists, the distinction is usually not significant so I just say arm to make things simple.
At another point, the hosts said that professors are rewarded for becoming more inaccessible. While this is technically true, it frames professors and academia is an unfair way. The vast majority of professors are researchers first and foremost. If you go read up on the academic industry, then you will quickly learn that a professor’s job responsibilities are balanced between research, teaching, and service work. Some schools weight each category differently, but nearly all schools still hold research above the other two. Professors gain tenure by doing good research. Good research is done by writing lots of good papers, and good papers are full of jargon. The jargon isn’t there to be inaccessible. The jargon exists to communicate technically with other researchers in their field. A massive portion of a professor’s job is not to maximize how many people can understand their work - it is to maximize how many experts in their field can understand and use their research. Doing that means they have to use jargon. And if they do good research, then they are rewarded. So they are not rewarded for becoming more inaccessible. They are rewarded for being good at their job.
- Research papers can be submitted more than once.
A paper being rejected does not mean that paper or work can never be published anywhere. When a paper is reviewed, the output of the review is 1) a decision about acceptance and 2) comments from each reviewer about the paper. The comments state what the reviewer thinks the paper is about (summarizing it), whether or not it is novel research, any shortcomings of the work, and how the paper can be improved. Common reasons to reject a paper are:
a) It’s not actually new.
b) It needs more experiments.
c) The experiments are not convincing.
After a paper is rejected, the author(s) can modify the paper to make it better based on the reviews they received, and then re-submit it to some other conference, journal, or sometimes even the same journal. Then the review process is repeated. After modifying the paper based on previous reviews, the paper is hopefully better and it may be accepted on a second submission. If so, then great. If not, then a new round of reviews are given to the authors and they can continue to revise and improve the paper. It is very rare for a paper to be accepted on the first submission, and it is not uncommon to go through 3 or more reviews before a paper is finally accepted somewhere. When a paper is rejected, journals and conferences don’t say “this is bad and you should feel bad”. They say “this is not good enough right now and here is how to make it better”. Sometimes papers are pretty bad and the reviews can be harsh, but the reviews still point out all the things that need to change about the paper for it to be published.
In theory, peer review doesn’t sound that great due to only a few people reviewing a submission each time and your paper’s publication depends on only a few people. However, in practice it works out pretty well. Each rejection and revision makes a paper better. If you get rejected, then you just revise the paper and submit again. If you’ve done good work, then your paper will eventually get through. Youtube and other social media platforms, on the other hand, sound amazing in theory - they’re platforms where anyone can post information and the audience can decide on the validity of it through comments, likes, etc. However, in practice we’ve seen that it has led to the rise of fake news, echo chambers for divisive rhetoric, and even terrorist recruitment.
I point this out because almost any time I hear this argument about how peer review is too gate-keepery, people only talk about the theory. And in theory, peer review does sound kind of gate-keeperish and social media sounds like a utopian invention. However, no one ever mentions what the practice of these things are. In practice, peer review is a pretty good system (though not perfect) and social media can be pretty scary. The peer review system is a well-oiled machine that publishes thousands of scientific papers each year and the authors are senior researchers, first year graduate students, and everyone in between. It is certainly not perfect, as Katie pointed out numerous times, but I definitely think it is a force for good.
Please keep in mind that I only say all of this because I want the podcast to be the best it can be. I’m actually interested in transitioning into game-based education research after I graduate and I’m broadly interested in any game-based research, such as games for mental health. I really wanted to point all of this out because I felt like the peer review process and research training process were mischaracterized during this episode by both hosts.
The last point I want to make is that formal research (scientific journals, conferences, etc.) and informal publication platforms (Youtube, Twitter, etc.) can co-exist. It doesn’t have to be one or the other, and one doesn’t have to be better than the other. Communicating really technical ideas to the general public is a different skill than being a researcher. They are similar, but different. There are some amazing people on Youtube that run math and science education channels (Welch Labs, Minute Physics, etc) that break down incredibly complex topics for the general public to understand, but that job is quite different than being a researcher. Similarly, there are some amazing people developing new medicine and social theories, but that job is quite different than making informative and digestible Youtube videos. The two types of platforms can collaborate and work together. I think a very interesting conversation can be had about how the types of platforms support one another.
