Why journal paper reviewing needs to change

Having written columns for newspapers and had letters published in them, the ethos around the press has carried through into the way I deal with academic publications. The former editor of the Pontypridd Observer, Wayne Nowaczyk said what he printed was based on whether it was ‘interesting’ and not simply whether it was written well – as being the editor these things could be changed.

I now edit Crocels News, am a director of a publishing company, The Crocels Press Limited, and edit books and special issues for publishers like IGI Global and Springer. For me the most important factor is not the academic rigour pe se, but whether the paper is one that is interesting so that a lot of people would want to read and cite it. It should be more important for an editor to be pleasing their readers than their reviewers in other words.

In this article I will discuss a recent review of one of my papers, which I felt would be really interesting to the journal’s readers. I have quoted from the editor and the reviewers throughout this article. But to make it more anonymous I changed the order of the sentences and put them through a translator a number of times. This was the opinion of the editor of the journal:

I sent manuscript to three reviewers. Comments manuscript has been received by three. In comments attached to e-mail, you see recommend not publish work. Based on comments, I decide reject manuscript for publication in journal.

To get a researcher paper into press and for it to count towards one’s academic record it needs to be peer-reviewed. This means that the research article in question has been either scrutinised by an editor, or people the editor asks to give their opinion, or both. My attitude to reviewers on publications I am the editor of is; “I am the editor, it doesn’t matter what anyone says, if I think the paper is interesting I will have it.

This is not the view of most people in academia – few of them have a background in journalism. Most of them have their books and journals published by established publishing houses, and they never have to deal with the business side of things. So all they care about is whether something ticks the boxes for rigour someone would need to on a PhD and things such as relevance and timeliness are less important. Below is one of the things a reviewer said about my paper:

This version much better to draft. Most important weakness is assess reliability. The paper essentially develop scale. Scale development need robust methodology, single factor analysis and Cronbach Alpha test can hardly considered thoroughly.

Firstly the reviewer has no clue what they are talking about – they have probably never used factor analysis! The method I applied was based on an assignment I did on my research methods Masters, accredited by the Economic and Social Research Council, which I got a good merit in. When I was at Kingston University, I learned from an expert in factor analysis – Professor Stavlos Kalafatis, and he said a Chronbach Alpha was one of the best options for testing a scale’s reliability – the other being using a Structural Equation Modelling output. It was up to me the research to decide which method I used, and it should not be for someone else to hold up publication of my paper because they have a different point of view, when the choice of method is equally rigourous and used by leading Professors like Stavlos Kalafatis.

Even so, for me, whilst it is important for research papers to be accurate and reliable, factors such as sample size, whether the papers used probability or non-probability, whether or not it is possible or impossible to generalise qualitative or quantitative research are all academic. It is all very well to have to demonstrate an awareness of these during one’s viva for a PhD, but otherwise they can get in the way of interesting research. Here is one of the comments from a reviewer of this paper:

Many allegations are not set correctly. You identify values not clearly explained. The contribution to discussion and conclusion are poor. The author not specified research questions, not importance of the subject, and not  inform rest of the article. How I choose questions? How people selected? The literature difficult to understand. Paper difficult to read. Paper presents alternative model and is empirically rigorous. They representative sample? Need to better explain preliminary analysis not clear.

This reviewer sounds like they are teaching a research methods course, and that if things don’t follow a prescribed model it is not good enough for them. What does it matter that research questions are not stated? That might be fine for an university assignment, but in the real world it doesn’t matter. Every teacher knows you’re supposed to, in theory, but the learning objectives on the screen at the start of the lesson, but most don’t because it doesn’t work in practice.

Here is a comment of one of the reviewers after I had changed the paper to make in more relevant using their feedback.

The authors have an important contribution to revise the document based on the feedback. However, data collection and development on a scale not sufficiently rigorous. The entire document is much more structured. I would not recommend paper.

From my point of view, it is more important that the author in question gives my readers what they want, rather than please reviewers like these who are likely to not be experts on the topic they are asked to review. This is why now for the publications I edit that I ask the authors to name three reviewers to look at their paper. Some might say they will pick people who are biased – but it does not matter. As the editor it is me that decides whether a paper is published and not the reviewers!

Leave a Reply

Your email address will not be published. Required fields are marked *