Here’s a little bit of news which may have escaped notice by some in the information theory community:
“In view of its concerns about excessive reviewing delays in the IT Transactions, the BoG authorizes the EiC in his sole judgment to delay publication of papers by authors who are derelict in their reviewing duties.”
Reviewers may be considered to be derelict if they habitually decline or fail to respond to review requests, or accept but then drop out; or if they habitually submit perfunctory and/or excessively delayed reviews.
I’ve said it before, but I think that transparency is the thing which will make reviews more timely — what is the distribution of review times? What is the relationship between paper length and review length? Plots like these may shock people and also give a perspective on their own behavior. I bet some people who are “part of the problem” don’t even realize that they are part of the problem.
Wait, isn’t this coming from a journal that gets its content (from authors) for free and also has reviewers work for free to provide quality assessments of said content?
Yep, but review times are atrociously long — it can take 3 years to get a paper through the review cycle. So IF it’s the case that a few (or many) bad seeds are, say, submitting tons of papers and shirking all reviews, then perhaps punishing those people can help rectify the system. All of those people would have to be very derelict, however.
However, I am very dubious that the long review times are due to this — I imagine it’s more of a culture of moderate delay (e.g. a few months late) and moderate shirking (doing 2 reviews for each paper submitted) that causes the whole system to be gummed up.
But then again, there’s not a whole lot of transparency about the cause of the problem (or perhaps getting the statistics out of the database would be onerous and awful).
I suppose the journals could also dispatch knuckle-dragging thugs to beat the living shit out of referees who are more than a few days late, or send them to Siberian death camps, or cut off their publishing privileges, or do something else to inspire fear in people who prioritize things like grant proposals (which allow them to run their groups) over referee reports (which, while vital to the scientific enterprise as a whole, serve only to exploit their labor and expertise for free). Or, I suppose the journals could adopt a system that rewards referees for their service, such as allowing good referees to have their articles made available for download without fees, or providing expedited review for their own papers. I can’t tall you how tired I am of having journals and editors treat me like shit when I submit papers and then expect me to do a bunch of review reports for them promptly for free, and also demand large fees for the content I provide them to publish things that can only be read once additional large fees.
Expedited review is the flip side of delaying the reviews of bad actors, which is the current proposal. Honestly though, what is needed in IT is to get people to shoulder their fair share for the good of the community. At least that’s what it seems like…
I think it was Anant Sahai who suggested that a lot of the delay in review times may be the time between when a reviewer agrees to review a paper and when they actually start working on it. Theoretically deadlines could be much shorter, if editors only gave papers to reviewers who had time to start working on the review immediately. That would require a lot more organization and communication between editors and potential reviewers than exists now, of course. And, as you say, without data it’s hard to say how much impact this could have.
It’s from a paper he wrote with Parv (in Allerton). I have heard anecdotal evidence that people get review requests and neither accept nor decline them in a timely fashion, leading to delay in even setting the reviewers.
But all of this is anecdote. What we want is numbers, no?
I write this all as someone who is behind on a review — but it’s not my fault! The apartment robbers stole my bag which had my notes on the paper!
Giving more referees both a short time to accept/decline and a short review timeline does sound like a way to expedite review, but great care would have to be taken to avoid collapsing the delicate system of goodwill and altruism the current system runs on. As someone who received a request to referee a paper that was then retracted less than twenty-four hours later, I can tell you that this does not lead a referee to be inclined to ever review a paper for that journal again.
The Transactions on Signal Processing give a review period of 6 weeks, which is somewhat reasonable and I think they have brought down times significantly. The difficulty with doing this for IT is that the papers are much longer and much more technical, so checking all of the proofs etc etc just *does* take longer.
The delicate system of goodwill works as long as you review at least one paper for every paper you submit. I say at least because there are a lot of rejected papers whose authors may not review those papers. This, of course works in a small academic community (e.g. information theory) where there is basically one “home” journal. In this setting there’s a little less of the “journal is exploiting my labor” feeling because it’s a community.
If there’s really such a small community and only one journal of note, and such drastic changes are being considered anyway, why not just migrate the community to an open-access journal with immediate publication and an open, extended review process? That way, publication is immediate, referee delay is irrelevant, and the open review/commentary procedure adds value over time and allows continued discussion and improvement while also gaining the benefit of preventing self-serving referees from hiding behind the mask of anonymity?
Well, there are other journals for certain sub-topics, but not ones broadly on information theory — the issue is that the community is really built around the journal (and the IEEE) and I don’t think people want to “split” the community by having a tiered system. Even if we made an OA journal, it would still be deemed “tier 2” by most members.
In any case, I’m too junior a guy to have any impact on this process, but I think having a second OA journal would be great.