Juking the stats in academic publishing

I heard recently of a case where someone got a paper back with revisions requested, and a deadline for said revisions. They ended up asking for a week extension, but then the journal said they would have to do a fresh submission and redo the whole review cycle. I found this baffling — but then that person pointed out that the journal has built a reputation on fast turnaround times, and so to keep their “sub-to-pub” numbers low, they don’t want to give any extensions to the authors. It’s better to do a resubmission than to continue with the same “paper ID” in the system.

This is a classic example of juking the stats:

I just got a rejection from KDD 2012 which smacks of the same ominous reasoning:

We try to notify authors once a decision on a submission is concretely made, and hope that the early notifications can reduce the average review turn-over time.

But the real kicker is that “due to technical constraints” they can’t give us the reviews until May 4th. So I’m not really sure what I am supposed to do with this information — I can’t really start on revisions without the reviews, so this “early notification” thing is really just to make them feel better about themselves, it seems. Or perhaps they can then report that the reviewing was “more efficient.”

In any case, no harm is done, per se. But optimizing metrics like “sub-to-pub” seems to be as misguided as teaching to the test. What do we really want out of our peer review process? Or should we abandon it?

Advertisements

2 thoughts on “Juking the stats in academic publishing

  1. Hi Anand,

    Very sorry to hear about the rejection!

    While I agree its odd to leave the authors hanging,
    at least its better for you to know now (early) that
    your paper is not going to make it, rather than leave
    you hanging.

    At least if you have to reformat it to submit to some
    other venue (e.g. ACM format to LNCS format) then
    one can get started early…

    -Jhala.

    • Well, I’m not super upset, really. The “extra lead time to reformat” is only useful inasmuch as I can incorporate whatever reviewer feedback there is — the majority of time spent for me in a resubmission is not in formatting, but in rewriting / improving the content. So it just seems like an empty gesture to reject the paper without providing the “justification” as it were.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s