conference selectivity, publication delay, and a proposal

I should preface this post by warning any readers that these thoughts are pretty rambling and tentative. I don’t fully understand all of these issues, but they do interest me, so I find it helpful to write about them. I’m not even certain that my proposed idea here would work, but I think there seems to be some problems with the status quo that need to be addressed.

A recurrent theme in many conversations I had at ISIT was about the nature and purpose of the conference. Part of this concern stems from the stressful nature of the job market, and some from changes in the culture of the field. With regards to the former, the less selective nature of ISIT and electrical engineering conferences in general makes it difficult to evaluate the quality of work of a graduate student without being familiar with the area. Because of the incredibly long delay in journal publication, there may be no objective approval of the work by the time the student graduates. The underlying problem is that information theory is expanding and changing rapidly, and the publication and conference infrastructure hasn’t adapted to deal with the higher pace and volume of research. A lot of different arguments and proposals were thrown out there, but one that seemed to garner some interest was the foundation of a new journal for smaller results — a new Information Processing Letters, if you will.

Several people complained that ISIT was too big and that there were too many “less interesting” results. The trite answers are that the field is pretty big and that what is interesting to specialists may not be interesting to the general information theory community. If we take a comparative view, another model to look at is that of CS theory. There there are two or three very selective conferences — FOCS, STOC, and SODA. The acceptance rate for these conferences is quite low, they are single track, and all of the results presented are therefore “interesting.” Adopting such as system off the shelf misses the point of a conference like ISIT, I think.

ISIT serves several different functions that I know about. The first is to take a snapshot of the field and its progress and capture this in the proceedings. Prior to last year, the proceedings only contained a single-page extended abstract and summary of the results. The original intent that those results which were big enough would get sent to the Transactions. Another function is get sufficient funds via registration fees to pay for the transactions and the operation of the Information Theory Society. Now the proceedings are on CDROM and contain the full (5 page) papers, so many of them are (mostly) fleshed-out short papers. Finally, the conference is supposed to facilitate the contact between different research subcommunities, e.g. to get the coding people to talk to the Shannon theorists.

The first fundamental problem is that information theory has expanded as a research field — there were 4+ sessions on Network Coding alone, and those problems didn’t even exist 10 years ago. As someone told me, the flavor of the conference has shifted slowly over the years, and the incremental changes are hard to detect. The net effect is that the Transactions have become enormous and bulky — many classic papers from years ago were 10 pages, but now it’s rare to see a paper under 15-20 I bet. The conference has also become enormous and bulky, with so many parallel sessions that it’s hard to even get an idea of what is going in the field. Is this because too many papers are accepted? Perhaps. I certainly wouldn’t mind if there were about 25% fewer papers there, just so that I could see a greater percentage of what was going on. Greater selectivity would mean higher quality as well, but there are costs to that. For example, next year the conference is in Nice, and I doubt many grad students in the US would be able to just go there unless they had a paper. A lower acceptance rate would also impact the fundraising aspects.

What about the positive benefits of a low acceptance rate? Some people have argued that a low acceptance rate provides a skewed but somewhat objective measure of the quality of research done by a graduate student. That is, four solid papers at “good conferences” (in the CS sense) mean that the student has developed a solid research program. This prestige factor doesn’t hold in EE and information theory because the prestige is with journal publications, not conferences. But I noted earlier, journals take forever to publish papers, and the published papers tend to be quite long (and not because they have lots of figures). So if the burden of prestige is shifted to conferences, it might be better.

The second fundamental problem is that the pace of research is speeding up, while the venues for providing thoroughly peer-reviewed research are diminished. While all papers at ISIT are reviewed, the reviewing is somewhat uneven (I had very few reviews of my paper, while others had many). Since conferences are more about sharing the results of investigations rather than presenting the most interesting new results of the field, it’s hard to separate those ides which are going somewhere from promising directions and dead-ends, both of which may be interesting in their own right.

One solution that might work for both these problems (increased research pace and more people) would be to have more venues for journal publication. A new journal with thorough peer review and editing but an emphasis on smaller results would possibly alleviate the burden on the Transactions and speed up the pace of journal publication. The conferences could still be big, but then conference presenters with more or less polished end results could journalize those rather than waiting to included in a larger paper sent to the Transactions. It’s worth thinking about in addition to pruning down the size of ISIT a bit to avoid having 12 parallel sessions 5 years from now.