I’m at EPFL right now on my way back from a trip to India. My last work-related stop there was the Tata Institute of Fundamental Research, where I am working with Vinod Prabhakaran on some follow-up work to our ISIT paper this year on correlated sampling. TIFR has a lovely campus in Navy Nagar, right next to the ocean in the south of Mumbai. It’s a short walk across the lawn to the ocean:

The beach at TIFR, looking north to the haze-obscured skyline of Mumbai

The grounds themselves are pretty rigorously landscaped and manicured:

The main building seen across the lawn in front of the ocean.

Earlier in my trip I visited IIT-Madras, hosted by Andrew Thangaraj and Radha Krishna Ganti and IIT-Bombay, where I was working with Bikash Dey on extensions to some of our AVCs-with-delay work. It’s been a great trip and it’s nice to visit so many different schools. The challenges facing Indian higher education are very different than those in the US, and it’s a good change of perspective from my usual US-centric view of things. Maybe I’ll write more on that later.

But for now, I wanted to get back to some actual technical stuff, so I thought I’d mention something that came up in the course of my discussions with Vinod today. For a joint distribution $P(X,Y)$, Wyner defined the common information in the the following way:

$\mathsf{CI}(X;Y) = \min_{X--U--Y} I(XY ; U)$

One fun fact about the common information is that it’s larger than the mutual information, so $I(X;Y) \le \mathsf{CI}(X;Y)$. One natural question that popped up as part of a calculation we had to do was whether for doubly-symmetric binary sources we could have a bound like

$\mathsf{CI}(X;Y) \le \alpha I(X;Y)$

for some “nice” $\alpha$. In particular, it would have been nice for us if the inequality held with $\alpha = 2$ but that turns out to not be the case.

Suppose $(X,Y)$ and are a doubly-symmetric binary source, where $Y$ is formed from $X \sim \mathsf{Bern}(1/2)$ by passing it through a binary symmetric channel (BSC) with crossover probability $\alpha$. Clearly the mutual information is $I(X;Y) = 1 - h(\alpha)$. For the common information, we turn to Wyner’s paper, which says $\mathsf{CI}(X;Y) = 1 + h(\alpha) - 2 h( \frac{1}{2}( 1 - \sqrt{1 - 2 \alpha} ) )$, which is a bit of a weird expression. Plotting the two for $\alpha$ between 0 and 1/2 we get:

Mutual and Common Information for a DSBS

If we plot the ratio $\mathsf{CI}(X;Y)/I(X;Y)$ versus $\alpha$, we get the following:

Ratio of Common to Mutual Information for a DSBS

The ratio blows up! So not only is Common Information larger than mutual information, it can be an arbitrarily large factor larger than the mutual information.

It might seem like this is bad news for us, but it just made the problem we’re working on more interesting. If we get any results on that I might be less engimatic and blog about it.

Avleen Bijral, one of the students here at TTIC, sent me a link to a pretty cool browser plugin for GMail called GMail TeX. Basically it lets you send LaTeX formatted emails via gmail. This can be quite a time saver for the reader (but maybe not the writer) if you’re remotely collaborating on some project (which I so often am these days, it seems). It’s supported on a wide variety of browsers and seems worth checking out!

I got an email from the Rutgers ECE department about a new scam that has been preying on international students. Some readers of the blog may wish to be aware of this.

Scammers posing as immigration officials – United States Citizenship and Immigration Services (USCIS) officers, Department of State (DOS) officials, or local policemen are contacting international students, & scholars. They may ask for your personal information (such as SSN, passport number, credit card information), identify false problems with your immigration record, and ask for payment to correct the record.

These scams can be very sophisticated with a USCIS like number appearing on your caller ID, the caller knowing a lot of your personal information and so on. Here are a few pointers to bear in mind if you receive this kind of a fake “immigration” call:

• Don’t wire/pay any money. USCIS will never call someone to ask for any form of payment over the phone.
• Just hang up and don’t call back.
• Call the real USCIS National Customer Service Center at 1-800-375-5283 to report the issue.
• Never give out your personal information. No matter who a caller claims to be, don’t give him/her your I-94 number, “A” number, visa control number or any other personal information. Hang up and call the real USCIS.

Apparently Rutgers had an incident involving a postdoc, where the caller posed as a police officer “and informed the scholar that he had immigration problems. This female caller had the scholar’s home address, cell phone number and SSN number. She informed the scholar that the problem can be resolved if he could make an immediate payment.”

Vi Hart explains serial music (h/t Jim CaJacob).

More adventures in trolling scam journals with bogus papers (h/t my father).

Vladimir Horowitz plays a concert at the Carter White House. Also Jim Lehrer looks very young. The program (as cribbed from YouTube)

• The Star-Spangled Banner
• Chopin: Sonata in B-flat minor, opus 35, n°2
• Chopin: Waltz in a minor, opus 34, n°2
• Chopin: Waltz in C-sharp minor, opus 64, n° 2
• Chopin: Polonaise in A-flat major, opus 53 ,Héroïque
• Schumann: Träumerei, Kinderszene n°7
• Rachmaninoff: Polka de W.R
• Horowitz: Variations on a theme﻿ from Bizet’s Carmen

The Simons Institute is going strong at Berkeley now. Moritz Hardt has some opinions about what CS theory should say about “big data,” and how it might be require some adjustments to ways of thinking. Suresh responds in part by pointing out some of the successes of the past.

John Holbo is reading Appiah and makes me want to read Appiah. My book queue is already a bit long though…

An important thing to realize about performance art that makes a splash is that it can be often exploitative.

I really love language, but my desire to be idiomatic often runs athwart to the goals of scholarly communication. Especially when motivating a paper, I enjoy writing sentences such as this:

In the data-rich setting, at first blush it appears that learning algorithms can enjoy both low privacy risk and high utility.

(Yes, I know it’s passive). However, consider the poor graduate student for whom English is a second (or third) language – this sentence is needlessly confusing for them. If we are really honest, this is the dominant group of people who will actually read the paper, so if I am writing with my “target audience” in mind, I should eschew idioms, literary allusions, and the like. Technical papers should be written in a technical and global English (as opposed to a specific canonized World English).

Nevertheless, I want to write papers in a “writerly” way; I want to use the fact that my chosen career is one of constant writing as an opportunity to improve my communication skills, but I also want to play, to exploit the richness of English, and even to slip in the occasional pun. Am I a linguistic chauvinist? Unlike mathematicians, I never had to learn to read scholarly works in another language, so I have the luxury of lazily indulging my fantasies of being an Author.

When I review papers I often have many comments about grammatical issues, but perhaps in a global English of scholarly communication it doesn’t matter as long as the argument is intelligible. Do we need so many articles? Is consistent tense really that important? I think so, but I don’t have a philosophically consistent argument for what may appear, in situ, to be a prescriptivist attitude.

The cynical side of me says that nobody reads papers anyway, so there is no point in worrying about these issues or even spending time on the literary aspects of technical papers. Cynicism has never been very nourishing for me, though, so I am hoping for an alternative…

One big difference between reviewing for conferences like NIPS/ICML and ISIT is that there is a “discussion” period between the reviewers and the Area Chair. These discussions are not anonymized, so you know who the other reviewers are and you can also read their reviews. This leads to a little privacy problem — A and B may be reviewing the same paper P, but A may be an author on a paper Q which is also being reviewed by B. Because A will have access to the text of B’s reviews on P and Q, they can (often) unmask B’s authorship of the review on Q simply by looking at the formatting of the reviews (are bullet points dashes or asterisks, do they give numbered points, are there “sections” to the review, etc). This seems to violate the spirit of anonymous review, which is perhaps why some have suggested that reviewing be unblinded (at least after acceptance).

The extent to which all of this matter is of course a product of the how fast the machine learning literature has grown and the highly competitive nature of the “top tier conferences.” Because the acceptance rate is so low, the reviewing process can appear to be “arbitrary” (read: subjective) and so questions of both review quality and author/review anonymity impact possible biases. However, if aim of double-blind reviewing is to reduce bias, then shouldn’t the discussions also be anonymized?

Michael Eisen gave a talk at the Commonwealth Club in San Francisco recently. Eisen is the founder of the Public Library of Science (PLoS), which publishes a large number of open-access journals in the biosciences, including the amazingly named PLoS Neglected Tropical Diseases. His remarks begin with the background on the “stranglehold existing journals have on academic publishing.” But he also has this throwaway remark:

One last bit of introduction. I am a scientist, and so, for the rest of this talk, I am going to focus on the scientific literature. But everything I will say holds equally true for other areas of scholarship.

This is simply not true — one cannot generalize from one domain of scholarship to all areas of scholarship. In fact, it is in the differences between dysfunctions of academic communication across areas that we can understand what to do about it. It’s not just that this is a lazy generalization, but rather that the as Eisen paints it, in science the journals are more or less separate from the researchers and parasitic entities. As such, there are no reasons that people should publish with academic publishers except for some kind of Stockholm syndrome.

In electrical engineering and computer science the situation is a bit different. IEEE and ACM are not just publishing conglomerates, but are supposed to be the professional societies for their respective fields. People gain professional brownie points for winning IEEE or ACM awards, they can “level up” by becoming Senior Members, and so on. Because disciplinary boundaries are a little more fluid, there are several different Transactions in which a given researcher may publish. At least on paper, IEEE and ACM are not-for-profit corporations. This is not to say that engineering researchers are not suffering from a Stockholm syndrome effect with these professional societies. It’s just that the nature of the beast is different, and when we talk about how IEEExplore or ACM Digital Library is overpriced, that critique should be coupled with one of IEEE’s policy requiring conferences to have a certain profit level. These things are related.

The second issue I had is with Eisen’s proposed solution:

There should be no journal hierarchy, only broad journals like PLOS ONE. When papers are submitted to these journals, they should be immediately made available for free online – clearly marked to indicate that they have not yet been reviewed, but there to be used by people in the field capable of deciding on their own if the work is sound and important.

So… this already exists for large portions of mathematics and mathematical sciences and engineering in the form of ArXiV. The added suggestion is a layer of peer-review on top, so maybe ArXiV plus a StackExchange thing. Perhaps this notion is a radical shift for life sciences where Science and Nature are so dominant, but what I learn myself from looking at the ArXiV RSS feed is that the first drafts of papers that get put up there are usually not the clearest exposition of the work, and without some kind of community sanction (in the form of rejection), there is little incentive for authors to actually go back and make a cleaner version of their proof. If someone has a good idea or result but a confusing presentation they are not going to get downvoted. If someone is famous they are unlikely to get downvoted.

In the end what PLoS ONE and the ArXiV-only model for publishing does is reify and retrench the existing tit-for-tat “clubbiness” that exists in smaller academic communities. In a lot of CS conferences reviewing is double-blind as a way to address this very issue. When someone says “all academic publishing has the same problems” this misses the point, because the problems is not always with publishing but with communication. We need to understand the how the way we communicate the products scholarly knowledge is broken. In some fields, I bet you could argue that papers are inefficient and bad ways of communicating results. In this sense, academic publishing and its rapacious nature are just symptoms of a larger problem.

I’m happy to announce that in January 2014 I will be joining the Electrical and Computer Engineering Department at Rutgers, The State University of New Jersey, as an Assistant Professor.

My department chair sent out a recent notice from the NSF about the impact of the sequestration order on the NSF awards.

At NSF, the major impact of sequestration will be seen in reductions to the number of new research grants and cooperative agreements awarded in FY 2013. We anticipate that the total number of new research grants will be reduced by approximately 1,000.

In FY2011 the NSF funded 11,185 proposals, so that’s an 8.94% reduction. Yikes.

As I’ve gotten farther along in this whole research career, I’ve found it more and more difficult to figure out the optimal way to balance the different things one does at a conference :

• Going to talks. This is ostensibly the point of the conference. It’s impossible to read all of the papers that are out there and a talk is a fast way to get the gist of a bunch of papers or learn about a new problem in less time than it takes to really read and digest the paper. We’re social creatures so it’s more natural to get information this way.
• Meeting collaborators to talk about research problems. I have lots of collaborators who are outside TTI and a conference is a good chance to catch up with them face-to-face, actually sit down and hammer out some details of a problem, or work on a new problem with a (potential) new collaborator. Time sitting over a notepad is time not spent in talks, though.
• Professional networking. I’m on the job market now, and it’s important to at least chat casually with people about your research, what you think is exciting your future plans, and the like. This is sometimes the “real” point of conferences.
• Social networking. Sometimes conferences are the only times I get to see my friends from grad school, and in a sense your professional peers are the only people who “get” your crazy obsession with esoteric problem $P$ and like to get a beer with you.

So the question for the readership : how do you decide the right balance for yourself? Do you go in with a plan to see at least N talks or a certain set $S$ of talks, or are you open to just huddling in the corner with a notepad?

I wrote this post in an attempt to procrastinate about ITA blogging, which I will get to in a bit. I went to far fewer talks than I expected to this year, but I’ll write about ‘em later.