As part of ITAVision 2015 we are soliciting individuals and groups to submit videos documenting their love of information theory and/or its applications. During ISIT we put together a little example with our volunteers (it sounded better in rehearsal than at the banquet, alas). The song was Entropy is Awesome based on this, obviously. If you want to sing along, here is the Karaoke version:
The lyrics (so far) are:
Entropy is awesome!
Entropy is sum minus p log p
Entropy is awesome!
When you work on I.T.
Blockwise error vanishes as n gets bigger
Maximize I X Y
Let’s party forever
I get you, you get me
Communicating at capacity
Entropy is awesome…
This iteration of the lyrics is due to a number of contributors — truly a group effort. If you want to help flesh out the rest of the song, please feel free to email me and we’ll get a group effort going.
More details on the contest will be forthcoming!
While trying to show a student a generic example of a paper’s structure, I came across this gem:
A sample from the IT Transactions of 1992
I feel like I am reading a MacWrite document while wearing a flannel shirt.
This one might be a keeper!
I saw a paper on ArXiV yesterday called Kalman meets Shannon, which got me thinking: in how many papers has someone met Shannon, anyway? Krish blogged about this a few years ago, but since then Shannon has managed to meet some more people. I plugged “meets Shannon” into Google Scholar, and out popped:
- Fourier: Wang and Giannakis, Wireless Multicarrier Communications: Where Fourier Meets Shannon, IEEE Signal Processing Magazine, 2000.
- Bode: Elia, When Bode meets Shannon: control-oriented feedback communication schemes, IEEE Transactions on Automatic Control, 2004.
- Maxwell: Chakraborty and Franceschetti, Maxwell meets Shannon: Space-time duality in multiple antenna channels, Allerton 2006, and Lee and Chung, Capacity scaling of wireless ad hoc networks: Shannon meets Maxwell, IEEE Transactions on Information Theory, 2012.
- Carnot: Shental and Kanter, Shannon Meets Carnot: Generalized Second Thermodynamic Law, Europhysics Letters, 2009.
- Nash: Berry and Tse, Shannon Meets Nash on the Interference Channel, IEEE Transactions on Information Theory, 2011.
- Walras: Jorswieck and Mochaourab, Shannon Meets Walras on Interference Networks, ITA Workshop 2013.
- Nyqust: Chen, Eldar, and Goldsmith,
Shannon Meets Nyquist: Capacity of Sampled Gaussian Channels, IEEE Transactions on Information Theory, 2013.
- Strang and Fix: Dragotti, Vetterli, and Blu, Sampling moments and reconstructing signals of finite rate of innovation: Shannon meets Strang–Fix, IEEE Transactions on Signal Processing, 2007.
- Blackwell and LeCam: Raginsky, Shannon meets Blackwell and Le Cam: channels, codes, and statistical experiments, ISIT 2011.
- Wiener: Forney, On the role of MMSE estimation in approaching the information-theoretic limits of linear Gaussian channels: Shannon meets Wiener, Allerton 2003, and Forney, Shannon meets Wiener II: On MMSE estimation in successive decoding schemes, Allerton 2004 and ArXiv 2004.
- Bellman: Meyn and Mathew, Shannon meets Bellman: Feature based Markovian models for detection and optimization, CDC 2008.
- Tesla: Grover and Sahai, Shannon meets Tesla: Wireless information and power transfer, ISIT 2010.
- Shortz: Efron, Shannon Meets Shortz: A Probabilistic Model of Crossword Puzzle Difficulty, Journal of the American Society for Information Science and Technology, 2008.
- Marconi: Tse, Modern Wireless Communication: When Shannon Meets Marconi, ICASSP 2006.
- Kalman: Gattami, Kalman meets Shannon, ArXiV 2014.
Sometimes people are meeting Shannon, and sometimes he is meeting them, but each meeting produces at least one paper.
Via Cynthia, here is a column by James Mickens about how horrible the web is right now:
Computer scientists often look at Web pages in the same way that my friend looked at farms. People think that Web browsers are elegant computation platforms, and Web pages are light, fluffy things that you can edit in Notepad as you trade ironic comments with your friends in the coffee shop. Nothing could be further from the truth. A modern Web page is a catastrophe. It’s like a scene from one of those apocalyptic medieval paintings that depicts what would happen if Galactus arrived: people are tumbling into fiery crevasses and lamenting various lamentable things and hanging from playground equipment that would not pass OSHA safety checks.
It’s a fun read, but also a sentiment that may echo with those who truly believe in “clean slate networking.” I remember going to a tutorial on LTE and having a vision of what 6G systems will look like. One thing that is not present, though, is the sense that the system is unstable, and that the introduction of another feature in communication systems will cause the house of cards to collapse. Mickens seems to think the web is nearly there. The reason I thought of this is the recent fracas over the US ceding control of ICANN, and the sort of doomsdaying around that. From my perspective, network operators are sufficiently conservative that they can’t/won’t willy-nilly introduce new features that are only half-supported, like the in Web. The result is a (relatively) stable networking world that appears to detractors as somewhat Jurassic.
I’d argue (with less hyperbole) that some of our curriculum ideas also suffer from the accretion of old ideas. When I took DSP oh-so-long ago (13 years, really?) we learned all of this Direct Form Transposed II blah blah which I’m sure was useful for DSP engineers at TI to know at some point, but has no place in a curriculum now. And yet I imagine there are many places that still teaching it. If anyone reads this still, what are the dinosaurs in your curriculum?
Some of my office furniture is on backorder, like the standing desk unit and my actual desktop, but in the meantime I have found a use for the hardcopy IEEE Transactions that I’ve been carting around with me from job to job:
A way to use IEEE Transactions
I think during my time hanging out with machine learners, no topic has received as much attention as the quality of the review process for competitive conferences. My father passed along this paper by Graham Cormode on “the tools and techniques of the adversarial reviewer”, which should be familiar to many. I had not seen it before, but a lot of the “adversarial” techniques sounded familiar from reviews I have received. I also wonder to what extent reviews I have written could be interpreted as deliberately adversarial. I don’t go into the review process that way, but it’s easy to ascribe malign intent to negative feedback.
Cormode identifies 4 characteristics of the adversarial reviewer: grumpiness, elitism, peevishness, and arrogance. He then identifies several boilerplate approaches to writing a negative review, specific strategies for different sections of the paper, and the art of writing pros and cons for the summary. My favorite in this latter section is that the comment “paper is clearly written” really means “clearly, the paper has been written.”
As Cormode puts it himself at the end of the paper: “I am unable to think of any individual who consistently acts as an adversarial reviewer; rather, this is a role that we can fall into accidentally when placed under adverse conditions.” I think this is all-to-true. When reviewing the 9th paper for a conference with 3 weeks to do all 9, the patience of the reviewer may be worn a bit thin, and it’s easy to be lazy and not take the paper on its own merits. What’s certainly true, however, is that “editors and PC members” often do not “realize when a review is adversarial.” In part this is because as a research community, we don’t want to acknowledge that there are real problems with the review process that need fixing.
I am traveling all over India at the moment so I’m not really able to write contentful posts. Here are even more links instead, sigh. Maybe later I’ll talk about log-Sobolev inequalities so I can be cool like Max.
Speaking of Max, he posted this hilarious bad lip reading version of Game of Thrones. Probably NSFW. I don’t even like the series but it’s pretty funny.
For those who are fans of Rejected, Don Hertzfeldt’s new film is available on Vimeo.
Those who were at Berkeley may remember seeing Ed Reed perform at the Cheeseboard. His album (which I helped fund via indiegogo, was named a Downbeat Editors’ Pick. It’s a great album.
In light of the Snowden leaks, some doubt has been cast on NIST’s crypto standards.
I’m super late to this, but I endorse Andrew’s endorsement of Sergio‘s interview with Robert Fano in the IT Newsletter. Here’s just the article, if you want that.
A map of racial segregation in the US.
Vi Hart explains serial music (h/t Jim CaJacob).
More adventures in trolling scam journals with bogus papers (h/t my father).
Brighten does some number crunching on his research notebook.
Jerry takes “disruptive innovation” to task.
Vladimir Horowitz plays a concert at the Carter White House. Also Jim Lehrer looks very young. The program (as cribbed from YouTube)
- The Star-Spangled Banner
- Chopin: Sonata in B-flat minor, opus 35, n°2
- Chopin: Waltz in a minor, opus 34, n°2
- Chopin: Waltz in C-sharp minor, opus 64, n° 2
- Chopin: Polonaise in A-flat major, opus 53 ,Héroïque
- Schumann: Träumerei, Kinderszene n°7
- Rachmaninoff: Polka de W.R
- Horowitz: Variations on a theme from Bizet’s Carmen
The Simons Institute is going strong at Berkeley now. Moritz Hardt has some opinions about what CS theory should say about “big data,” and how it might be require some adjustments to ways of thinking. Suresh responds in part by pointing out some of the successes of the past.
John Holbo is reading Appiah and makes me want to read Appiah. My book queue is already a bit long though…
An important thing to realize about performance art that makes a splash is that it can be often exploitative.
Mimosa shows us what she sees.
The English version of the Japanese cooking site Cookpad was launched recently. The launch means more lunch for me!
In case you wanted to listen to old African vinyl albums, you’re in luck.
I have a burning-hot hatred of payday loan places, so this Pro Publica piece just stoked the fire.
Talking robots… in spaaaaaaaaace!
A tumblr on how we make progress in research.
My friend Amrys worked on the Serendip-o-matic, a tool that may be more useful for those in the humanities than us engineer types, but is pretty darn cool.