Monthly Archives: December 2014

Annals of bad academic software: letters of recommendation

Posted on by

‘Tis the season for recommendation letters, and I again find myself thwarted by terrible UX and decisions made by people who manage application systems.

  • Why do I need to rank the candidate in 8 (or more!) different categories vs. people at my institution? Top 5% in terms of “self-motivation” or top 10%? What if they were an REU student not from my school? What if I have no point of comparison? What makes you think that people are either (a) going to make numbers up or (b) put top scores on everything because that is easier? Moreover why make it mandatory to answer these stupid questions to submit my letter?
  • One system made me cut and paste my letter as text into a text box, then proceeded to strip out all the line/paragraph breaks. ‘Tis a web-app designed by an idiot, full of incompetent input-handling, and hopefully at least signifying to the committee that they should admit the student.
  • Presumably the applicant filled out my contact information already, so why am I being asked to fill it out again?

It’s enough to make me send all letters by post — it would save time, I think.

“The needs of the many,” privilege, and power

Posted on by

There’s a certain set of sentiments which undergird a lot of thinking in engineering, and especially engineering about data. You want a method which has good performance “on average” over the population. The other extreme is worst-case, but there are things you can only do in the average case. By focusing on average-case gain, you get a kind of “the needs of the many outweigh the needs of the few” way of thinking about the world.

The needs of the many outweigh the needs of the one... or the few?

The needs of the many outweigh the needs of the one… or the few?

Now in the abstract land of mathematical models and algorithms, this might seem like a reasonable principle — if you have to cram everything into a single population utility function you might as well then optimize that. However, this gets messier when you start implementing it in the real world (unless of course you’re an economist of a certain stripe). The needs of the many are often the needs of the more powerful or dominant groups in society. The needs of the few are perhaps those who have been historically marginalized or victimized. Extolling the benefits to the many is often taking a stand for the powerful against the weak. It’s at best deeply insensitive.

Two instances of this have appeared on the blogosphere recently. Scott Aaronson blogged recently about MIT’s decision to take down Walter Lewin’s online videos after Lewin was found to have sexually harassed students in connection with the course. Scott believes that depriving students of Lewin’s materials is a terrible outcome, even (possibly) if he were a murderer. Ignoring the real hurt and trauma felt by those who are affected by Lewin’s actions is an exercise in privilege — because he is not hurt by it, he values the “the good of the many” trumping the “good of the few.”

The whole downplaying of sexual harassment as being somehow “not serious” enough to warrant a serious response (or that the response “makes the most dramatic possible statement about a broader social issue”) in fact trivializes the whole experience of sexual violence. Indeed, by this line of argument, because the content created by Lewin is so valuable, it may be ok to keep online even “had [he] gone on a murder spree.” The subtext of this is “as opposed to merely harassed some women.” I recommend reading Priya Phadnis on this case — she comes to a very different conclusion, namely that special pedestal that we put Walter Lewin on is itself the problem. Being able to downplay the female victims’ claims is exercising the sort of privilege that members of the male professoriat (myself included) indulge in overtly, covertly, and inadvertently. If STEM has a gender problem, it’s in a large part because we do not pay attention to the ways in which our words and actions reinforce existing tropes.

The second post was by Lance Fornow on dying languages in response to an op-ed by John McWhorter on why we should care about language diversity. Lance thinks that speaking a common language is a good thing:

I understand the desire of linguists and social scientists to want to keep these languages active, but to do so may make it harder for them to take advantage of our networked society. Linguists should study languages but they shouldn’t interfere with the natural progression. Every time a language dies, the world gets more connected and that’s not a bad thing.

I guess those poor bleeding-heart social scientists don’t understand that those languages are dying for a good reason. The good of the many — everyone speaking English, the dominant language — outweighs the good of the few. This attitude again speaks from a place of privilege and power, and it reinforces a kind cultural superiority (although I am sure Lance doesn’t think of it that way). Indeed, in many parts of the world, there is and continues to be “a strong reason to learn multiple languages.” By casually (and incorrectly) dismissing the importance of linguistic diversity, such a statement reinforces a chauvinist view of the relationship between language and technology.

We start with desirable outcomes: free quality educational materials that lower the barrier to access or speaking a common language to help facilitate communication and cooperation. By choosing to focus on those outcomes and their benefits to the many, we value their well-being and delegitimize the harm done to others. If we furthermore are speaking from a position of power, our privilege reinforces stigmas, casting a value judgement on the rights, experiences, and beliefs of the few. It’s something to be careful about.

Paper writing as a POMDP

Posted on by

During a research conversation this week I came to the realization that we really should model the paper writing as a partially observed Markov decision problem (POMDP). The writer has an action space of writing SIMPLE or FANCY statements. A SIMPLE statement follows very clearly from the previous statement and so should be easy to follow. A FANCY statement is a bit more challenging but might sound more impressive the reviewer. The reviewer has several states: BORED, IMPRESSED, and CONFUSED. Based on the input being SIMPLE or FANCY, the reviewer changes state according to a Markov chain with this state space.

The writer suffers a cost (or gains a reward, depending on how we want to model it) based on the current action and reviewer state. For example, issuing a FANCY statement when the reviewer is CONFUSED might suffer a higher cost than a FANCY statement when IMPRESSED. The goal is to minimize the discounted cost. This makes sense since reviewers become more fatigued over time, so extracting reward earlier in the paper is better.

Now, conventional wisdom might suggest that clarity is best, so a pure-SIMPLE policy should be a dominant strategy for this problem. However, depending on the cost and transition structure, I am not sure this is true.

And now, back to paper writing for me…

PaperCept, EDAS, and so on: why can’t we have nice things?

Posted on by

Why oh why can’t we have nice web-based software for academic things?

For conferences I’ve used PaperCept, EDAS (of course), Microsoft’s CMT, and EasyChair. I haven’t used HotCRP, but knowing Eddie it’s probably significantly better than the others.

I can’t think of a single time I’ve used PaperCept and had it work the way I expect. My first encounter was for Allerton, where it apparently would not allow quotation marks in the title of papers (an undocumented restriction!). But then again, why has nobody heard of sanitizing inputs? The IEEE Transactions on Automatic Control also uses PaperCept, and the paper review has a character restriction on it (something like 5000 or so). Given that a thorough review could easily pass twice that length, I’m shocked at this arbitrary restriction.

On the topic of journal software, the Information Theory Society semi-recently transitioned from Pareja to Manuscript Central. I have heard that Pareja, a home-grown solution, was lovable in its own way, but was also a bit of a terror to use as an Associate Editor. Manuscript Central’s editorial interface is like looking at the dashboard of a modern aircraft, however — perhaps efficient to the expert, but the interaction designers I know would blanche (or worse) to see it.

This semi-rant is due to an email I got about IEEE Collabratec (yeah, brah!):

IEEE is excited to announce the pilot rollout of a new suite of online tools where technology professionals can network, collaborate, and create – all in one central hub. We would like to invite you to be a pilot user for this new tool titled IEEE Collabratec™ (Formerly known as PPCT – Professional Productivity and Collaboration Tool). Please use the tool and tell us what you think, before we officially launch to authors, researchers, IEEE members and technology professionals like yourself around the globe.

What exactly is IEEE Collabratec?
IEEE Collabratec will offer technology professionals robust networking, collaborating, and authoring tools, while IEEE members will also receive access to exclusive features. IEEE Collabratec participants will be able to:

* Connect with technology professionals by location, technical interests, or career pursuits;
* Access research and collaborative authoring tools; and
* Establish a professional identity to showcase key accomplishments.

Parsing the miasma of buzzwords, my intuition is that this is supposed to be some sort of combination of LinkedIn, ResearchGate, and… Google Drive? Why does the IEEE think it has the expertise to pull off integration at this scale? Don’t get me wrong, there are tons of smart people in the IEEE, but this probably should be done by professionals, and not non-profit professional societies. How much money is this going to cost? The whole thing reminds me of Illinois politics — a lucrative contract given to a wealthy campaign contributor after the election, with enough marketing veneer to avoid raising a stink. Except this is the IEEE, not Richard [JM] Daley (or Rahm Emmanuel for that matter).

As far as I can tell, the software that we have to interact with regularly as academics has never been subjected to scrutiny by any user-interface designer. From online graduate school/faculty application forms (don’t get me started on the letter of rec interface), conference review systems, journal editing systems, and on, we are given a terrible dilemma: pay exorbitant amounts of money to some third party, or use “home grown” solutions developed by our colleagues. For the former, there is precious little competition and they have no financial incentive to improve the interface. For the latter, we are at the whims of the home code-gardener. Do they care about user experience? Is that their expertise? Do they have time to both make it functional and be a pleasure to use? Sadly, the answer is usually no, with perhaps a few exceptions.

I shake my fist at the screen.

A quote for these times

Posted on by

You may well ask: “Why direct action? Why sit ins, marches and so forth? Isn’t negotiation a better path?” You are quite right in calling for negotiation. Indeed, this is the very purpose of direct action. Nonviolent direct action seeks to create such a crisis and foster such a tension that a community which has constantly refused to negotiate is forced to confront the issue. It seeks so to dramatize the issue that it can no longer be ignored. My citing the creation of tension as part of the work of the nonviolent resister may sound rather shocking. But I must confess that I am not afraid of the word “tension.” I have earnestly opposed violent tension, but there is a type of constructive, nonviolent tension which is necessary for growth. Just as Socrates felt that it was necessary to create a tension in the mind so that individuals could rise from the bondage of myths and half truths to the unfettered realm of creative analysis and objective appraisal, so must we see the need for nonviolent gadflies to create the kind of tension in society that will help men rise from the dark depths of prejudice and racism to the majestic heights of understanding and brotherhood. The purpose of our direct action program is to create a situation so crisis packed that it will inevitably open the door to negotiation.

Martin Luther King, Jr., Letter from a Birmingham Jail

Vegan Caldo Verde (Portuguese Kale Soup)

Posted on by

I am not a vegetarian but I don’t usually cook meat when eating at home. Back in grad school I had a CSA and they would put a recipe in with every box (along with some news from the farm). One week it was a recipe for Portuguese kale soup, or caldo verde, and I remember it being delicious. Since the weather has been getting cold here I decided last night to make a batch to keep we warm during the last week of classes. When I went to the store to pick up the chorizo, however, I thought it would be more fun (and easier to share) to make a vegetarian version — that way I could use up my shiitake mushrooms too!

Vegan Caldo Verde

Vegan Caldo Verde

The proportions are not too fussy — it depends on how starchy/soupy you want it.

Vegan Caldo Verde

2 vegan chouriço (or chorizo) sausages, sliced
12 shiitake mushrooms, sliced (should cook down to same volume as chorizo)
1/2 – 1 lb potatoes, diced (chunk size based on how you want to eat it)
1 onion, sliced
2 cloves garlic
6 cups liquid (used a 1:2 mix of veg. broth and water)
1 lb kale, shredded (thin slices are more traditional, but laziness wins often)
olive oil
salt and pepper

  1. Using a soup pot or dutch oven, brown sausage slices in olive oil (you don’t need too much), remove and set aside.  Add additional oil if needed and cook shiitakes until they lose their liquid.  Remove those too.
  2. Add additional oil and cook onion (with a little salt) until translucent, then add garlic and cook until aromatic (be sure not to burn).
  3. Add potatoes and mix to coat, then add broth (make sure you cover the vegetables, add more if needed), cover, and bring to a boil. Uncover and reduce to a simmer until the potatoes are cooked through (15 min or maybe longer depending on the variety and size of your dice).
  4. (Optional) Use an immersion blender to partially puree some of the onion/potato mixture to thicken the soup.
  5. Add mushrooms and chorizo. Return to a boil.
  6. Add kale and cook down, around 5 minutes.  Be sure not to overcook the kale. Grind generous amounts of pepper and mix in.

If you are feeling fancy you can add some additional spice by adding pimenton ahumado (smoked paprika) or other spicy element.  I purposefully diluted the broth so that the mushrooms and spices in the chorizo could lend some flavor. I think you could also cook some more chorizo and garnish the bowl with a slice or two of browned chorizo in the middle. The mix of mushrooms and chorizo adds some textural interest and additional flavor, I think. Perhaps a little soy sauce in there would help up the umami.

I have no idea how many portions this makes, but I am guessing it’s at least 4-6 servings for me.  Appetites vary of course.

WIFS 2014

Posted on by

This week I took a quick jaunt down to Atlanta to attend part of WIFS 2014 (co-located with GlobalSIP 2014). Kamalika and I were invited to give a talk on differential privacy and machine learning, based on our IEEE Signal Processing Magazine article. I’ve uploaded the slides of the tutorial to my website and we’re planning on making a video (audio over slides) version for SigView as well as on YouTube.

Much like last year, GlobalSIP had a somewhat disjointed, semi-chaotic feel (exacerbated by tiredness, I am sure) — it’s really a collection of semi-interacting workshops in the same space, and I knew people in several of the other workshops. Since I was there for a day and giving a tutorial at WIFS, I decided to stick with WIFS for the day. To give a sense of how confusing it all was, here’s a picture of the guide to deciphering the program book:

Overly-complicated rules for encoding sessions

Overly-complicated rules for encoding sessions

The keynote for GlobalSIP was given by Vince Poor on information-theoretic privacy via rate distortion (this is the work with Lalitha). Vince did a good job of not over-IT-ing it I think, which was good because the audience was pretty diverse and it’s not clear that many of the people there had even taken a course on information theory. This seems to be the big challenge in multi-disciplinary conferences like GlobalSIP (or large signal processing conferences in general) — everyone is in signal processing, but it’s a big tent and it’s hard to reach everyone.

Min Wu was the keynote speaker for the WIFS workshop on the day I attended. Her talk, on “Exploring Power Network Signatures for Information Forensics” was about how to glean information from power fluctuations in networks, or electronic network frequency (ENF). Different processes or operations have different power demands — by matching these signatures to an observed signal (e.g. a video), one can make inferences about the time/location/integrity of the data. For example, were the audio and visual tracks in a video taken at the same time or merged later? This whole area is quite interesting, and while I was sort of aware of this work I hadn’t really read up on much of it.

Perhaps it was the end of the semester kicking in, but I sort of took terrible notes on most of the talks and poster sessions at the conference, so I can’t really write coherently about the papers I saw. Unfortunately I had to run back to teach the penultimate lecture in my class. I guess now that I have a “real job” this is going to be the way it works from now on. Kind of sad, really.

Feature Engineering for Review Times

Posted on by

The most popular topic of conversation among information theory afficionados is probably the long review times for the IEEE Transactions on Information Theory. Everyone has a story of a very delayed review — either for their own paper or for a friend of theirs. The Information Theory Society Board of Governors and Editor-in-Chief have presented charts of “sub-to-pub” times and other statistics and are working hard on ways to improve the speed of reviews without impairing their quality. These are all laudable. But it occurs to me that there is room for social engineering on the input side of things as well. That is, if we treat the process as a black box, with inputs (papers) and outputs (decisions), what would a machine-learning approach to predicting decision time do?

Perhaps the most important (and overlooked in some cases) aspects of learning a predictor from real data is figuring out what features to measure about each of the inputs. Off the top of my head, things which may be predictive include:

  • length
  • number of citations
  • number of equations
  • number of theorems/lemmas/etc.
  • number of previous IT papers by the authors
  • h-index of authors
  • membership status of the authors (student members to Fellows)
  • associate editor handling the paper — although for obvious reasons we may not want to include this

I am sure I am missing a bunch of relevant measurable quantities here, but you get the picture.

I would bet that paper length is a strong predictor of review time, not because it takes a longer time to read a longer paper, but because the activation energy of actually picking up the paper to review it is a nonlinear function of the length.

Doing a regression analysis might yield some interesting suggestions on how to pick coauthors and paper length to minimize the review time. This could also help make the system go faster, no? Should we request these sort of statistics from the EiC?