Conditional growth charts

The Annals of Statistics published a “discussion paper” on Conditional growth charts by Y. Wei and and X. He, with 4 comment papers and a rejoinder from the authors. It’s reminiscent of the Crooked Timber book events (Iron Council was a particular favorite), only significantly more formal. It’s a new format to me for mathematical publication, and almost feels like “peer review in public,” with the mean comments taken out. Perhaps if I get some time I’ll read the paper for real and see what the fuss is about…

a little puzzle

Prasad brought work to a halt earlier this week by issuing the following little conundrum. One indepdenent fair 6-sided die is rolled for each of n people. Each person’s number is written on a card and stuck on their head so that they can see everyone else’s number but their own. The people are not allowed to communicate after the numbers are assigned, and from just viewing the others’ numbers they must guess their own. They can come up with any rule that they like for doing this.

What rule should you choose to that is the probability that all of them guess correctly is maximized, and what is that probability?

This is a variation on the “colored hats game,” but the criterion you want to maximize is slightly different than in some instances of the game.

A first rehearsal with Adams and Sellars

On Monday the SF Symphony Chorus had their first rehearsal with John Adams and Peter Sellars for A Flowering Tree. The opera, which received its premiere at the New Crowned Hope festival, is adapted from a Kannda folktale. The libretto is an amalgamation of the folktale text, Tamil love poems, and Virasaiva religious poems. Our rehearsals to this point have been somewhat routine — learning the notes, getting a handle on the tricky rhythms and meter changes, and using little tricks to help ourselves be heard over the orchestra.

Monday’s rehearsals catapulted us into the world of characterization and theater. Our performance will be “semi-staged,” from what I understand, and I saw bits of scenery backstage before the rehearsal. Turning a massive chorus into a dramatic agent is no small task, and Sellars was as clear and effective as any director I have seen. He knew at what level he had to talk to the chorus to get the effect he wanted, and we the change in effect from the first readthrough to the second (after some direction) was huge. Adams will be conducting us, and he is likewise clear and direct in his requests and his conducting. It’s a real treat working on a piece like this, and after Monday’s rehearsal I am certain that the audience will be wowed. Although I do have to say that next week will put my voice through the wringer, so I better stock up this weekend on lemon juice, honey, and ginger…

Concert Announcement : A Flowering Tree

Sometime if I have time I’ll write about our first rehearsal with Adams and Sellars. I have also written a small note on some of the religious poetry used in the libretto.

A Flowering Tree

by John Adams
libretto by John Adams and Peter Sellars

John Adams, conductor
Peter Sellars, director
Jessica Rivera, soprano
Russell Thomas, tenor
Eric Owens, bass
SFS Chorus, chorus

America’s foremost living composer, John Adams, imagines rich and beautiful worlds. This SFS co-commission, inspired by The Magic Flute, is an escape into dream and myth and comes on the heels of Adams’s opera Doctor Atomic. Peter Sellars returns to direct this semi-staged production. The premiere of any new Adams work is an event not to be missed.

Thursday 3/1 — Saturday 3/3, 7:30 PM

paper a day : approximating high-dimensional pdfs by low-dimensional ones

Asymptotically Optimal Approximation of Multidimensional pdf’s by Lower Dimensional pdf’s
Steven Kay
IEEE Transactions on Signal Processing, V. 55 No. 2, Feb. 2007, p. 725–729

The title kind of says it all. The main idea is that if you have a sufficient statistic, then you can create the true probability density function (pdf) of the data from the pdf of the sufficient statistic. However, if there is no sufficient statistic, you’re out of luck, and you’d like to create a low-dimensional pdf that somehow best captures the features you want from the data. This paper proves that a certain pdf created by a projection operation is optimal in that it minimizes the Kullback-Leibler (KL) divergence. Since the KL divergence dictates the error in many hypothesis tests, this projection operation is good in that decisions based on the projected pdf will be close to decisions based on the true pdf.

This is a correspondence item, so it’s short and sweet — equations are given for the projection and it is proved to minimize the KL divergence to the true distribution. Examples are given for cases in which sufficient statistics exist and do not exist, and an application to feature selection for discrimination is given. The benefit is that this theorem provides a way of choosing a “good” feature set based on the KL divergence, even when the true pdf is not known. This is done by estimating an expectation from the observed data (the performance then depends on the convergence speed of the empirical mean to the true mean, which should be exponentially fast in the number of data points).

The formulas are sometimes messy, but it looks like it could be a useful technique. I have this niggling feeling that a “bigger picture” view would be forthcoming from looking at information geometry/differential geometry viewpoint, but my fluency in those techniques is lacking at the moment.

Update: My laziness prevented me from putting up the link. Thanks, Cosma, for keeping me honest!

Acting Like a Thief

I finally watched this documentary that was sent to me a few weeks ago called Acting Like a Thief. It is about the a street theatre organization in India from the Chhara community, a group that was labeled by the British as a “criminal tribe.” The disctrimination continues to this day. This is what taking community action via theater is about.

Also related, the Human Rights Watch report on discrimination against the Dalit community in India.