Quals! What are they good for? Absolutely…

… nothing? So sayeth some in the business. Bill Gasarch wrote a post up last week about the “point” of the qualifying exam in which he says the two points of the process are to get students out of the program who may not be able to finish, and to make sure students are “well rounded.” While this is nice from an administrative/pedagogical point of view, I think they first question to ask is “what can you measure with a qualifying process?”

At Berkeley we had an exam after your first year — the “prelim” in your subject area. In CS (theory, at least, not sure about other areas) this was a presentation of a paper (or so I recall). You had to know it cold and be able to provide context, answer questions etc. In EE it was a 1 hour oral exam on 3 topics — two undergrad and one grad-ish related to your general sub-area. I took mine in DSP so I had to know basic DSP, more advanced DSP (filter banks etc), and stochastic processes. We spent a good part of the summer studying and in the end a little more than half of us passed, and the others had to retake the exam or got a conditional pass (meaning they had to take or TA some course). In EE, the prelim process is designed to make you learn the basics at certain level — if you can answer these questions correctly under some performance pressure and seem comfortable/fluent, then you pass. It measures how “comfortable” you are with basic (undergraduate) material.

In the Rutgers ECE department we have a 4-part exam — three oral 1-hour exams on different subjects (I got to do one exam in Communications with my colleague David Daut) and a written math exam. Students have to meet an average score threshold to pass, otherwise they have to retake some (or all) of the exams. Many take the exam in their third year, and the result of this intensive assay is that many students basically don’t do much research for the semester before their exams. In general, the material is a little more advanced, I think, than the Berkeley prelim, and focuses a but more on concepts encountered in graduate school. The exam also measure how “comfortable” you are with basic graduate material, but in a much more drawn-out manner, and later in graduate school.

Neither exam particularly measures your ability to do research, and instead focuses on core competence. The first criterion of Gasarch’s is really about whether someone will be able to complete a PhD in a reasonable amount of time; this is, in fact, a very difficult thing to measure. One approach is to say that it’s all up to the advisor, but some oversight is necessary, and having the other faculty in the department also evaluate students outside the classroom is desirable from an academic community standpoint. The CS theory exam of presenting a paper actually seems better in this regard, but then its entirely about research. Furthermore, I think that for many students, reading and understanding a paper (not of their choosing) might be a tall order very early in their career. How can we really assess whether someone would be best served getting an MS vs. a PhD?

I do think core competence is important. Getting a PhD in Electrical Engineering should mean that you have basic knowledge about some topic within EE. The alternative is that you can do research on anything, as long as the committee signs off on it, and it counts as EE. I’m not a fan of boundary drawing, but there is a value to getting students to integrate some of their undergraduate knowledge across classes (as opposed to taking the final and forgetting it), because this process of integration is also an important research skill. But the shifting nature of research areas means the cluster of topics most relevant for a solid foundation may not slot neatly into DSP/Comm/Solid State, etc. Is the problem only our disciplinary boundaries?

How does the qualifying process work in your department? Is it good, bad, or ugly?

Mental health in graduate school

I recently posted a link to an article on mental health in graduate school on Facebook (via a grad school friend of mine), and it sparked a fair bit of discussion there. The article is worth reading, and I am sure will echo with many of the readers. The discussion veered towards particularities of graduate school pressures in STEM, and the contributing factors to mental stress that are driven by funding structures and the advisor/student relationship. The starting point comes from this part of the article:

In this advisor-advisee arrangement, the student trades her labor as a researcher for the advisor’s mentorship and, ultimately, the advisor’s approval of her degree before she can graduate. For students seeking an academic position after graduate school, an advisor’s letter of recommendation can be the difference between landing a job and being left out in the cold, a harsh reality given today’s sparse academic job market. All of these factors mean that the faculty advisors hold tremendous power in the advisor-advisee relationships. They are the gatekeepers of success in the graduate endeavor.

This notion of “trading labor for mentorship” is most directly monetized in grant-funded fields like engineering, where graduate students are “working in the lab” on a project that is (hopefully) related to their thesis topic. In some cases, this works out fine, but in others, the research for the grant-relevant project does not contribute directly to their thesis. For funding agencies which want “deliverables,” this pressure to produce results on schedule creates a tension. The advisor becomes a boss.

Some of the points raised in the discussion on Facebook seemed important to bring out to a wider audience. One suggestion is to disentangle NSF support for projects and research from grad student salaries. So students could apply for NSF support and then they take their funding with them to find an advisor. In STEM this would be difficult, given the large number of international students who would not be qualified for such support, but it does give some power to students to walk away from a bad situation and more incentives for PIs to be more mentors than bosses. I am not entirely convinced it would help in terms of mental health though — students need more and better mentoring, not just the means to walk away. Also, Roy pointed out, having the student and advisor both convinced that a problem is important and solvable creates a shared commitment that helps students feel less isolated. For postdocs, though, this model would be a significant improvement over the status quo. Right now, there is almost no consensus on what a postdoc should be, and I’ve seen postdoc jobs that range from factotum to co-PI.

When one is on the other side (post-PhD), it’s tempting to say that grad school would have been easier if I had been a bit more organized or had better time-management skills. Perhaps the difficulties one has can be solved with “one weird trick.” I think that’s terribly naïve. As advisors, we definitely can do things to help students learn to work better — that’s the transition from being a student to being a researcher. But the notion that depression comes about as a result of simply not being productive enough, or feeling behind, or any other “outcomes”-based reason, misses the environmental and social factors that are equally important.

Graduate research is often very isolating. Perhaps some STEM students actually enjoy this kind of solitary work, but generalizing is dangerous. Having a grad student social organization, weekly happy hour, softball league, or other “outlet” isn’t enough. I used my startup funds to help buy a table-tennis table for my department at Rutgers, and while the students seem pretty happy about it, it’s not actually creating a community. One important question to ask is how the faculty and the department can help create and support that kind of community so that it can go on its own, organically.

In a department like mine, the majority of graduate students are international, and have a host of other stressors about being in a new (and often much more expensive) country. Using mental health resources may not be normalized in their home country or culture. Regardless of where they are from however, the big challenge is this:

…awareness of the existing resources among the graduate student population remains frustratingly low, due in part to the insular nature of traditional academic departments. A broader culture of wellness may prove even more elusive in the face of a rigidly hierarchical academic culture that often rewards drive and sacrifice without encouraging balance. In this climate, graduate student mental health advocates—students, staff, and administrators—face an uphill struggle in the years to come. The consequences of this struggle tear at the very fabric of the academic experience and suggest fundamental misalignment of priorities.

It’s only a misalignment of priorities if we don’t interrogate our priorities. This isn’t two trains crashing into each other, but it does require a “structural” recognition that graduate students are a part of the family, as it were, and treating them as such.

technical writing and the “language barrier”

One thing that strikes me about US graduate programs in electrical engineering is that the student population is overwhelmingly international. For most of these students, English is a second or third language, and so we need to adopt more “ESL”-friendly pedagogical approaches to teaching writing. I came across a blog post from ATTW by Meg Morgan from UNC Charlotte that raises a number of interesting issues. For one, the term “ESL” is perhaps problematic. The linguistic and social differences in pedagogy between other countries and the US mean that we need to use different methods for engaging the students.

In terms of teaching technical writing at the graduate level, the issues may be similar but the students are generally older — they may have even had some writing experience from undergraduate or masters-level research. How should the “ESL” issue affect how we teach technical writing?

Some tips for new research-oriented grad students

I’ve been at a lot of different institutions over the last few years, and I think that there are number of things that new graduate students in can do on their own to get them the mindset and skills to do research more effectively. An advisor is not even needed! This advice is of course oriented towards more technical/theory types in engineering, but some of it is general. Note: I say research-oriented because there are many MS programs where students don’t really care too much about research. On the one hand, this is still good advice for them, but on the other hand, they are not trying to find a PhD advisor.

  • Go to lots of seminars. This was some great advice I got from Anant Sahai when I was starting grad school. As soon as you get to grad school, sign up for all of the seminar mailing lists in your department and outside your department that you think may be interesting to you. For me it was statistics, networking/communications/DSP, one of the math seminars, and some of the CS seminars. Go to the talk, take notes, and try to understand what the problem is, why it’s important, and what tools they use to solve it. Without the right classes you may not understand the technical aspects of the talk, but you will learn about different areas of active research, how to present research (or how not to, sometimes), and new tools and terminology that may not be covered in coursework. You may see a paper referenced that you would want to look at later. Faculty will see that you’re interested in research and trying to learn something outside of class. Go to talks outside your area to learn some new things. Go to broad-audience colloquium talks to understand trends and developments across other areas of engineering outside of your interests.
  • Read papers regularly. This is hard. You’re not going to understand the papers. But much like learning a foreign language, you have to read and then make notes of things that you don’t understand and want to look up later. At first, read the abstract, introduction, model, and main results, or as much as you can handle. It will be confusing, but you will get a sense of what research is being done, what kinds of questions people ask, and so on. Bookmark the things that sound interesting so you can come back to it later. Set aside a little time every few days to do this. It’s like exercise — you have to practice regularly. Read broadly so you can get a sense of how different problems/models/questions relate to each other.
  • Learn LaTeX if you don’t know it already. There is nothing worse than trying to write your first paper and trying to learn LaTeX at the same time. You can practice by trying to write up a homework solution or two in LaTeX. In general, being familiar with the tools used in research before you actually “need” them is a great idea.
  • Learn to program. I’m still a mediocre programmer, but I’m trying to get better. Most entering grad students in ECE don’t know MATLAB beyond the level of doing homework assignments. You don’t have to become a code ninja, but learning to write and document code that others can read, and that you can debug easily, will save a lot of headaches down the road.
  • Make a website for yourself. You want to be top hit when someone searches your name and institution. It doesn’t have to have a ton of information on it, but it makes a difference. I’ve seen job candidates who somehow don’t have a homepage with information about their publications and papers. In this day and age, the first thing people are going to do after meeting you at a conference is Google you.
  • In general, entering graduate school can be quite daunting, and many students fall into the trap of just taking a bunch of classes in search of “what’s interesting.” The dirty secret is that most first-year graduate courses don’t have a lot of active research topics in them (maybe this is a problem). If you’re interested in doing research, you need to practice by expanding your horizons through going to talks and reading papers, building technical skills like programming and writing LaTeX effectively, and professionalizing by making a website to communicate your interests and research.

Starting up, and some thoughts on admissions

It’s been a busy January — I finished up a family vacation, moved into a new apartment, helped run the MIT Mystery Hunt, started teaching at Rutgers, and had two conference deadlines back to back. One of my goals for the year is to blog a bit more regularly — I owe some follow-up to my discussion of the MAP perturbation work, which I will be talking about at ITA.

In the meantime, however, one of the big tasks in January is graduate admissions. I helped out with admissions at Berkeley for 4 years, so I’m familiar with reviewing the (mostly international) transcripts, but the level of detail in transcript reporting varies widely. The same is true for letters of recommendation. I’m sure this is culturally mediated, but some recommenders write 1-2 sentences, and some write paeans. This makes calibrating across institutions very difficult. While the tails of the distribution are easy to assess, decisions about the middle are a bit tougher.

Rutgers, like many engineering school across the country, has a large Masters program. Such programs serve as a gateway for foreign engineers to enter the US workforce — it’s much easier to get hired if you’re already here. It’s also makes money for the university, since most students pay their own way. In that regards, Rutgers is a pretty good deal, being a state school. However, it also means making admissions decisions about the middle of the distribution. What one wants is to estimate the probability an applicant will succeed in their Masters level classes.

It’s a challenging problem — without being able to get the same level of detail about the candidates, their schools, and how their recommenders feel about their chances, one is left with a kind of robust estimation problem with a woefully underspecified likelihood. I’ve heard some people (at other schools) discuss GPA cutoffs, but those aren’t calibrated either. More detail about a particular individual doesn’t really help. I think it’s a systemic problem with how graduate applications work in larger programs; our model now appears better suited to small departments with moderate cohort sizes.

David Blackwell has passed away

Via Inherent Uncertainty I learned that David Blackwell passed away on July 8th.

Prof. Blackwell’s original paper (with Leo Breiman and Aram Thomasian) on the arbitrarily varying channel was an inspiration to me, and he served on my thesis committee a scant 2 years ago.

I’ll always remember what he told me when I handed him a draft of my thesis. “The best thing about Bayesians is that they’re always right.”