… nothing? So sayeth some in the business. Bill Gasarch wrote a post up last week about the “point” of the qualifying exam in which he says the two points of the process are to get students out of the program who may not be able to finish, and to make sure students are “well rounded.” While this is nice from an administrative/pedagogical point of view, I think they first question to ask is “what can you measure with a qualifying process?”
At Berkeley we had an exam after your first year — the “prelim” in your subject area. In CS (theory, at least, not sure about other areas) this was a presentation of a paper (or so I recall). You had to know it cold and be able to provide context, answer questions etc. In EE it was a 1 hour oral exam on 3 topics — two undergrad and one grad-ish related to your general sub-area. I took mine in DSP so I had to know basic DSP, more advanced DSP (filter banks etc), and stochastic processes. We spent a good part of the summer studying and in the end a little more than half of us passed, and the others had to retake the exam or got a conditional pass (meaning they had to take or TA some course). In EE, the prelim process is designed to make you learn the basics at certain level — if you can answer these questions correctly under some performance pressure and seem comfortable/fluent, then you pass. It measures how “comfortable” you are with basic (undergraduate) material.
In the Rutgers ECE department we have a 4-part exam — three oral 1-hour exams on different subjects (I got to do one exam in Communications with my colleague David Daut) and a written math exam. Students have to meet an average score threshold to pass, otherwise they have to retake some (or all) of the exams. Many take the exam in their third year, and the result of this intensive assay is that many students basically don’t do much research for the semester before their exams. In general, the material is a little more advanced, I think, than the Berkeley prelim, and focuses a but more on concepts encountered in graduate school. The exam also measure how “comfortable” you are with basic graduate material, but in a much more drawn-out manner, and later in graduate school.
Neither exam particularly measures your ability to do research, and instead focuses on core competence. The first criterion of Gasarch’s is really about whether someone will be able to complete a PhD in a reasonable amount of time; this is, in fact, a very difficult thing to measure. One approach is to say that it’s all up to the advisor, but some oversight is necessary, and having the other faculty in the department also evaluate students outside the classroom is desirable from an academic community standpoint. The CS theory exam of presenting a paper actually seems better in this regard, but then its entirely about research. Furthermore, I think that for many students, reading and understanding a paper (not of their choosing) might be a tall order very early in their career. How can we really assess whether someone would be best served getting an MS vs. a PhD?
I do think core competence is important. Getting a PhD in Electrical Engineering should mean that you have basic knowledge about some topic within EE. The alternative is that you can do research on anything, as long as the committee signs off on it, and it counts as EE. I’m not a fan of boundary drawing, but there is a value to getting students to integrate some of their undergraduate knowledge across classes (as opposed to taking the final and forgetting it), because this process of integration is also an important research skill. But the shifting nature of research areas means the cluster of topics most relevant for a solid foundation may not slot neatly into DSP/Comm/Solid State, etc. Is the problem only our disciplinary boundaries?
How does the qualifying process work in your department? Is it good, bad, or ugly?
An Ergodic Walk 
The UCSD CS “research exam” process is similar to the Berkeley CS Theory “prelim”, except that you read several papers on a coherent topic, and you have to produce a report that “synthesizes” these papers. I find it a very useful exercise, as it forces students to read papers on their research topic, think critically about them, and then write. Research exam committees I have been on generally place a lot of emphasis on the synthesis part, and ask several critical questions on the connection between the papers.
But when is this done? After the first year or after 2 years? I think there’s a difference also if it’s pre-MS or post-MS…
At Michigan in CS, there were two prelim-ish exams — I think one was called quals and one was called prelims. One was a class (at least for AI — I think it was different for different subfields) and you more or less had to take the class with its four essay-style exams and get an A. We studied for these, but I don’t remember being particularly stressed out about them, but not everyone passed the first time, so maybe I was lucky. (I did it the spring of my first year.)
The other one, which actually caused stress for students, sounds a little bit more like what you want prelims to be. You had to do a small research project (not as intense as a master’s thesis, but close) and then write a ~10 page research paper on what you had done, including a related works section. Then you had to present the research to three professors who weren’t your advisor (and at least one was not in your subfield), after which they could ask you anything about the project or the related work or other ways to solve the research problem. You didn’t so much study and abandon research as you focused primarily on doing research and, if you were doing it right (I wasn’t), getting a head start on your PhD thesis at the same time. Whether or not you passed was based on a combination of the quality of your paper and how well you answered all the questions.
I specifically remember being asked “Now, I could see you trying to solve this with a POMDP, but I’m not going to ask you to set that up because you took my class and I know you know how to do that. But if you were going to attempt to solve this with a linear program, how would you go about setting that up?” Which started out terrifying as I mentally blanked and forgot everything about POMDPs (because that class had been two years before) and wound up being a cakewalk because I had just finished taking a class on linear programming outside the department (because there wasn’t a specific class on it inside the department). Whether the professor knew this about me or not, I don’t know, but I’m fairly certain my answer to that question sealed my fate in passing.
Well, I am not sure what I want prelims to be — I think that’s the problem. You can’t really have a research exam for people who haven’t done much research, and there is a value to having a core competency exam, but the definition of what that is shifts…
UCLA Electrical Engineering has a Preliminary Exam and a Qualifying Exam, both oral. The Prelim is administered once a year and must be taken by the second year in the PhD program (usually after the MS). It consists of six 25-minute, one-on-one sessions with faculty, who test for basic undergrad and MS competency. For the Signals & Systems professors I had assigned to me (and faculty matching is a huge factor in determining success), most asked about probability rather than strictly EE concepts. I recall that the pass rate is around 2/3. The Oral Qualifying Exam is scheduled at the student’s request and passing advances one to PhD candidacy. This is basically a proposal for dissertation research to one’s PhD committee.
I didn’t feel that the Prelim Exam was particularly suited to sussing out success in EE research, but I think it has similar gatekeeper features that generic exams like the quantitative portion of the GRE do. The GRE tests competence with high school math, which is necessary but certainly not sufficient.
More info:
http://www.ee.ucla.edu/academics/programs/ph.d.-in-electrical-engineering-program-requirements