The Observation Deck

Close this search box.

Month: July 2004

As I mentioned earlier, I recently returned from USENIX ’04, where we presented the DTrace paper. It was a little shocking to me that our paper was the only paper to come exclusively from industry: most papers had no industry connection whatsoever, and the papers that had any authors from industry were typically primarily written by PhD students interning at industry labs. The content of the General Session was thus academic in the strictest sense: it consisted of papers written by graduate students, solving problems in systems sufficiently small to be solvable by a single graduate student working for a reasonably short period of time. The problem is that many of these systems — to me at least — are so small as to not be terribly relevant. This is important because relevance is sufficiently vital to USENIX to be embodied in the Mission Statement: USENIX “supports and disseminates research with a practical bias.” And of course, there is a more pragmatic reason to seek relevance in the General Session: most of the attendees are from industry, and most of them are paying full-freight. Given that relevance is so critical to USENIX, I was a little surprised that — unlike most industry conferences I have attended — there was no way to provide feedback on the General Session. How does the Steering Committee know if the research has a “practical bias” if they don’t ask the question?

This leads to the more general question: how do we keep the “practical bias” in academic systems research? Before I try to answer that directly, it’s worth looking at the way research is conducted by other engineering disciplines. (After all, one of the things that separates systems from the rest of computer science is its relative proximity to engineering.) To me, it’s very interesting to look at the history of mechanical engineering at MIT. In particular, note the programs that no longer exist:

  • Marine engineering, stopped in 1913
  • Locomotive engineering, stopped in 1918
  • Steam turbine engineering, stopped in 1918
  • Engine design, stopped in 1925
  • Automotive engineering, stopped in 1949

Why did these programs stop? It’s certainly not because there weren’t engineering problems to solve. (I can’t imagine that anyone would argue that a 1949 V8 was the ne plus ultra of internal combustion engines.) This is something of an educated guess (I’m not a mechanical engineer, so I trust someone will correct me if I’m grossly wrong here), but I bet these programs were stopped because the economics no longer made sense: it became prohibitively expensive to meaningfully contribute to the state-of-the-art. That is, these specialities were so capital and resource intensive, that they could no longer be undertaken by a single graduate student, or even by a single institution. By the time an institution had built a lab and the expertise to contribute meaningfully, the lab would be obsolete and the expertise would have graduated. Moreover, the disciplines were mature enough that there was an established industry that understood that research begat differentiated product, and differentiated product begat profit. Industry was therefore motivated to do its own research — which is a good thing, because only industry could afford it.

And what has happened to, say, engine design since the formal academic programs stopped? Hard problems are still being solved, but the way those problems are solved has changed. For example, look at the 2001 program for the Small Engine Technology Conference. A roughly typically snippet:

  • G.P. BLAIR – The Queen’s University of Belfast (United Kingdom)
    D.O. MACKEY, M.C. ASHE, G.F. CHATFIELD – OPTIMUM Power Technology (USA)
    Exhaust pipe tuning on a four-stroke engine; experimentation and simulation

    The Queen’s University of Belfast (United Kingdom)
    D.O. MACKEY – OPTIMUM Power Technology (USA)
    Maps of discharge coefficient data for valves, ports and throttles

    TVS-Suzuki (India)
    4 stroke gasoline engine performance optimization using statistical techniques

    TVS-Suzuki (India)
    K.V. GOPALKRISHNAN – Indian Institute of Technology (India)
    Study and development of lean burn systems on small 4-stroke gasoline engine

Note that there’s some work exclusively by industry, and some work done in conjunction with academia. (There’s some work done exclusively by academia, too — but it’s the exception, and it tends to be purely analytical work.) And here’s the Program Committee for this conference:

Of these, three are clearly academics, and seven are clearly from industry.

Okay, so that’s one example of how a traditional engineering discipline conducts joint academic/industrial research. Let’s get back to USENIX with a look at the Program Committee for USENIX ’05. Note that the mix is exactly the inverse: twelve work for a university and five work for a company. Worse, of those five putatively from industry, all of them work in academic labs. In our industry, these labs have a long tradition of being pure research outfits — they often have little-to-no product responsibilities. (Which, by the way, is just an observation — it’s not meant to be a value judgement.)

Even more alarming, the makeup of the FREENIX ’05 program committee is almost completely from industry. This leads to the obvious question: is FREENIX becoming some sort of dumping ground for systems research deemed to be too “practically biased” for the academy? I hope not: aside from the obvious problem of confusing research problems with business models, having the General Session become strictly academic and leaving the FREENIX track to become strictly industrial effectively separates the academics from the practitioners. And this, in my opinion, is exactly what we don’t need…

So how do we keep the “practical bias” in the academic work presented at USENIX? For starters, industry should be better represented at the Program Committee and on the Steering Committee. In my opinion, this is most easily done by eliminating FREENIX (as such), folding the FREENIX Program Committee into the General Session Program Committee, and then having an interleaved “practical” track within the General Session. That is, alternate between practical sessions and academic ones — forcing the practitioners to sit through the academic sessions and vice versa.

That may be too radical, but the larger point is that we need to start having an honest conversation: how do we prevent USENIX from becoming irrelevant?

So I just got back from USENIX ’04, and I had planned to spend the flight writing up some observations on the conference. Unfortunately, these observations — as pithy as they no doubt will be — will have to wait: I ended up spending the flight inhaling Bringing Down the House by Ben Mezrich. While the book itself is not very well written,1 the subject is fascinating: a well-disciplined (and apparently successful) card-counting team from MIT. The book was brain candy in the purest sense: it was exhilerating and fun — but it definitely ruined my dinner.

If you’re looking for something with a little more meat in it, check out Tom Bass’s classic, The Eudaemonic Pie. Bass’s subjects are more interesting to me, if only because the problem they’re solving is so much harder: a group of physicists and computer scientists develop a device to give them an advantage over roulette. (After all, it’s just Newtonian physics, right?) And if the idea sounds incredibly implausible, just wait until you see how they implemented it. And while the “Bringing Down the House” protagonists seem destined for a life of overpaid corporate consulting and/or 12 step programs, the leader of the “Eudaemonic” tribe, Doyne Farmer, now writes papers for academic journals like Quantitative Finance from his roost at The Santa Fe Institute. Meatier stuff, to be sure.

1The author had an incredibly difficult time separating himself from the story — I don’t particularly care if a stripper was “on his lap” for an interview, and I care even less that he knew the principal protagonist through “a friend from Harvard.” I didn’t drop fifteen bones to read “The Making of ‘Bringing Down the House'”…

Recent Posts

November 26, 2023
November 18, 2023
November 27, 2022
October 11, 2020
July 31, 2019
December 16, 2018
September 18, 2018
December 21, 2016
September 30, 2016
September 26, 2016
September 13, 2016
July 29, 2016
December 17, 2015
September 16, 2015
January 6, 2015
November 10, 2013
September 3, 2013
June 7, 2012
September 15, 2011
August 15, 2011
March 9, 2011
September 24, 2010
August 11, 2010
July 30, 2010
July 25, 2010
March 10, 2010
November 26, 2009
February 19, 2009
February 2, 2009
November 10, 2008
November 3, 2008
September 3, 2008
July 18, 2008
June 30, 2008
May 31, 2008
March 16, 2008
December 18, 2007
December 5, 2007
November 11, 2007
November 8, 2007
September 6, 2007
August 21, 2007
August 2, 2007
July 11, 2007
May 20, 2007
March 19, 2007
October 12, 2006
August 17, 2006
August 7, 2006
May 1, 2006
December 13, 2005
November 16, 2005
September 13, 2005
September 9, 2005
August 21, 2005
August 16, 2005