Werner Vogels, a member of the USENIX ’04 Program Committee, has written very thoughtful responses to some of my observations. And it’s clear that Werner and I see the same problem: there is insufficient industrial/academic cooperation in computer science systems research — and the lack of cooperation is to the detriment of both groups.
That said, it’s clear that there are some different perspectives as to how to address the problem. A common sentiment that I’m seeing in the comments is that it is up to industry to keep USENIX relevant (in Werner’s words, “industry will need to be more pro-active in making researchers aware of what the problems are that they need to solve”). I don’t entirely agree; in my opinion, the responsibility for keeping USENIX relevant doesn’t lie exclusively with industry — and it doesn’t lie exclusively with academia, either. Rather, the responsibility lies with USENIX itself, for it is the mission of USENIX to encourage research with a “practical bias.” As such, it is up to USENIX to assemble a Program Committee that will reflect this mission, and it is up to both academia and industry to participate as requested. This means that USENIX cannot simply wait for volunteers from industry to materialize — USENIX must seek out people in industry who understand both the academic and the industrial sides of systems research, and they must convince these people to work on a Program Committee. Now, I know that this has happened in the past — and frankly I thought that the USENIX ’04 Program Committee was a step in the right direction: where USENIX ’03 had four (of sixteen) members from industry, USENIX ’04 had six (of seventeen). But unfortunately, USENIX ’05 seems to be a marked decline in industry participation, even from USENIX ’03: the number from industry has dropped back to four (of eighteen). Worse, all four are from industry labs; where both USENIX ’03 and USENIX ’04 had at least one product-oriented member from industry, USENIX ’05 has none.
Examining these three years of USENIX brings up an interesting question: what has the Program Committee composition looked like over time? That is, is the situation getting better or worse vis a vis industry participation? To answer this question, I looked at the Program Committee composition for the last nine years.
The results are perhaps well-known, but they were shocking to me:
To me, this trend should be deeply disconcerting: an organization that has dedicated itself to research with a “practical bias” is clearly losing that bias in its flagship conference.
So what to do? First, we need some recognition from the USENIX side that this is a serious issue, and that it requires substantial corrective action. I believe that the USENIX Board should charter a committee that consists of academia and industry (both labs and product groups) in roughly equal measure. This committee should hash out some of the misconceptions that each group has of the other, clearly define the problems, develop some long-term (measurable) goals, and make some concrete short- and medium-term recommendations. The deliverable of the committee should be a report summarizing their findings and recommendations — recommendations that the Board should consider but is obviously free to ignore.
The situation is serious, and there is much work to be done to rectify it — but I am heartened by the amount of thought that Werner has put into this issue. If we can find more like him from both industry and academia, we can get the “practical bias” back into USENIX.
3 Responses
As a “Trackforward” pointer, I’ve referenced a number of your web log entries on this subject in my livejournal entry Comparing OLS and Usenix: <tt>http://www.livejournal.com/users/tytso/24445.html</tt>. From reading your entries, it’s clear we disagree on potential solutions (you seem to be against separating the academic and product-group papers into separate tracks, and I think it may be the most practical solution), and you probably have a somewhat lower opinion of the technologies in the Linux/OSS world than I would as a member of the Linux development community, but hopefully you won’t mind my using some of your observations in order to make my thesis.
At the high level, I think we are both in agreement about the general problem statement; it’s mainly a question about whether we agree about some of the details, and what the appropriate solution might be. As a Usenix board member, I will say that I certainly agree that chartering some kind of task force that tries to reconcile the multiple stakeholders: academic, industry labs, industry product engineeers, the Open Source world, etc., would be a good thing, and I will try to push for something like that happening. I imagine we will probably get some resistance from people who will argue that we have done quite enough navel gazing already when we rearchitected the 2004 Annual Technical Conference, and that more such introspection would not be useful. I’d disagree with such an observation, but I have no doubt it will be made.
Just as a thought, it would be great if your chart could break apart industry lab versus product group engineers, since I suspect it would drive the point home in an even stronger fashion. It may be hard to gather such data, since there are inevitably some fuzzy lines about who is a lab member versus a product-oriented practitioner, still, I think it would be very helpful.
It’s great to see that this problem is at least getting visibility. Yes, we disagree on the value of the technologies in the Linux world: part of the reason that OLS “feels” like USENIX from the 1990s is because the technical content is disconcertingly similar; instead of solving new problems, Linux is having to re-solve old ones — often badly. As evidence of this, I point to the RAS work in particular — Linux efforts in this domain are a complete joke to anyone who has ever cared about the reliability of a computing system. Part of the reasons that it’s important (in my opinion) to resuscitate USENIX is to filter out domains in which Linux is merely rehashing old solutions instead of crafting new ones. This academic tradition of clearly differentiating new work from past work is something that is completely absent from OLS-style conferences…