The Observation Deck

Close this search box.

DTrace, Leopard, and the business of open source

October 29, 2007

If you haven’t seen it, DTrace is now shipping in Mac OS X Leopard. This is very exciting for us here at Sun, but one could be forgiven for asking an obvious question: why? How is having our technology in Leopard (which, if Ars Technica is to be believed, is “perhaps the most significant change in the Leopard kernel”) helping Sun? More bluntly, haven’t we in Sun handed Apple a piece of technology that we could have sold them instead? The answer to these questions — which are very similar in spirit to the questions that were asked over and over again internally as we prepared to open source Solaris — is that they are simply the wrong questions.

The thrust of the business questions around open source should not be “how does this directly help me?” or “can’t I sell them something instead?” but rather “how much does it cost me?” and “does it hurt me?” Why must one shift the bias of the questions? Because open source often helps in much more profound (and unanticipated) ways than just this quarter’s numbers; one must look at open source as long term strategy rather than short term tactics. And as for the intense (and natural) desire to sell a technology instead of giving away the source code, one has to understand that the choice is not between “I give a customer my technology” and “I sell a customer my technology”, but rather between “a customer that I never would have had uses my technology” and “a customer that I never would have had uses someone else’s technology.” When one thinks of open source in this way, the business case becomes much clearer — but this still may be a bit abstract, so let’s apply these questions to the specific case of DTrace in Leopard…

The first question is “how much did it cost Sun to get DTrace on Leopard?” The answer to this first question is that it cost Sun just about nothing. And not metaphorical nothing — I’m talking actual, literal nothing: Adam, Mike and I had essentially one meeting with the Apple folks, answering some questions that we would have answered for anyone anyway. But answering questions doesn’t ship product; how could the presence of our software in another product cost us nothing? This is possible because of that most beautiful property of software: it has no variable cost; the only meaningful costs associated with software are fixed costs, and those costs were all borne by Apple. Indeed, it has cost Sun more money in terms of my time to blog how this didn’t cost anything to Sun than it did in fact cost Sun in the first place…

With that question answered, the second question is “does the presence of DTrace on Leopard hurt Sun?” The answer is that it’s very hard to come up with a situation whereby this hurts Sun: one would have to invent a fictitious customer who is happily buying Sun servers and support — but only because they can’t get DTrace on their beloved Mac OS X. In fact, this mythical customer apparently hates Sun (but paradoxically loves DTrace?) so much that they’re willing to throw out all of their Sun and Solaris investment over a single technology — and one that is present in both systems no less. Even leaving aside that Solaris and Mac OS X are not direct competitors, this just doesn’t add up — or at least, it adds up to such a strange, irrational customer that you’ll find them in the ones and twos, not the thousands or millions.

But haven’t we lost some competitive advantage to Apple? Doesn’t that hurt Sun? The answer, again, is no. If you love DTrace (and again, that must be presupposed in the question — if DTrace means nothing to you, then its presence in Mac OS X also means nothing to you), then you are that much more likely to look at (and embrace) other perhaps less ballyhooed Solaris┬átechnologies like SMF, FMA, Zones, least-privilege, etc. That is, the kind of technologist who appreciates DTrace is also likely to appreciate the competitive advantages of Solaris that run far, far beyond merely DTrace — and that appreciation is not likely to be changed by the presence of DTrace in another system.

Okay, so this doesn’t cost Sun anything, and it doesn’t hurt Sun. Once one accepts that, one is open to a much more interesting and meaningful question: namely, does this help Sun? Does it help Sun to have our technology — especially a revolutionary one — present in other systems? The answer is “you bet!” There are of course some general, abstract ways that it helps — it grows our DTrace community, it creates larger markets for our partners and ISVs that wish to offer DTrace-based solutions and services, etc. But there are also more specific, concrete ways: for example, how could it not help Solaris to have Ruby developers (the vast majority of whom develop on Mac OS X) become accustomed to using DTrace to debug their Rails app? Today, Rails apps are generally developed on Mac OS X and deployed on Linux — but one can make a very, very plausible argument that getting Rails developers hooked on DTrace on the development side could well the change the dynamics on the deployment side. (After all, DTrace + Leopard + Ruby-on-Rails is crazy delicious!) This all serves as an object lesson of how unanticipatable the benefits of open source can be: despite extensive war-gaming, no one at Sun anticipated that open sourcing DTrace would allow it to be used to Sun’s advantage on a hot web development platform running on a hip development system, neither of which originated at Sun.

And the DTrace/Leopard/Ruby triumvirate points to a more profound change: the presence of DTrace in other systems assures that it transcends a company or its products — that it moves beyond a mere a feature, and becomes a technological advance. As such, you can be sure that systems that lack DTrace will become increasingly unacceptable over time. DTrace’s shift from product to technological advance — just like the shifts in NFS or Java before it — is decidedly and indisputably in Sun’s interest, and indeed it embodies the value proposition of the open systems vision that started our shop in the first place. So here’s to DTrace on Leopard, long may it reign!

12 Responses

  1. I’m also happy to see DTrace on OS X for the reasons you mention. Apple has shown that it has the guts to buck the "not invented here" trend to adopting new technology with it’s decision to use BSD a few years back. Today, their integration of DTrace and eventual integration (read-only for now) of ZFS shows they haven’t wavered in that stance – bravo!
    That said, I think Solaris is making massive progress on the desktop, to the extent that I’m not sure my 2002 iMac will be getting replaced with another Mac, but Apple, feel free to convince me otherwise.
    Oh and speaking of the cost of software, there’s a weird pricing model with Leopard here in Europe, hint $129 != 129EUR.

  2. I think DTrace is so cool, it’s a profound contribution to the world, a tool made by engineers for engineers. It kind of reminds me of when I first discovered oscilloscopes and, later, spectrum analyzers.
    However I’ve always found the arguments for open source to be convoluted and in direct contravention of Occam’s Razor, and yours is unfortunately no exception. The elephant in the room, of course, is the popularity of the Windows and Linux operating systems. Within that context it makes perfect sense that Sun as a systems company would open source its software. Loss lead the software and make money on the hardware and support services.
    "As such, you can be sure that systems that lack DTrace will become increasingly unacceptable over time."
    And what could be better than to pick up OpenSolaris with DTrace for free and run it on an x86 whitebox? Of course, if the customer happens to care about heat and power consumption per CPU cycle, as your CEO Jonathan is quite fond of talking about, then perhaps a Nigara 2 based system would be a better fit. And once you give a CIO some fast, cool, servers he’s going to want some storage, perhaps a Thumper? And if you give a CIO some fast servers and lots of storage he’ll want to optimize his investment so he’ll fire up DTrace. And if you give a CIO DTrace he’ll want some help interpreting the numbers, and where better to go than the inventor of all those technologies?

  3. Hey Andrew,
    I strongly agree with your first, third and fourth paragraphs. ­čśë And actually, I don’t think that your second paragraph invalidates your other reasoning. It all comes down to the fact that we have confidence in our ability to deliver and support innovative systems, not just innovative bodies of code (though that too), so the presence of Windows and Linux doesn’t really change our perception of the value of open source…

  4. The DTrace on linux concept needs a rest for the moment, but the concept of DTrace on MS-Windows…. *that* would prove interesting. If Microsoft ever ported DTrace – and released that port – then everybody whose OS of choice competes with them would have to really make a quantum leap in order to overtake MS.

  5. "…and becomes a technological advance. As such, you can be sure that systems that lack DTrace will become increasingly unacceptable over time"
    Isn’t that argument based on the assumption that DTrace will NOT be ported to Linux, at least in the near future, because of license incompatibility? What, in your opinion, would change if DTrace is ported to Linux or Windows in about one year? I know it is a long shot, but I wonder what your point of view about this hypothetical scenario is. By the way, how long did it take since your meeting with Apple engineers and OS X DTrace implementation?

  6. Antonello: I don’t think BMC’s assumption is that DTrace won’t be ported to linux. If "the linux community" wants to do a serious DTrace port then I’m sure that Team DTrace would be very glad to help. I know that members of the DTrace community would like to see DTrace on linux. If DTrace was ported to linux that would be a good thing. Same goes for MS-Windows.

  7. Antonello,
    Just what James said: I’m not assuming that it wouldn’t be ported to Linux (as we have said quite a few times, we believe that such a port is both technically and legally possible, and that the license issue is with the GPL, not the CDDL), quite the contrary in fact: my point was that the presence of DTrace in a variety of systems allows users to start expecting it everywhere, and thus in the long term Linux will HAVE to port DTrace if they wish to remain competitive.
    As for how long the port took, you would need to ask Apple: by the time we had our meeting with them, it was pretty clear that the port was well underway, and that they were just trying to clear up some niggling issues. Certainly, it wasn’t long after that meeting that they announced to us that they had DTrace working and that they intended to ship it in Leopard…

  8. Having DTrace on MacOS X Leopard looks good indeed, for Solaris, MacOS X and developers using at least one of those, for the reasons stated here ­čÖé
    > and that the license issue is with the GPL, not the CDDL
    Hmm. The CDDL came after the GPL(v2 at the time), and it’s incompatible with it. Fact, as far as I’ve read on multiple places.
    This does not necessarily mean that the CDDL was explicitely designed in order to be incompatible with the GPL(v2) (although many people think that it was, and I rather agree with that), but this at least means that the CDDL was not designed in order not to be incompatible with the GPL(v2).
    And who benefits of the licensing trouble between GPL, CDDL (and BSD to a lesser extent), which delays/prevents progress in the open, for everyone’s benefit ? Microsoft (R)(TM) Windows (R)(TM) and the ecosystem surrounding it, period.
    Lowering the licensing barriers would be A Good Thing, and all parties involved (Sun, Linux kernel + Systemtap people) could do more for that to happen.
    On the technical side, now:
    I guess you developers of DTrace keep on adding features to DTrace ?
    Judging from
    Systemtap is supposed to have (and has partial or complete programming for) some features that DTrace doesn’t have, especially in the scripting language itself. Some of these features don’t look impossible to add in DTrace, don’t they ?
    (since Adam Leventhal edited the comparison, the information on the page should be correct enough)

  9. Lionel,
    The timing of the GPL vs. the CDDL is not terribly relevant — this is software licensing, not calling shotgun. As for why we developed CDDL instead of GPL or BSD, the reasons are philosophical: unlike BSD, we believe in copylefting; unlike GPL, we believe in allowing proprietary derived works — but we wanted to allow them in a way that didn’t create a patent mess. Personally, I happen to think the CDDL is a very accurate reflection of the values of Solaris — and I think many of the supposed license incompatibilities are invented as a cover for NIH.
    As for SystemTap, the differences here, too, are philosophical: the lack of "full" control structures in DTrace, for example, actually represents our value of safety above all else. (Safety is considered a goal — not a constraint — of SystemTap.) As for their criterion "aggregate value readable by script", I can tell you that SystemTap’s answer doesn’t square with their assertion that they have scalable aggregations: in the DTrace architecture, anyway, it is the infinite scalability of aggregations (both in terms of concurrency and time) that makes it impossible to act on aggregation values in probe context.
    Finally, while Adam was ultimately permitted to contribute to the wiki to correct its most glaring errors, the wiki remains very much a SystemTap construction; it is designed to make SystemTap look much more competitive with DTrace than it in fact is. More balanced comparisons can be found by googling "dtrace systemtap"…

  10. I haven’t used DTrace yet but I understand the fundamental advantage it will give to developers coding on Leopard. I would just like to thank you (and the other DTrace developers) for creating DTrace in the first place. You make it sound like you didn’t have a whole lot to do with the Apple port, but the fact that the software exists in the first place is a tribute to the efforts of the hard-working programmers at Sun. And without the original code, there could be no port.

  11. I first saw DTrace used at Usenix by this guy named "Jim Mauro"; my first thought "this is sooo COOL!". With DTrace now in Leopard I can more easily spend time learning how to best use this awesome tool to make my work life more productive, Thanks SUN!

Leave a Reply

Recent Posts

November 18, 2023
November 27, 2022
October 11, 2020
July 31, 2019
December 16, 2018
September 18, 2018
December 21, 2016
September 30, 2016
September 26, 2016
September 13, 2016
July 29, 2016
December 17, 2015
September 16, 2015
January 6, 2015
November 10, 2013
September 3, 2013
June 7, 2012
September 15, 2011
August 15, 2011
March 9, 2011
September 24, 2010
August 11, 2010
July 30, 2010
July 25, 2010
March 10, 2010
November 26, 2009
February 19, 2009
February 2, 2009
November 10, 2008
November 3, 2008
September 3, 2008
July 18, 2008
June 30, 2008
May 31, 2008
March 16, 2008
December 18, 2007
December 5, 2007
November 11, 2007
November 8, 2007
September 6, 2007
August 21, 2007
August 2, 2007
July 11, 2007
May 20, 2007
March 19, 2007
October 12, 2006
August 17, 2006
August 7, 2006
May 1, 2006
December 13, 2005
November 16, 2005
September 13, 2005
September 9, 2005
August 21, 2005
August 16, 2005