The Observation Deck

Close this search box.

Hello Joyent!

July 30, 2010

As I mentioned in my farewell to Sun, I am excited by the future; as you may have seen, that future is joining Joyent as their VP of Engineering.

So, why Joyent? I have known Joyeurs like Jason, Dave, Mark and Ben since back when the “cloud” was still just something that you drew up on a whiteboard as a placeholder for the in-between crap that someone else was going to build and operate. But what Joyent was doing was very much what we now call cloud computing — it was just that in describing Joyent in those pre-cloud days, I found it difficult to convey exactly why what they were doing was exciting (even though to me it clearly was). I found that my conversations with others about Joyent always ended up in the ditch of “virtual hosting”, a label that grossly diminished the level of innovation that I saw in Joyent. Fortunately for my ability to explain the company, “cloud” became infused with much deeper meaning — one that matched Joyent’s vision for itself.

So Joyent was cloud before there was cloud, but so what? When I started to consider what was next for me, one of the problems that I kept coming back to was DTrace for the cloud. What does dynamic instrumentation look like in the cloud? How do you make data aggregation and retention scale across many nodes? How do you support the ad hoc capabilities that make DTrace so powerful? And how do you visualize the data that in a way that allows for those ad hoc queries to be visually phrased? To me, these are very interesting questions — but looking around the industry, it didn’t seem that too many of the cloud providers were really interested in tackling these problems. However, in a conversation at my younger son‘s third birthday party with Joyeur (and friend) Rod Boothby, it became clear that Joyent very much shared my enthusiasm for this problem — and more importantly, that they had made the right architectural decisions to allow for solving it.

My conversation with Rod kicked off more conversations, and I quickly learned that this was not the Joyent that I had known — that the company was going through a very important inflection point whereby they sought a leadership position in innovating in the cloud. To match this lofty rhetoric, the company has a very important proof point: the hiring of Ryan Dahl, inventor and author of node.js.

Before getting into the details of node.js, one should know that I am a JavaScript lover. (If you didn’t already know this about me, you might be somewhat surprised by this — and indeed, there was a time when such a confession would have to be whispered, if it could be said at all — but times have changed, and I’m loud and proud!) My affection for the language came about over a number of years, and crescendoed at Fishworks when I realized that I needed to rewrite our CLI in JavaScript. And while I’m not sure if I’m the only person or even the first to write JavaScript that was designed to be executed over a 9600 baud TTY, it sure as hell felt like I was a pioneer in some perverse way…

Given my history, I clearly have a natural predisposition towards server-side JavaScript — but node.js is much more than that: its event driven model coupled with the implicitly single-threadedness of JavaScript constrains the programmer into a model that allows for highly scalable control logic, but only with sequential code. (For more on this, see Ryan’s recent Google tech talk — though I have no idea what was meant when Ryan was introduced as “controversial”.) This idea — that one can (and should!) build a concurrent system out of sequential components — is one that Jeff and I discussed this in our ACM Queue article on real-world concurrency:

To make this concrete, in a typical MVC (model-view-controller) application, the view (typically implemented in environments such as JavaScript, PHP, or Flash) and the controller (typically implemented in environments such as J2EE or Ruby on Rails) can consist purely of sequential logic and still achieve high levels of concurrency, provided that the model (typically implemented in terms of a database) allows for parallelism. Given that most don’t write their own databases (and virtually no one writes their own operating systems), it is possible to build (and indeed, many have built) highly concurrent, highly scalable MVC systems without explicitly creating a single thread or acquiring a single lock; it is concurrency by architecture instead of by implementation.

But Ryan says all that much more concisely at 21:40 in the talk: “there’s this great thing in Unix called ‘processes.'” Amen! So node.js to me represents a confluence of many important ideas — and it’s clean, well-implemented, and just plain fun to work with.

While I am excited about node.js, it’s more than just a great idea that’s well executed — it also represents Joyent’s vision for itself as an innovator up and down the stack. One can view node.js as being to Joyent was Java was to Sun: transforming the company from one confined to a certain layer into a true systems company that innovates up and down the stack. Heady enough, but if anything this analogy understates the case: Joyent’s development of node.js is not merely an outgrowth of an innovative culture, but also a reflection of a singular focus to deliver on the economies of scale that are the great promise of cloud computing.

Add it all up — the history in the cloud space, the disposition to solving tough cloud problems that I want to solve like instrumentation and observability, and the exciting development of node.js — and you have a company in Joyent that I believe could be the next great systems company and I’m deeply honored (and incredibly excited) to be a part of it!

9 Responses

  1. Pingback: Twitted by fitzage
  2. As a former {OpenSolaris Engineer, DTrace user} and a current {Cloud Engineer, node.js user}, I cannot be more thrilled to hear that you have arrived at the right place.

    The abstraction offered by node.js “feels just right” to a Unix Programmer. Using libev, v8 is a pragmatic choice for performance.

    Arguments against event based systems, such as:
    seem academic. And your arrival on this scene, I hope, will firmly set a clear course for high performance unix server code.

    Good Luck with everything !

  3. Ananth,

    Interesting point about node.js “feeling right” to the Unix programmer. I absolutely agree with you, and I think it’s a result of the same kind of minimalism that we saw with Unix. (And after all, if you agree with my potentially absurd claim that JavaScript is the dynamic C that K&R would have intended, it’s not much of a leap to assert that node.js is its Unix.) I have been interested to see some of the arguments against node.js (including a patronizing and relatively content-less screed from Alex Payne) that essentially amount to complaining that the model doesn’t work (or might not work) under all circumstances. To me, this seems to be a classic strawman: I don’t see Ryan or anyone else making the argument that all software should be rewritten in node.js — just that the node.js model seems to be one that works for one (large) class of systems…

  4. Actually there are many companies interested in solving software execution monitoring & management in the cloud.

    Enterprise Software Resource Metering & Application Performance Metric Monitoring

    Before you left Oracle I had requested a meeting with Oracle product management to explore how OpenCore’s activity based costing & metering runtime could be integrated with dtrace which I see offering unlimited meter creation complimenting OpenCore’s application activity analysis.

    Unfortunately no one responded.

    You should check out the following cloud related articles.

    You are right that no many of the cloud providers are interested in offering insight into the performance, capacity, and cost mgmt of cloud services and applications. But that will change and hopefully our push for a standard metering interface on the wire will pay off in the long run.

  5. William,

    Correct me if I’m wrong but isn’t your free advertisement comment referring to a technology that is specific to Java and Java-based applications? If not, are there public clouds that currently use this infrastructure?

  6. I had hoped you would have looked at the references provided and formed an opinion that was based on serious consideration of how we might marry both of our works in the context of cloud computing.

    I have been traveling around California for the last month meeting various vendors up and down the stack trying to convince them of the benefit of activity based costing & metering in the cloud. It is not exactly an easy task especially as it has far reaching consequences up and down stack, across process boundaries & across cloud service interactions.

    Maybe the listing of a dtrace benchmark on the site was not favorable to initiating such discussions between us but bear in mind this is in relation to the integration offered by Java runtimes which fortunately do represent a very big part of todays and future enterprise and cloud platforms.

    I would love the opportunity to discuss in person how OpenCore explicit metering model could be served by adhoc query diagnostic solutions which can offer up meters (thread counters) which can then be tied to activities closer to the application & business context.

    Our plan for OpenCore is to push for standardization w/ language & runtime mappings as well as integration on the wire for metered cloud service interactions.

    This is my last week in the bay area but maybe there is still time for us to meet in person and discuss the pro’s and con’s of both of our designs & works.

  7. Am I the only one who is sad to see a good hacker become a manager. Hope this is not Peter’s principle at work. Good luck in your new role, Bryan. I will miss seeing you create something.

Leave a Reply

Recent Posts

November 18, 2023
November 27, 2022
October 11, 2020
July 31, 2019
December 16, 2018
September 18, 2018
December 21, 2016
September 30, 2016
September 26, 2016
September 13, 2016
July 29, 2016
December 17, 2015
September 16, 2015
January 6, 2015
November 10, 2013
September 3, 2013
June 7, 2012
September 15, 2011
August 15, 2011
March 9, 2011
September 24, 2010
August 11, 2010
July 30, 2010
July 25, 2010
March 10, 2010
November 26, 2009
February 19, 2009
February 2, 2009
November 10, 2008
November 3, 2008
September 3, 2008
July 18, 2008
June 30, 2008
May 31, 2008
March 16, 2008
December 18, 2007
December 5, 2007
November 11, 2007
November 8, 2007
September 6, 2007
August 21, 2007
August 2, 2007
July 11, 2007
May 20, 2007
March 19, 2007
October 12, 2006
August 17, 2006
August 7, 2006
May 1, 2006
December 13, 2005
November 16, 2005
September 13, 2005
September 9, 2005
August 21, 2005
August 16, 2005