Out, damned spot! Out, I say! [Pt 1]

02 March 2013

Another post that starts its life as an email response to a coworker:

What's our spot instance strategy?

We don't really have one. Nor, I suspect, do most people. If you've somehow managed to end up here, you're most likely at least passingly familiar with EC2 spot instances, a concept that sounds cooler in theory than it is, but whose practicality can reach peak awesome, if you build your systems correctly, which I'll get into.

At first glance, it seems very interesting because, hey, it's raw computing power, sold on the open market! This sounds like the future, man. About a year ago, I was working closely with a product manager at a commodities exchange on a project involving all manner of derivatives, and in discussing cloud computing at the time, it was hard not to joke about crazy, exotic leveraging schemes and complex derivative instruments, based on computing. After all, the use of the term "spot" here is borrowed directly from a related concept in financial markets.

But you'll immediately notice some differences in the market behavior though, looking at the pricing history charts available. There's no Brownian-style motion (you can also check out how Brownian motion is applied to markets, but there aren't any gifs). Pricing over time tends to resemble a step function. There are some ways in which the EC2 spot market differs from a real, actual Market which play out in odd ways.

So how does the spot market differ from, say, an equities market (think NASDAQ)? This list may lean somewhat on financial jargon, but I'll disambiguate immediately after:

• Amazon only quotes one side of the book
• Market participants do not have full depth-of-book insight (or any insight)
• Spot instance prices are necessarily chained to the on-demand (and reserved) price

Amazon only quotes one side of the book

Typical financial markets have a concept known as a market maker: an entity from whom you can both buy and sell a thing. One way to conceptualize this would be a currency exchange booth at an international airport; they have different prices for each direction of conversion, and pocket the difference, hoping to make enough money on the spread to offset the market volaltility (jay kay, they always do, especially at airports). This differs greatly from what Amazon does, because you can't buy a spot contract, wait for some upward price pressure, and unload your position for a profit--at the cost of system throughput, obviously; we are still talking about computers, believe it or not. There's nothing inherently wrong with this model, but it's not particularly market-like. That said, while I'm not a financier by training or trade, I'm pretty certain that in the absence of the other bullet points (among other things), even if you AMZN did quote both sides of the book, it would make working with spot instances a hellish, unlivable nightmare, unless you really, truly could not care less about spot availability.

Market participants do not have full depth-of-book insight (or any insight)

This is also a pretty big departure from real, adult markets. You may not actually need full depth-of-book access to trade securities. Maybe you got a hot tip from an AdWords ad in Gmail. Maybe you're timing the market. Maybe you're actually an idiot. Who knows! Point being, even free-ass Yahoo! Finance will give you top-of-book right now, and what any respectable algorithm will be doing is taking into account just what the stakes are on either side of the book.

To de-jargonate a bit, going back to bullet #1, "the book" in finance refers to the order book. Orders get placed into the exchange for something, and bids lower than the lowest offer, or offers higher than lowest bid, get stacked with each other, along with their respective volumes. Real market participants pay Cash Money to have access to exactly what the full state of the market looks like, because it drives the direction of the price. If people are looking to unload AAPL (which they have for some weeks now), this will be reflected in downward price pressure, as they'll quickly exhaust the existing Best Bids, and people will replace them with even lower bids, ad nauseum. Basically: just knowing the intentions of others in the market (at least, as much as they're willing to divulge; this too is a strategy) affects the market.

None of this nonsense exists in the spot world, and as a result, everyone just guesses how high everyone else is willing to go, and [hopefully] sets their price appropriately to how much risk they can tolerate, again in terms of system throughput. Here we are, talking about stupid computers again.

Spot instance prices are necessarily chained to the on-demand (and reserved) price

This is really the largest departure, in my opinion. For however much people are prattling on about the price of gold or Google, there is no externally-mandated value for these things. By comparison, it would make no sense to start making spot requests in a hot market, and live in constant fear of your instances being yanked, when you could just start up some plain ol' on-demand instances. This creates some odd effects, the most noticeable of which being that the market will always tend back to a very cheap equilibrium. Between the inherent risk and the fact that there's a fixed price for the underlying good, unless the standard, on-demand EC2 instances start seeing high rates of startup requests being denied due to capacity problems, it would make no sense for spots to ever stray. You'll never actually see that play out though, because that would involve both an incredible lack of capacity planning on Amazon's part, but also an overt move to not draw capacity for spot from the same pool as on-demand instances. That would not only be insane business--not to mention cutting off your nose to spite your face--but it would run counter to their [sensible] goal of consistency. You get a hefty discount just by paying upfront for reserved instnaces, allowing them to better plan capacity as it is.

There are certainly other deviations, but these are the ones that really pop out to me; put together, you get...not a market. I suspect this is why for the first few years, you'd see the term 'market' pop up in the AWS marketing, product descriptions, news articles, but its use has pretty much been scrubbed, save for some leftover Google ghosts. I didn't initially intend for this two be a two-part post, and I already have pictures to look at for the second half, but it was just too engrossing to give up the armchair markets discussion. I'll get into the actual computers in the next post.