What is B? What The Data Can Never Tell You: Games and The Flash Crash

The 2010 Flash Crash was a United States stock market crash on Thursday May 6, 2010 in which the Dow Jones Industrial Average plunged about 1000 points (about 9%) only to recover those losses within minutes. It was the second largest point swing, 1,010.14 points, and the biggest one-day point decline, 998.5 points, on an intraday basis in Dow Jones Industrial Average history.

The Flash Crash was caused by human error.

The [official] report said that this was an unusually large position and that the computer algorithm the trader used to trade the position was set to “target an execution rate set to 9% of the trading volume calculated over the previous minute, but without regard to price or time”.

And that original error was magnified by a sequence of automated knock-on effects:

The New York Times [wrote]: “Automatic computerized traders on the stock market shut down as they detected the sharp rise in buying and selling.” As computerized high-frequency traders exited the stock market, the resulting lack of liquidity “…caused shares of some prominent companies like Procter & Gamble and Accenture to trade down as low as a penny or as high as $100,000.”

Remarkably, the problem self-corrected after a few minutes. But it was not an isolated incident:

The growth of computerized and high-frequency trading in commodities and currencies has coincided with a series of ‘flash crashes’ in those markets. The role of human market makers, who can match buyers and sellers and provide liquidity to the market, is now more and more played by computer programs. If those program traders pull back from the market, then big “buy” or “sell” orders can lead to sudden, big swings. It increases the probability of surprise distortions… In February 2011, the sugar market took a dive of 6% in just one second. On March 1, Cocoa-futures prices dropped 13% in less than a minute on the IntercontinentalExchange. Cocoa plunged $450 to a low of $3,217 a metric ton before rebounding quickly. The U.S. dollar tumbled against the yen on March 16, falling 5% in minutes, one of its biggest moves ever. According to a former cocoa trader: “The electronic platform is too fast; it doesn’t slow things down” like humans would.

We have so much data, and so many smart tools for managing and manipulating it. These are tools so smart that they work automatically, without human direction or intervention. It’s like when your hand touches the cooker: you pull it away before the pain message even reaches your brain, because your nerves respond automatically, much faster than your thoughts.

But with all this data, and all these smart tools – are we cutting ourselves out of the loop too fast?

An analogy from games: dozens of companies are running thousands of A/B tests on millions of data points to try to figure out how to optimise their products.

But even though examining the data might tell you what you’re doing wrong, it cannot tell you how to put it right.

An A/B test divides players into two groups: A is the control, the normal version. B is the test, the new version. For example – you could run an A/B test which changes the way a new type of archer in Age Of Empires is introduced to the player in a tutorial (is that game still going? classic!). The games guys run versions A and B alongside each other and compare the results – checking which group used the new archer type more, were more likely to return to the game the following day, or were more likely to do more of whatever else they were looking to improve.

But if A is what you have now – the current version – then what is B?

B must be defined, built, designed by humans.

It can’t be automated. So you have to invent it yourself.

Using big data and smart tools is an art as well as a science.



Age of Empires is still going! Info on the series can be found here.

All the quotes above are from the Wikipedia article on the 2010 Flash Crash. The best stories are the true ones.

For more posts like this, follow me on Twitter – I’m @toddmgreen – or sign up for posts by email:

Author: toddmgreen

I really like making internet projects. I work on apps, games and websites at a TV company. I write stuff, make stuff, and accidentally break stuff. You should probably follow me on Twitter - @toddmgreen.

What do you think?