What is B? What The Data Can Never Tell You: Games and The Flash Crash

The 2010 Flash Crash was a United States stock market crash on Thursday May 6, 2010 in which the Dow Jones Industrial Average plunged about 1000 points (about 9%) only to recover those losses within minutes. It was the second largest point swing, 1,010.14 points, and the biggest one-day point decline, 998.5 points, on an intraday basis in Dow Jones Industrial Average history.

The Flash Crash was caused by human error.

The [official] report said that this was an unusually large position and that the computer algorithm the trader used to trade the position was set to “target an execution rate set to 9% of the trading volume calculated over the previous minute, but without regard to price or time”.

And that original error was magnified by a sequence of automated knock-on effects:

The New York Times [wrote]: “Automatic computerized traders on the stock market shut down as they detected the sharp rise in buying and selling.” As computerized high-frequency traders exited the stock market, the resulting lack of liquidity “…caused shares of some prominent companies like Procter & Gamble and Accenture to trade down as low as a penny or as high as $100,000.”

Remarkably, the problem self-corrected after a few minutes. But it was not an isolated incident:

The growth of computerized and high-frequency trading in commodities and currencies has coincided with a series of ‘flash crashes’ in those markets. The role of human market makers, who can match buyers and sellers and provide liquidity to the market, is now more and more played by computer programs. If those program traders pull back from the market, then big “buy” or “sell” orders can lead to sudden, big swings. It increases the probability of surprise distortions… In February 2011, the sugar market took a dive of 6% in just one second. On March 1, Cocoa-futures prices dropped 13% in less than a minute on the IntercontinentalExchange. Cocoa plunged $450 to a low of $3,217 a metric ton before rebounding quickly. The U.S. dollar tumbled against the yen on March 16, falling 5% in minutes, one of its biggest moves ever. According to a former cocoa trader: “The electronic platform is too fast; it doesn’t slow things down” like humans would.

We have so much data, and so many smart tools for managing and manipulating it. These are tools so smart that they work automatically, without human direction or intervention. It’s like when your hand touches the cooker: you pull it away before the pain message even reaches your brain, because your nerves respond automatically, much faster than your thoughts.

But with all this data, and all these smart tools – are we cutting ourselves out of the loop too fast?

An analogy from games: dozens of companies are running thousands of A/B tests on millions of data points to try to figure out how to optimise their products.

But even though examining the data might tell you what you’re doing wrong, it cannot tell you how to put it right.

An A/B test divides players into two groups: A is the control, the normal version. B is the test, the new version. For example – you could run an A/B test which changes the way a new type of archer in Age Of Empires is introduced to the player in a tutorial (is that game still going? classic!). The games guys run versions A and B alongside each other and compare the results – checking which group used the new archer type more, were more likely to return to the game the following day, or were more likely to do more of whatever else they were looking to improve.

But if A is what you have now – the current version – then what is B?

B must be defined, built, designed by humans.

It can’t be automated. So you have to invent it yourself.

Using big data and smart tools is an art as well as a science.



Age of Empires is still going! Info on the series can be found here.

All the quotes above are from the Wikipedia article on the 2010 Flash Crash. The best stories are the true ones.

For more posts like this, follow me on Twitter – I’m @toddmgreen – or sign up for posts by email:

My German TV debut (3.1m YouTube views and counting)

I was on German TV in a police comedy sketch show, and the clip now has over 3 million views on YouTube!

The show was called Alles In Ordnung? (Is Everything In Order?), and it ran on ProSieben from 2005-06. It was pretty cool, a very dry comedy in which useless police officers would bumble through serious and not so serious incidents, from shoot-outs to health & safety infringements.

The clip went viral because it featured the game Counterstrike. Two police officers had been called to a block of flats where residents had heard shots being fired. The officers busted into a flat, only to find three students playing Counterstrike, a multiplayer shoot-em-up game, at full volume.

I was one of the students. I’d only just arrived in Germany, so my language skills were, well… still developing. It’s pretty clear at times in the clip that I don’t understand what the thickly-accented officers are saying to me – e.g. at 0:40 in the video when I look extremely confused by what people are saying to me.

The students’ flat was a mess. Curtains drawn, covered in empty beer cans and pizza boxes, and stinking because we were meant to look like we hadn’t had a shower for days. One of the original guys didn’t turn up, so I was asked to stand in. I went to the costume/make-up lady and asked: “What do we need to do to make me look right for the part?“. She looked me up and down: “No need to do anything, you’re good to go on.”

That was during my first attempt to grow a beard. There’s still some work to do there, but back in 2006 I had no idea that beard trimmers even existed – I had clumps missing where I’d been over-zealous with the nail scissors.

I’ve only just passed 20k views on this blog, so it may be some time before I match the 3.1m views on that YouTube video. But I wanted to write this post to say thanks for reading. I really appreciate it.

~ Todd

Football vs. Data Analysis

Rooney vs Montenegro 2nd half (Wired)

Wayne Rooney’s movement – England vs. Montenegro, second half (Wired)

It’s the day of the World Cup final, huzzah!

The tournament has produced a blizzard of social media-friendly infographics. Even the good old Beeb has got in on the act, and the Press Association has hired two talented footy data/visualisation chaps (cf Matchstory) to produce things like this:

Data ist sexy, ja?

I work with data every day (analysing games), and I’m season ticket holder at Luton Town – winners of last year’s (Blue Square Bet Conference) Premier League – so I was curious about how games and football data analysis compare.


In games there is a ridiculous amount of data, millions and billions of data points – level success rates, clicks on a given button, use of boosters, and much much much more. Perhaps surprisingly, it’s the same in football. At German nouveau-riche TSG Hoffenheim, sensors are attached to players, cones and goalposts, and inserted into the footballs. With ten players training for just ten minutes with three balls, those sensors will track more than 7m data points.

Reading the Matrix

When you have lots of data, it needs analysis and interpretation. I’m lucky to work with a super-talented team who can identify and understand the patterns in our data. Top football clubs work the same way. Manchester City now have eleven people crunching the numbers, so they could field a team made up entirely of analysts.

Action Men

The aim of the analysis in both fields is to come up with ideas for action – now that we know x, we should do y. We do this in games the whole time, working with the production team to decide on new AB tests and feature priorities. Same in football: Manchester City hadn’t scored from a corner for 22 games, but by switching from out-swinging to in-swinging corners (as suggested by their data team), they scored nine goals from corners in 12 games, and won the title when Vincent Kompany headed in from a corner against Manchester United.

Art + Science

Data alone will not give you the answers. I wrote about this here: What is B?. When you find a pattern or a problem in the data, and want to make a change, what do you change it to? Should we add a new booster? / Should Liverpool switch to a different shape in central midfield? Data can help you identify a problem, but it doesn’t always provide the solution. So using data is an art as well as a science.

Too much love will kill you

There is an absolute avalanche of data available in games, like in sports. So it’s easy to have too much data. Bolton’s Head of Analytic Development admitted that since their goalkeeper had started studying opposition penalty takers, he had actually saved fewer penalties – not the intended result. Sports players at their best operate in a state of flow, so over-thinking is a real risk to performance.

Ignore the ignorables

That sounds like an Ian Holloway quote, but I just made it up. In The Name Of The Rose, wise old William of Baskerville says that:

Learning does not consist only of knowing what we must or we can do, but also of knowing what we could do and perhaps should not do.

How true. With millions/billions of data points, you can’t look at everything – so you have to use instinct and experience to filter out the stuff that is of less importance or which will have less impact. Prioritisation, a constant battle! Simon Kuper, football data guy and writer, is convinced that football is in the very early stages of understanding how to use data to improve performance, because it’s pretty new and because knowing the right things to look at is not easy in a dynamic, unpredictable environment like a football match.

Measuring the wrong things

You have to look at the right things, because the wrong things can lead you astray. Alex Ferguson, grumpy erstwhile Manchester United manager, sold defender Jaap Stam in 2001 because Stam’s number of tackles was decreasing. Ferguson thought Stam was in decline – but he went on to play successfully at big clubs for several more years. It turns out that tackles are not a good yardstick for the defender’s value. Kuper points out that great defenders like Paolo Maldini actually don’t need to tackle that much, because their positional skills alone reduce opportunities for the opposing side.

The limits of data

Football is not like baseball (Moneyball) – it’s more dynamic, less structured, more anarchic; that makes it harder to apply analysis to improving performance. There’s a lot of stuff that cannot be measured or understood in a quantitative way. What does a player think or feel when they’re playing one a mobile game? That’s tough to see in the data (though there are many qualitative ways to learn the answers). If you can only manage what you measure, then the limitations of your measurements are crucial. The same applies to football: there are many limitations, even when it comes to one of the most structured elements of the game – the penalty kick. Again per Kuper, some players are predictable – e.g. at one stage, Diego Forlan alternated which side he would hit his penalties, left-right-left-right. So tracking his penalty taking would have been helpful. But other players are unpredictable: Franck Ribery mixes up his penalties seemingly at random. Unpredictability is rewarded, because it’s harder to combat – and unpredictability also reduces the usefulness of data analysis. You can’t see a pattern that’s not there.

So there you go. Several similarities – rather more in fact than I thought there would be when I started researching football data yesterday. Worthwhile further reading if you’re interested: New StatesmanThe GuardianBBC, and Wired.

My first academic publication

Read more: http://www.create.ac.uk/blog/2014/06/26/new-research-examines-ip-status-of-user-generated-contributions-to-tv-production/

Don’t mention the Work

A surprising weekend recently – we went away to visit family in Paris, and met 20+ new people, but didn’t talk about work once.

Work is such an important part of identity, and of conversation, for me and in the lives of those around me that I was somewhat surprised – and rather delighted.

That weekend not a single person asked us what we do for a living, where the office was, were we busy at the moment, how did we get into to doing that, or anything of the sort. And we in turn did not ask those questions back.

No quick-matching of a new friend with an old one who happened to have a similar job. Less shorthand, more longhand.

In his Life of Alexander The Great (and, indeed, Julius Caesar), Plutarch said that:

When a portrait painter sets out to create a likeness, he relies above all upon the face and the expression of the eyes, and pays less attention to the other parts of the body. In the same way, it is my intention to dwell upon those actions which illuminate the workings of the soul, and by this means to create a portrait of each man’s life. I leave the story of his greatest struggles and achievements to be told by others.

Meeting someone new is like inching open a box of secrets. You don’t know what will emerge first or, when it does emerge, how representative it is of what remains hidden. Knowing that person’s job or chosen career might provide a glimpse into what’s inside. But only rarely does it allow you to remove the hinges and see the whole.

So it was lovely to build up a picture of our new acquaintances more gradually – to be removed from our own professional identities – by exchanging not a single word about work.

I can’t remember the last time that happened. Really.

Plutarch has the last word:

My preamble [to the Life] shall consist of nothing more than this one plea: if I do not record all their most celebrated achievements or describe any one of them exhaustively, but merely summarise for the most part what they accomplished, I ask my readers not to regard this as a fault. For I am writing biography, not history, and the truth is that the most brilliant exploits often tell us nothing of the virtues or vices of the men who performed them, while on the other hand a chance remark or a joke may reveal far more of a man’s character than the mere feat of winning battles in which thousands fall, or of marshalling great armies, or laying siege to cities.


Photo credit: Laura Liberal.

For more posts like this, subscribe by email or follow @toddmgreen on Twitter.

Calvin and… sorry, what did you say?

I’m guilty of this, I think we all are. Listening > Emails.

Leaving West Ealing

Balcony view over Drayton Green… *sigh*…

The bare-chested hooligan next to me threw a water bottle filled with suspiciously yellow fluid towards the stage, so I decided to squeeze away from him. Oasis, live at Wembley Stadium, 22 July 2010.

The next day I was at the Ealing Blues Festival. Elderly patrons bopped behind us while families picnicked and middle-aged rockers nodded sagely to the beat. Less than ten miles away in the same city, but the two gigs were a world apart.

Ealing is a village surrounded by a city. London keeps its distance. Since moving here four years ago, my wife and I have settled happily into life in West Ealing, and now that we’re moving out we have created a lengthy Bucket List of places to revisit: Santa Maria, The Red Lion, Crispins, Mamas, Brent Lodge Park (where we got engaged, at the heart of the maze), The Village Inn, the canal walk and the Osterley Locks. If the list looks a little pub-heavy… well, that’s because we made it at The Drayton Court, our second living room.

Now we’re moving up to Hertfordshire, driven out (like so many young couples) by the British urge to buy and the difficulty of finding somewhere affordably spacious in Ealing. We always knew that we wouldn’t be able to buy here, yet we stayed because we fell in love with the community feeling we sensed on that July day back in 2010. We’ve been fortunate to make local friends, and through them we’ve developed a sense of belonging that most people miss when they move to London after finishing university. I’ve spent many years living in west London; Ealing is the first place I’ve ever bumped into someone I know in the street.

Now it’s time to buy, and it’s time to go. To some friends it seems that the south-east is divided into two halves: there’s London, and there’s outside London – so it’s a big thing when we tell them we’re going to move out. But for us, we left London four years ago. Ealing is a village surrounded by a city. And we can’t wait for this year’s Blues Festival.


This post originally appeared in Ealing Today.

For more posts like this, subscribe by email or follow @toddmgreen on Twitter.

“Cambridge United, We’re Coming For YOU”


My beloved Luton Town won the Football Conference (the 5th tier of English football) in fine style yesterday, with 101 points and 102 goals to boot.

At one stage they were 10 points behind main rivals Cambridge, and all seemed lost. I remember being at a game last autumn when “Cambridge United, we’re coming for you” echoed from the stands, but there was still a long way to go.

So – how did they win it?

I was curious. And when I’m curious, I make charts.

Here’s a chart showing the cumulative points totals of the two teams:

As you can see, Cambridge were some way ahead until about game #26, when Luton drew level, then overtook.

But the real story here is that Cambridge’s form changed more than Luton’s. Luton’s trajectory trended gently upwards from around game 19 onwards, but Cambridge’s nosedived from around game 16 and took about 20 weeks to recover. So if Cambridge had continued at the same pace as they set at the start of the season, they would have won the title comfortably.

We can see this more easily if we look at a second chart. This one shows each team’s average points per game, using a five-game moving average:

Here the trends are clearer. Luton started slowly but then really got very strong – meanwhile, Cambridge went downhill: the red line drops and drops up to game 31, and their recovery was too little, too late.

I wanted to know how Luton’s cumulative points compared to previous winners. So I made a third chart: this one shows the game-by-game cumulative points for Luton and Cambridge this year, and for the champions of the previous three years:

So in fact Luton only got the third highest total – two other teams also topped 100 points, and they got even more than the Town. Luton actually tailed off somewhat at the end compared to Fleetwood and Crawley in their championship seasons.

What’s most striking here is that Cambridge actually had an even stronger trajectory than any of the winners from the past four years! Their points total (the red line) is the highest of all up to game 21. But as that red line shifts sideways rather than upwards, you can see how Cambridge fell away after the first 1/3rd of the season. The other four teams all won the league.

So, what distinguishes a champion?

And is it possible to predict a champion from their form as the year goes on?

Here’s one more chart, showing the average points per game for Luton, Cambridge, and the previous three winners:

Well well well. There’s a clear jump up in average points per game for all the champions from about game 18. Before then, they averaged around 2.0-2.5 points. After then, the team that wins the league consistently averages 2.5-the max 3.0 points per game on a five-game average.

So if you like a gamble, look out for that next year – that’s how to predict the next Conference champions.

Hard luck to Cambridge – perhaps they will get to League Two next year through the play-offs. But whoever you support, thanks for reading!

~ Todd


For more posts like this, subscribe by email or follow @toddmgreen on Twitter.



Me: “I’m writing a post about minimalism”

My wife: “Keep it short”


We have more books and DVDs and clothes than we need. So I donated my last birthday to charity:water instead, and they just emailed to say the money is at work in Tanzania.

I’m getting very interested in the idea of minimalism.

Current minimalist activities:

1. Inbox Zero

This means getting your unread message count done to zero. I have managed it three times in the past month, can do better. To do so I am:

  • Replying to / Marking as read / Deleting everything straight away – otherwise I read emails more than once
  • Scheduling email blocks in my day to make time for this
  • Using Mailbox on my phone to archive emails easily
  • Aiming to stop using my inbox as a to-do list

2. Trello Zero

I now use Trello as my to-do list. To some extent I have simply shifted to-dos from inbox -> Trello. So I am also attempting to break my habit of turning every idea into a to-do item.

3. Moving house

This is a great catalyst for minimalism. It is a pretty extreme technique – we are moving for other reasons, not as part of my experiments with minimalism! But the pain of moving house is somewhat proportional to the amount of stuff you have to move, so less stuff = less hassle.

We are moving house in four weeks, so a major declutter is underway. We plan to remove anything we haven’t used in the past year, or won’t use in the next year. This morning alone we took five bags of books, DVDs and clothes to the charity shop. Last time we moved we recycled / gifted / binned 15% of our stuff. This time we’re aiming for 25%.

4. Workspace clutter

Physical – notes everywhere = distraction. Any colleagues reading this will laugh, as I usually keep an untidy desk. But they may notice that my desk is now 100% clear – I took all my notes home with me to sort out last week :)

Digital – my desktop now gets a regular clean-up. I have turned off a lot of push notifications. I am also a ruthless unsubscriber.

And now: breathe…

If you’re interested in minimalism, start here: Zen Habits – http://zenhabits.net/


Image credit: Billy Lam

For more posts like this, subscribe by email or follow @toddmgreen on Twitter.


I did a marathon… Aside from that, here’s what surprised me

I finished the Manchester Marathon in 3 hours 38 mins. That’s how I felt at the end.

If you are bloody-minded enough to do the same, there’s masses of advice on the internet already. So instead of repeating or summarising, here’s a list of things that surprised me.

Surprise #1: Starting fast worked well

The night before the race I made a last-minute change of plan. I decided to start fast. That was the opposite of what everyone told me to do: you’re supposed to start slow, then speed up towards the end if you can manage it. Pah! I knew I’d be knackered at the end and wouldn’t want to speed up even if I could. I’m 100% certain that I got a better time as a result.

Surprise #2: Gel packs to the max

I showed up with two energy gel packs, but the experienced-looking runners had bandoliers full of them. I had brought too few. By the end I was dependent on well-wishers’ jelly babies – goddamn it I loved those little guys. A shot of sugar straight to the bloodstream. Would’ve liquified and injected them if I could.

Surprise #3: Pain, pain, go away

I was fine for the first 5 miles, then my right leg started to stiffen up. Not good, that was way too soon. I decided to push on at the same pace and hope it went away. It did – but then my left ankle started to hurt. That stopped about the same time as my arms started aching (arms, wtf?). And so on and so on. Different bits hurt at different times, you’ve just got to roll with it. Had a couple of painkillers in my pocket, and finally gave in and took one at about 15 miles. It made no noticeable difference to my body, but having painkillers with me helped psychologically.

Surprise #4: It’s a race against the course and the clock, not against the other runners


The consequence of starting fast is that you will inevitably slow down as the race progresses. Even those going at a steady pace – never mind the freaks who are speeding up – will begin to overtake you. This is not a good thing psychologically. I felt like I was going backwards from about 9 miles in. Runners streamed past me, like I was like driving at 40mph in the middle of the motorway. It took me a mile or so to reset: I’m running my own race, for my own time; I don’t need to beat all these people.

Surprise #5: Obsession with my split time

My gradual slow-down was measured in precise detail by my Nike+ app. Every kilometre a robotic American lady told me how long I had been running, how much distance I had covered, and what my average time per kilometre was. Time per kilometre was my main guide. I knew that an average of 4:59/km = 3 hrs 30 mins, that 5:19 = 3 hrs 45 mins, and 5:39 = 4 hrs 00 mins. I started out around 4:51 per km, but I knew I couldn’t maintain that pace. The average km time kept creeping up. As I passed the halfway mark, I worked out that I’d need to stay under 5:13 to finish in under 3 hrs 40 mins. As the average time per km crept up, I got more and more nervous – 5m05s, 5m06s, 5m07s… every time a the American lady started on a new kilometre announcement I whispered a silent prayer that the average pace would not have increased. The battle lines were drawn: I had to slow down my slow-down.

Here’s my pace chart, showing speed per km. Thankfully I managed to stay just ahead of the 3 hrs 40 mins pace. I averaged 5:12/km for the marathon as a whole.

I didn’t feel like I hit the famous ‘Wall’ at any specific point – but looking at this chart, I guess it was at around 16 miles, when I started to shift down before stabilising at a lower pace. Coincidentally, it was at around that point that I decided I would never run a marathon again.

Surprise #6: Terry Prachett, runner’s friend

I spent ages beforehand crafting a lengthy, pumped-up playlist for the race, and deliberately avoided the songs on it so that they would sound fresh on the day. But by the time I reached halfway through the race, I was bored. Even I can only listen to so much Britpop. So I switched to an audio book, and found salvation. I can recommend Terry Prachett’s The Night Watch as a pleasant distraction to all future marathon runners – though I have had to re-listen to the chapters that played during the last few miles, as somehow I don’t seem to have followed that part of the story.

Surprise #7: Random supporters are there for you

Thousands of people lined the route and cheered the runners on. That really helped, much more so than I expected. I managed a weak thumbs up to most of those who shouted for me personally. Pro tip: write your name on your top in big letters – it will substantially increase your share of random personalised encouragement.

Surprise #8: Runners’ cameraderie

Another nice surprise. It’s daunting to see paramedics treating stricken runners, and it’s nerve-wracking to see ambulances racing past you to some unknown pain point – what lurks just a few miles ahead? But those who were still standing would actively encourage fellow runners who were in trouble. Many people began to walk near the end, but they got regular pats on the back and kind words in the ear from those passing them – come on mate, nearly there now!

Surprise #9: Time and space warp towards the end

The last 10km seemed like entire marathons that had been surreptitiously added on to the main event, and the same for the last 3km. I literally could not imagine how I was going to run that far. It boggled my mind, the sheer thought was exhausting. (This is rather ridiculous in hindsight, because by the time you’ve only got 3km to go you have already run 39km!). So I tried to trick myself. The last 15km became three sets of 5km, a distance which I know I can do ok. But when the first kilometre of the first set of 5km took what seemed like half an hour, I realised that my targets had to get shorter. By the time I got into the final kilometre, I was running two traffic cones at a time – just get to that one… now just get to that one… And then for the final 500m, all I could aim for was to get from one group of people to the next – just get to the girl in the pink coat… now just get to the guy in the glasses… come on, nearly there…

Surprise #10: What’s next?

You can’t train for a marathon just by running a bit further each time. I ran 220 miles over 6 months in training – but I also had to cross-train (interval training, hill running), do weights in the gym for the first time ever (intimidating!), eat more healthily, drink less beer, etc. Without a specific goal in mind, I’m rather lacking in direction sportwise – I need a new (and preferably less gruelling) challenge!

Here’s my training plan for the final three months.

For now, I’m happy to fill the time with writing :)

So thanks for reading!

~ Todd


For more posts like this, subscribe by email or follow @toddmgreen on Twitter.