By Stephen A Chadwick 
Technology Editor 

It comes as little surprise to discover that the finance industry has been leading the way on big data. This is after all, a sector that employs mathematicians and physicists to come up with increasingly complex algorithms to manage hedge funds, FX trading strategies and risk analyses. We humans are an emotional, unpredictable lot and our judgement can be impaired by myriad things. Just this week UKIP leader and former metals trader, Nigel Farage revealed how he lost a seven figure sum in one morning when his zinc trades went pear-shaped. Nick Leeson’s singlehanded destruction of Barings is testament to what can happen when you leave a human in sole charge. 

Minimising risk is something that we tend to think computers are exceptionally good at. They don’t let “gut feelings” get in the way, they don’t think their luck might change, they’re never hung-over in the morning or a bit sleepy after a big lunch. They follow a pre-determined set of parameters and in the case of a market trade can implement a “stop-loss” should a trend change or a “sell” instruction once a desired profit has been reached. 

However, the problem with computers is that they’re only as good as the mathematical models they’ve been programmed to follow. Long-Term Capital Management is a perfect example of how blind faith in a pricing model can have disastrous consequences. LTCM employed the Black – Scholes Formula and enjoyed tremendous success initially with its LTC Portfolio. It netted impressive returns for investors of 21% in its first year, 43% in its second and 41% in its third. However, being a formula and not a crystal ball it failed to predict the 1997 Asian financial crisis or the 1998 Russian financial crisis. LTCM ended up with losses of over 4 and a half billion dollars, required Federal Reserve intervention and was eventually wound up in 2000. 

Big data’s ability to instantly recognise trends in billions of data sets in something very close to real-time allows rapid modification of these pricing models. It’s been described as “The Singularity” in stock market trading. Rather than having a trading strategy or risk assessment algorithm that is set in stone, it’s now possible to interpret vast amounts of data from seemingly unrelated sources and apply them with immediate effect. Things like weather patterns, natural disasters, election results, terrorist attacks; these all have effects on markets. Whereas algorithms can interpret and compare structured data sets such as price, yield and volume, big data can bring things like live news feeds into the equation. 

It’s not just on the trading floors of the world’s financial centres that big data is having an impact. Your High Street bank will undoubtedly be employing the services of a big data company in one form or another. You might start noticing that some of the offers on your credit card reward scheme are more interesting than usual, or perhaps you’ll get a discount voucher for a restaurant that you ate at only last month. 

Big data analytics is allowing banks to implement something known as “Personalised Product Offering”. By mining data on shopping habits, social media posts and internet transactions, big data service providers can supply banks accurate information on what products and services their customers are likely to be interested in. The banks can then target their marketing campaigns and offers with laser-like precision. On the surface that would appear to be a win-win situation for everyone involved. 

And indeed the future does look mighty bright whilst banks are talking about the huge benefits to customers, their ability to analyse data and make a rapid distinction between normal and fraudulent activity. We’ll enjoy lower banking costs, added security and a more personalised service – it’s good news all round, right? Well, not necessarily… 

How long will it be before customer segmentation goes that step further? When a personal loan is refused because two of your contacts on Twitter have County Court Judgements against them. When the historical medical records of you and your family are part of the formula used to calculate risk on a mortgage application or you’re refused a credit card because a facebook “friend” you’ve never actually met declared bankruptcy in 2004? What if you’re refused a temporary overdraft because you once listened to an S Club 7 song on Spotify? 

Alright, I exaggerate a smidge, but in order to make a point. What control do we actually have over this industrial mining of data? Who are the custodians of it? Do we trust them? Unless you live in a tent on Bodmin Moor and gather berries for sustenance, you’re leaving an everincreasing trail of data behind you, data that someone, somewhere will hoover up and use for their own purposes. But I can tell you now, without reference to a formula, algorithm or big data analytics, if you were listening to S Club 7 on Spotify, you don’t deserve a temporary overdraft.