When the data lake feels like it’s overflowing, what can you do about it? These five recommendations will help when choosing a data platform.
You can have too much of a good thing, especially where data’s involved.
The truth is, the explosion of data — whether it’s financial data, market data, company data, economic data, historic data, real-time data or any one of the thousands of other data sources out there — is only going to increase. And by an astonishing amount.
In 2013, the IDC estimated the total amount of data in the world was 4.4 zettabytes. A figure set to rise steeply to 44 zettabytes by 2020.
Apart from its sheer volume, all this data raises many other challenges. Such as, can you trust it? Can you make sense of it? Can you use it to your advantage? And, if so, how can you find the golden nuggets that start your gold rush?
Watch video — Doug Munn — How do you manage the data you need to power your applications?
In other words, how can you make so much data work for your business, not against it?
Whenever I’m asked questions like these, my advice is to look for five key things. Five tell-tale criteria for judging data.
These are: data quality, enrichment, speed, reliability and delivery. Admittedly, these are all rather abstract terms, so let’s look at each one in more detail:
Quality data is trusted data
By quality data, I mean trusted data. Can your data deliver high quality, trusted data consistently? If you’re to have complete confidence in it, the answer must be a resounding ‘yes’.
Real-time, reference, tick by tick history, analytics, fundamentals, news and insight — with so many sources of data out there, there will inevitably be some that are spurious, irrelevant or unreliable (and occasionally all three).
You don’t want to be slowed down and distracted by having to identify, verify and eliminate data you don’t need. As the saying goes in my line of work, “bad data is bad business” — and there’s a lot of it around.
According to IBM, bad data quality costs U.S. companies alone an eye-watering US$3.1 trillion a year.
So look for data with trusted contributors and sources that have a strong reputation and a proven track record.
The breadth and accuracy of data should also give you confidence that you’ll be fully compliant with the demands of MiFID II, MiFIR, FRTB and all other relevant data handling and processing regulations.
Content alone is not enough
That’s a vital step towards data quality, yet you still need to deal with the quantity.
For your decisions to be really effective, you need more than a huge, one-dimensional spread of content.
As well as gathering huge content sets across regulated and unregulated markets, your data platform’s analytics should be able to regularize disparate formats, types and classes of information.
That way it can compare, cross-reference and find the smarter connections between them.
A platform with powerful analytics will reveal formerly hidden insights to help you make better informed, more inspired and more rewarding decisions, whether your concern is trading, analysis, investment advice, risk management or procurement.
Timing is everything
But, of course, a great insight only pays out if it’s implemented effectively.
Striking at precisely the right moment requires the certain knowledge your data has up-to-the-second accuracy, fed by the fastest possible updates and made meaningful by low-latency analytics.
This is where performance and dependable, high-speed connectivity really count. A platform that lags behind, if only by a few milliseconds, means you miss landing that big fish. Speed is the new edge.
All the data, all the time
Imagine the impact of losing your data feed for a day, an hour, even just a minute. That’s enough to make any CIO, Head of Data, or trader break into a cold sweat.
The resulting uncertainty and loss of business-critical information will completely disrupt your trading strategy, rendering you blind and unable to make any decision.
Data that leaves you guessing, even briefly, can have very costly consequences. The resilience of your data collection and distribution services is paramount.
So make sure you demand the strongest end-to-end service level agreements, with proactive service that protects the quality of delivery to your business. As we like to say, trusted, speedy, and reliable data isn’t just valuable, it’s priceless.
A la carte, not set menu
So, there you have it. A reliable source of comprehensive, up-to-the-second data, made more meaningful by intelligent, rapid analytics. And it’s all conveniently supplied as a streaming or non-streaming feed, according to the needs of your business.
Sounds like you’ve covered everything off, yes? Well, not quite.
Timely, quality data deserves flexible, specialist delivery in a format that suits whoever needs it in your organisation — from analysts and researchers to traders and developers.
In fact, it’s worth making sure any data platform you choose has an open API-enabled architecture, so your developers can build their own custom apps or have a third-party developer community they can tap into to help aggregate, distribute, manage and publish your own data in your own way.
Delivery also means ensuring traders have the information they need — on mobile or desktop add in digital and apps for cloud.
Your data delivery should be built around your specific requirements. Why adapt or restrict your methods to conform to the platform, if you can find a platform that adapts to you?
An eye on today, the other on tomorrow
Once you’ve found data you can trust to deliver everything you need, my guess is you’ll want to stick with it. Which means it will also need to be scalable, to grow as you grow.
It also needs to be easily expandable — so you can bolt on the assets that match your exact requirements — and future proof, so it can adapt to market changes, new regulations, and help you keep one step ahead of fintech disruption.
Data is your enemy, data is your friend
Your decision-making success stands or falls on how well your data serves you.
Too much disconnected and untrusted data, delivered without intelligent analysis, is counter-productive. By contrast, data that helps you discover and make connections and build relationships, empowers you to make better decisions.
The moral is, don’t be tempted to pursue and acquire data for its own sake.
Make sure data works for you and provides more insightful, pertinent answers. It can often mean the difference between profit and loss, success and failure, a pat on the back and a slap on the wrist.
Thomson Reuters Elektron Data Platform provides better content, smarter analytics and controlled distribution, the three elements that together deliver the unique insights you need to make smarter trading and investment decisions — ahead of your competitors.