Data is finally growing up. For years, businesses and even governments have been keen to collect, store and pick data apart for valuable insights.
Though data analysis has had different names over time, the concept is an old one. But data has come on in recent years. Because of online advances and an increasingly connected world population, the amount of information being generated by individuals, businesses and other organisations is colossal.
People tweet, shop and bank online. And large organisations, from governments to supermarkets, are busy tracking events and activities. This could be totting up sales figures or looking at consumer demographics and spending habits.
Tech giant IBM claims that, every day, 2.5 quintillion bytes of new data are generated. But the second advance is that data analysis is becoming more sophisticated.
And firms, as well as others in worlds as diverse as science and governance, are starting to wake up to the possibilities.
Dr Shirley Ann Jackson, president at Renssellaer Polytechnic Institute and
one of Barack Obama’s advisers on
science, technology and innovation, says the potential benefits are huge – but involve work.
“We have this explosion of data, coming from multiple places,” she says. “There’s the internet but also the internet of things and social media, and we have to make sense of that.
“Years ago people used to talk about text mining. Then it was data mining, and this has evolved into big data.
“We now have more analytical capability, and the more data one has, the more informed one can be in making decisions. This could involve anything, such as tuning a manufacturing process.”
Big data involves analysis, but this needs to be translated into something that non-experts can understand.
Allen Bonde, who set up the Small Data Group, says insights need to be presented in a way anyone can grasp.
He says: “With data, it’s like in The Hitchhiker’s Guide to the Galaxy where you are looking for a number which is the meaning of life.
“But with big data, you need to know that number, but also how that number was derived. And you then need to explain the insights from that in a way everyone can understand.”
Super computers and analytics tools make more sophisticated work possible. But a new workforce needs to be developed.
“There are some distinctive skill sets in dealing with this,” Bonde says. “There is now the idea of the data scientist.
“That involves the analysis of data at the back end. But you also need someone who has reporting and analytical skills.”
Jackson adds: “People are now turning to experts to aggregate and organise data, and develop analytical approaches to extract useful information.
“But big data also requires an interdisciplinary approach, and bringing people together from everything ranging from computer science to engineering, to cognitive science, to linguistics and logistics.”
As people turn to experts for data insights, an industry is forming, with its own issues.
Dr Taha Yasseri, big data research officer at the Oxford Internet Institute, believes that as data analysis becomes more of a science, big data professionals will have to weigh up ethical questions in the same way other disciplines do.
He says: “There is an emerging area of data science, and we will face ethical issues in the same way we have around medicine and biological research.”
Decades on from the early days of data, the work around analysing it has some way to go. But the opportunities cannot be ignored.