Big Data is accelerating our transition to a digital economy. Here’s a look at how it’s changing the way we live, work and play.
The US presidential election of 2012 was a tough fight. Democratic candidate Barack Obama pitched a message of hope and change against Republican candidate Mitt Romney amid a backdrop of an existing war in the Middle East and escalating international tensions. But Obama’s campaign team had a secret weapon – Big Data, something other electioneers had never before depended on so heavily.
What had changed was the increased move for voters to using online forums to express opinions and to verify claims made by individual candidates. The Obama team knew what issues they had to address and who they had to win over. In real time, they could identify what issues people were discussing, what the deeper concerns were and develop a platform to address people at an individual level.
The campaign represented one of the most sophisticated uses of data analytics by government to date and gave his team a new way to understand, engage with and deliver personalised responses.
In Australia, Transport for NSW is using an innovative approach to Big Data better understand off-peak commuter behaviour. By sifting through anonymised Opal card data, linking in weather data, car park capacity near train stations, employment data and a host of other datasets, Transport for NSW is developing off-peak demand models for each station in the Sydney metropolitan area.
US manufacturing giant GE is using data and analytics to improve business. The company now offers predictive maintenance and optimisation services for more than $1 trillion worth of internet-connected industrial equipment, ranging from medical equipment to jet engines. This business was worth an estimated $1 billion incremental revenue for the company in 2014, mostly in the form of advanced asset performance management services.
Governments are also developing an understanding of the benefits of Big Data analytics. The New Zealand Government has established a centrally-led Data Analytics Agency. Most recently, the NSW Government has established the Data Analytics Centre to explore challenges ranging from transport optimisation and improving fire crew responses, to government service delivery and child protection.
What is Big Data?
There have been a staggering number of reports, white papers, advertorials and considered statements on what Big Data actually is. Most sources agree that it is best described by the ‘Vs’ like ‘volume, variety and velocity’, although not everyone agrees how many Vs are relevant.
If you are suddenly faced with much more data (volume), arriving at a rate (velocity) much greater than your systems could cope with, then you are in the world of Big Data. The variety of data sources is what makes Big Data relevant and important. The more factors involved in decision making, the harder it is for us to make sense of it. Simplifications are often applied to help people make sense of data. When we can bring analytical techniques to bear on a problem, we simplify at our own risk.
In his influential 1973 paper The Strength of Weak Ties, Mark Granovetter argued that in marketing, information science or politics, weak ties enable the reaching of populations and audiences that are not accessible via strong ties. The paper also highlighted the importance of linking micro-level interactions to macro-level patterns.
The Big Data equivalent might be rephrased, as secondary data can be used to provide insight into a system or process for which primary data are not available. If the data you need to answer a question is not available (set A), it is possible to build a portfolio of information from related data sets (sets B, C, D) that gives insight to the answer you would have obtained from set A.
Serving up bits not atoms
Companies like Kodak and Blockbuster Video were once household names. These companies are no more, or at best, a shadow of their former selves. They not only missed some important business and technology trends – they missed the move from the physical economy to the digital economy.
The term digital economy was first used in Don Tapscott’s 1995 book The Digital Economy: Promise and Peril in the Age of Networked Intelligence. Nicholas Negroponte subsequently used the metaphor of shifting from “processing atoms to processing bits” as a means of explaining the transition to a digital economy. This is now described by some economists as the sector of the economy associated with zero marginal cost intangible goods available over the internet.
“Anything that can be digitised can be represented and delivered as a digital file.”
Digitisation of commerce has had a dramatic impact on all industry sectors and led to the demise of those who failed to adapt. The growing dominance of digital services is also increasingly making the world a single global marketplace. A recent McKinsey Global Institute report shows that in 2012, flows of goods, services and finance reached $26 trillion, or 36 per cent of global GDP.
If the spread of digital technologies and rising prosperity in emerging economies continues, global flows could nearly triple by 2025. Underlying these flows is the dramatic increase in exchange of data and communication across borders. Global online traffic has grown from 84 PB per month in 2000, to more than 40,000 PB per month in 2012.
Digital products
Anything that can be digitised can be represented and delivered as a digital file. The interest in 3D printing is driven by the ability to model and map physical objects in digital form and then send anywhere for re-creation. This has unleashed waves of new service offerings and creative thinking.
The recent example of a NASA astronaut using a 3D printer to create a spanner onboard the orbiting space station, based on a design sent from earth, hints at how far this technology could extend. In the ‘Extinct Species’ display in the Queensland museum, you can see dinosaur vertebrae that have been scanned while still encased in the surrounding rock, then 3D printed to help complete skeletons on display.
These examples seem miraculous, but in practice we have been living with digital products for many years. The market for in-game purchases in computer games is valued in the billions of dollars annually, including the trading of these digital products between players. Similarly, in-app purchases that provide updates for maps for navigation, update the locations of speed-cameras for the GPS, or give information-rich reports on property or investment options are commonplace. We expect the instant delivery of our digital product, with the minimum of interruption to our business or entertainment activity.
Data associated with the end user’s interaction, feedback and experience with the digital goods can be captured to improve the offering and allow for enhanced personalisation. An application downloaded to a phone will request access to location information, ask to report anonymous use statistics and even ask a few questions about preferences.
These requests are often associated with a benefit to end users of additional functionality from the applications: restaurant suggestions based on learned preferences, directions based on current location, news based on personal interests. Taken over a large number of users, the aggregation of these datasets are extremely useful to the provider.
It’s about productivity
We are at an important crossroad in realising the potential of Big Data and analytics. As more of the physical world is reflected in the digital world, the opportunities to generate insights, model and analyse behaviours, and customise service offerings increases dramatically. The benefits can be astonishing.
Data-driven platform providers offering digital services can dramatically increase production and productivity with minimal increases of the ‘traditional’ factors of production: land, labour, capital and entrepreneurship. Digital economy companies like Airbnb, eBay and Uber have grown enormous revenues and market capitalisations by processing massive amounts of user-generated data and providing personalised services, which in turn encourages more users to generate and share their data. Data is driving productivity.
“Digital economy companies have grown by processing massive amounts of user-generated data and providing personalised services.”
The concern faced by all service providers, commercial or government, is the protection of individual privacy. In a world with Facebook, Twitter and LinkedIn, ‘privacy’ is a more difficult concept to define than you might imagine. Pervasive use of data means organisations can continually experiment, simulate and analyse outcomes to better understand their own business, improve operational performance, guide investment decisions and improve customisation of services.
The prize for Australia is the opportunity to create new and exciting business opportunities here, and improve the quality and transparency of our governments while still protecting the rights and the sensitive, personal information associated with each of us as individuals.
