Despite the promise of big data being the new ‘oil,’ adoption of analytics in the healthcare sector faces challenges. In the short term, the path will be different for different players in the market, based on their needs and maturity levels.

Last week, I had the opportunity to speak at an event on the state of healthcare analytics. The event was about big data technology adoption, specifically Hadoop. While I was there, I sat in on a presentation by the chief data officer (CDO) of a major financial services company that is harnessing big data and deploying advanced technologies such as Hadoop. Listening to his comments, I reflected that several drivers of analytics in the financial sector seemed to be common to healthcare, including increased regulation, changing consumer preferences, competition from disruptive startups, rapid growth in the use of devices and rising security threats.

However, there were some significant differences as well. To begin with, healthcare is not a B2C business, and consumerism is a nascent and emerging trend. Healthcare is not a real-time environment, nor is healthcare a “big data” environment, at least not relative to consumer finance environments.

Big data is the new oil — what kind of oil, exactly?

No one disputes the value of data in business strategy today. According to the Harvard Business Review, 70% of organizations across industries report that big data is of critical importance to them. Big data analytics is expected to be a $187 billion market by 2019, according to IDC. And healthcare is identified as a fast-growth segment, expected to be a $43 billion market. New data sources such as wearables and the internet of things (IoT) present new opportunities and challenges — research firm Gartner predicts that 21 billion IoT devices will be online by 2020.

All of this has led to big data being referred to as the new “oil.” But what kind of oil are we referring to?

I am a chemical engineer by training, and my specialization in college was in petroleum engineering. I see most of the big data “oil” today as crude oil. It takes good crude and a lot of refining for the crude to be used as jet fuel that takes you higher, farther and faster. Otherwise what you get is mostly sludge that can only be used for paving roads.

The MIT Sloan Management Review refers to this as the “unsexy” side of analytics — the challenges of data quality and the processes required to “wrangle” the data effectively for analytics success. The promise of the big data hype outstrips reality by a wide margin in most companies, mostly because of the lack of ability of enterprises to overcome the data challenges. Here are some of the big challenges healthcare faces when it comes to big data:

• Data stewardship: Big data programs are more about people than about technology. While CDO roles have been on the rise, the challenges related to organizational alignment, change management and data governance have not gone away.
• Interoperability: The “I” word has been the primary bane for healthcare analytics programs in the effort to unlock the value of electronic health records.
• Data security and privacy: 155 million Americans have had their personal information exposed by data breaches since 2009, according to the Brookings Institution, and June 2016 was the worst month on record, with cybercriminals calling themselves the “Darkoverlord” illegally accessing more than 10 million healthcare records. The heightened sensitivity to data breaches can put a damper on healthcare analytics programs. The CDO I listened to last week talked about the effect of the HIPPO in the room — the highly paid person’s opinion, which often overrules rational thinking. Healthcare’s version of this is called HIPAA, and fear of data privacy violation often drives decisions related to data access and data governance.

These challenges impact healthcare’s ability to develop analytics programs, and they put healthcare behind other sectors, such as financial services. They also act as headwinds in the adoption of open-source technologies such as Hadoop, except among the largest and most pioneering healthcare enterprises. However, even among mature organizations in financial services, Hadoop has yet to become fully mainstream, with core processes expected to run on tried and tested legacy platforms for the foreseeable future.

Last, but not least, enterprises across the board are struggling to find people to run advanced analytics programs, because IT professionals with data sciences capabilities and expertise in new technologies are scarce.

Big data and the multilane expressway

Several forces are driving healthcare to accelerate the pace of big data analytics programs. In the short term, the path will be different for different players in the market, based on their needs and maturity levels. In effect, healthcare enterprises will cruise at different speeds in a multilane expressway, choosing their lanes based on their immediate business needs and a careful assessment of risks and costs such as these:

Value-based care: This is uppermost on the minds of healthcare executives, as the industry transitions to an accountable care model. Despite quality measures being seen as too complex today, the stated goals of the Centers for Medicare and Medicaid Services of getting at least 50% of Medicare to value-based payments will be a strong driver for near-term investments in healthcare analytics.

Workflow integration: Healthcare’s primary focus today is on improving health outcomes while controlling the cost of care, and the ability of healthcare technology to provide real-time insights at the point of care is more important than the sophistication of the technology itself. For most of healthcare, this means descriptive analytics in the form of dashboards and reports that focus on historical data. Many health systems are focusing on the last mile interaction between providers and patients and are seamlessly integrating analytics insights into clinical dashboards at the point of care.

Predictive analytics, IoT and advanced technologies: In the future, the highest-value use cases in big data analytics in healthcare will combine the power of predictive analytics with the rapid proliferation of data — from wearables, smart medical devices and other IoT systems — that can be ingested into a data lake in a Hadoop environment. However, these technologies are in early stages and only a small fraction of healthcare enterprises are rolling out pilot programs and doing the pioneering work that will take it mainstream in the coming decade.

Originally published at www.cio.com on Jul 19, 2016.