Digging for Dark Data

Dark Data.  A favorite topic.  Data assets are not only your own, but those you don't even know about.   But often only useful in combination with your own data.  How do you figure out the hidden opportunities?  Note the particular mention of sensor streams.  Often these are gathered for very narrow  reasons, not for strategic efforts.  And thus remain 'dark'.    Good talk at the link:

' .... “The insight economy is about how to derive value, actionable insights, in a fast and timely way from disparate data,” said Bob Picciano, senior VP of analytics for IBM. “More and more, the data is coming from a combination of internal and external sources.”

Picciano told Dave Vellante and Paul Gillin, cohosts of theCUBE, from the SiliconANGLE Media team, during IBM Insight 2015 that much of that data is “dark,” or unexploited data. The capabilities of IBM Spark and Streams help to extract insights from this underused resource.

“Data that is difficult to work with creates a high barrier to entry,” Picciano said. “People typically forego trying to get any information out of it. About 90 percent of data generated by most sensors and A-to-D conversions on the market never get utilized, and 60 percent of that data loses its true value within milliseconds. With the advent of capabilities from IBM Spark or Streams, we can extract insight from that information almost immediately. This enables businesses to build data rich products and services that use that information to transform the customer experience.”   ... '