A Short History of Big Data

Overview by Bernard Marr.  Really about data systems in general. Not enough about the actual distinctions between data Big or otherwise, because there are not that many.  Lots about volume, too little about analytics needed.  Still worth a read.

Palindrome Games at Bletchley Park

Brought to my attention in the Language Log,  Which points to a much longer piece on  Palindrome construction games played by the code breakers at Bletchley Park,  as shown in the recent movie about Alan Turing and the Enigma Machine: The Imitation Game.   Still in theaters.  For you text analytics, game playing code breakers out there, this is meaty stuff.  For others, bear with us.   At very least it shows the complex structure and richness of human language.

Longest clever English palindrome (It reads the same in both directions, ignoring spaces and punctuation):

Doc, note: I dissent. A fast never prevents a fatness. I diet on cod

Exploiting the non Randomness of People

Fascinating piece in the Wolfram blog on methods to exploit the inability of people to create random patterns, using the example of the 'Where's Waldo' search game. Nicely done analysis using Wolfram analytical software.  Applications for the real world?   I recall examples we worked on to find 'hidden' items in a space.  To what degree does this work best when the human is trying harder to be random?  They mention another example of building a system to win the rock-paper-scissors game against humans.

Design Sketching and Coding

From IEEE: A favorite topic, how do we take the informal sketches of a non graphic artist and convert them into knowledge with precision?    See also post on CogSketch.

How Software Designers Interact with Sketches at the Whiteboard
Nicolas Mangano, Thomas D. LaToza, Marian Petre, and Andre van der Hoek, Member, IEEE

Abstract—Whiteboard sketches play a crucial role in software development, helping to support groups of designers in reasoning about a software design problem at hand. However, little is known about these sketches and how they support design ‘in the moment’ particularly in terms of the relationships among sketches, visual syntactic elements within sketches, and reasoning activities. To address this gap, we analyzed 14 hours of design activity by eight pairs of professional software designers, manually coding over 4000 events capturing the introduction of visual syntactic elements into sketches, focus transitions between sketches, and reasoning activities. 

Our findings indicate that sketches serve as a rich medium for supporting design conversations. Designers often use general-purpose notations. Designers introduce new syntactic elements to record aspects of the design, or re-purpose sketches as the design develops. Designers constantly shift focus between sketches, using groups of sketches together that contain complementary information. 

Finally, sketches play an important role in supporting several types of reasoning activities (mental simulation, review of progress, consideration of alternatives). But these activities often leave no trace and rarely lead to sketch creation. We discuss the implications of these and other findings for the practice of software design at the whiteboard and for the creation of new electronic software design sketching tools. ... "

(Full technical paper at link above)

Useful Data Science Tutorials

Have mentioned this resource before, but I passed it on to a colleague today and see it is worth mentioning again.  A resource of key short tutorials for data scientists.  Technical details for beginners and all practitioners.  I agree, this is an essential starting point.  From the often useful Data Science Central.  Subscribe.

Fembots and the Gender of AI

Interesting piece on the presented gender of AI and notably robotics as it is delivered today.  Why is this often presented as female?  I think this will change as the idea extends more broadly into our everyday office and home world.

Enterprise Data Usage

Statement of a growing problem.  Recently saw this in a project.  More about the kind of and completeness of data rather than the volume of data.

"As the volume and complexity of data barraging businesses from all angles increases, IT organizations have a choice: they can either succumb to information-overload paralysis, or they can take steps to harness the tremendous potential teeming within all of those data streams," said EMC's Jeremy Burton.

Enterprises have made headway in their data management since then, given the number of big data initiatives and the thousands of organizations of all sizes that are undertaking them. If nothing else, there has been a reshaping of thinking in the data center about data under management."As the volume and complexity of data barraging businesses from all angles increases, IT organizations have a choice: they can either succumb to information-overload paralysis, or they can take steps to harness the tremendous potential teeming within all of those data streams," said EMC's Jeremy Burton  ... " 

Tech Megatrends and IT

In CIOInsight.   A good survey on the influence of tech megatrends on IT.   Obvious examples like Bringing your own devices in the enterprise.

Spontaneous Discovery

Thoughtful piece via a scan from the Tom Peters Blog:

Surprise, Transformation & Excellence through “Spontaneous Discovery”:
A Personal Saga,  this is a revision of an antique—but arguably more relevant than ever (PDF version also available):  "Thinking is the enemy of creativity. It's self-conscious and anything self-conscious is lousy. You simply must ... Do things."—Ray Bradbury.

Book: Big Data Revolution is Big Data at work.

Just received a preview copy of: Big Data Revolution: What farmers, doctors and insurance agents teach us about discovering big data patterns  by Rob Thomas and Patrick McSharry.  To be published next week.

Will read shortly.  An initial scan would indicate this is a very practically oriented, entirely non-technical look at why the Big Data analytics revolution is important for many industries.  Ideal for the executive scan and selective read in related contexts. Deals with actual business decisions to be made and the consequence of making them better!  Will comment later after a read.

They write:

" ... Exploit the power and potential of Big Data to revolutionize business outcomes

Big Data Revolution is a guide to improving performance, making better decisions, and transforming business through the effective use of Big Data. In this collaborative work by an IBM Vice President of Big Data Products and an Oxford Research Fellow, this book presents inside stories that demonstrate the power and potential of Big Data within the business realm. Readers are guided through tried-and-true methodologies for getting more out of data, and using it to the utmost advantage. This book describes the major trends emerging in the field, the pitfalls and triumphs being experienced, and the many considerations surrounding Big Data, all while guiding readers toward better decision making from the perspective of a data scientist. ... " 

Favorite Data Science Books from Kirk Borne

Correspondent Kirk Borne posts a selection of his favorite books on Data Science, Machine Learning and Analytics.    Taking a look now.  Met Kirk at a recent conference and was impressed by his mingling of the learnings of physical science (notably also in my own specialty, Astrophysics)  and business data science.  There is much to learn from mixing deep quant science and data.  I am trying to convince him that this could be taken further, into the realm of decision science.    See also his Twitter stream.  @KirkDBorne.  I follow it. Keep up the good reporting work.

Planograms for Profits

Planograms are diagrams of where goods are placed in a physical retail store.   We did much work on how to implement them consistently and to comply with manufacturer - retailer agreements and produce real value.   Plus linking them to actual consumer shopping behavior.   Retailwire discusses their application.

How Enterprises Use AI

In CWorld:   Good overview of the expansion of the use of cognitive, AI and links to Machine learning.  Based on learnings at the recent AAAI meeting.   My experience and connections, have led to getting lots of queries about where to start.

" .... While A.I. is being used in smartphones and self-driving cars, it's also working its way into the enterprise to filter spam out of email, handle complicated scheduling or detecting fraud in big data deployments.

"I think IT probably needs artificial intelligence," said Stephen Smith, a professor who specializes in robotics and A.I. at Carnegie Mellon University. "There are increasing cyberattacks we're dealing with. We have bigger, more complex problems with all the issues arising out of the explosion of the Web and all of our big data. We're already using A.I. It's already there. I think A.I. in the enterprise is going to start to cascade." ... " 

Service Innovation in a Digital World

In McKinsey Quarterly:  New Business models for services.  Discussion of key technologies that are modifying service delivery.

" ... New digital upstarts are threatening the bottom lines, growth prospects, and even business models of traditional service providers. It’s time for incumbents to innovate—or be left behind.

A growing number of companies are finding their service businesses under threat. The culprits are members of a new wave of digital upstarts that capitalize on changes in technology, customer behavior, and the availability of data to create innovative, customer-friendly alternatives to the services incumbents offer. Indeed, the sorts of digital disruptions that began in retailing with the likes of Amazon, two decades ago, are fast coming to an industry near you—if they haven’t already. Examples include Uber and Zipcar in transportation, Airbnb in hotels and hospitality, AngelList in venture capital, and Castlight Health and Healthgrades in healthcare. Attackers such as these may be small now, but they represent a growing challenge to traditional companies. .. " 

Google for the Enterprise

We participated in several Google tests at work, starting with internal search and knowledge organization methods.  Now they are taking another run at the enterprise workplace via Android:

 " ... On Wednesday, Google Inc. announced its Android for Work program in an effort to tap into the “business and innovation potential” of the more than one billion Android smartphones in use.

Google claims that for many Android smartphone owners the devices have become essential tools in their day to day lives, both personal and professional. But Rajen Sheth, director of product management of Android and Chrome for Work feels that “for the majority of workers, smartphones and tablets are underutilized in the workplace.” ... ' 

Bluetooth Weaving a Mesh

In CWorld:     A natural next step is outlined.   " ... Bluetooth starts weaving its mesh for IoT ... Low-power Bluetooth devices should get a standard way to organize themselves in networks next year .... " 

Good Bad and the Ugly about Big Data

What is next? Wondering what to expect? Check out this presentation from Juan Miguel Lavista, Principal Data Scientist. Bing.  'The Good, the Bad and the Ugly about Big Data & Data Science':   Requires registration.

Sample Naive Bayes Text Classification in Python

Via O'Reilly: Nice Python example of using Naive Bayes for text classification.  With actual code, and links to talk and slides.

" ... Full repo here: https://github.com/arnicas/NLP-in-Python
This is an example of going from labeled text to machine classification, first with NLTK and then the Python machine learning library scikit-learn. Examples updated from my OpenVis Conf talk here, which is more entertaining: https://www.youtube.com/watch?v=f41U936WqPM and slides: http://www.slideshare.net/arnicas/the-bones-of-a-bestseller
Warning: Rated NC-17. Using text samples from "50 Shades of Gray"! (Because spam is boring.) ... " 

Gamification for Testing and Learning Analytic Tools

In the HBR: Gamification Can Help People Actually Use Analytics Tools ... We did this to test supply chain design alternatives ...  since any game includes a simulation to evaluate a particular set of parameters, typically applied over time, its a natural idea.  We used the idea more for testing the results,  rather than learning the tool.    It should be tried more often.                

Pringles as Global Kelloggs Snack Equity

Was a participant in the design of Pringles supply chain.  I saw first hand how widely it was sold in Japan.  In clothing stores?   It is interesting to see how this very global brand,  after its sale to Kellogg, has changed its global snack presence.  An interesting article with numbers in Food Business News.

Status and New Product Adoption

In Knowledge@Wharton: 

" ... Certain consumers are keen about adopting new products and are prone to mimicking the product choices of others. But when it comes to status-enhancing products, people in the middle of the status hierarchy are most likely to adopt a new product quickly, according to a recent paper co-authored by Yansong Hu, a professor at the University of Warwick in the U.K., and Wharton marketing professor Christophe Van den Bulte. The paper, “Nonmonotonic Status Effects in New Product Adoption,” was published in the May 2014 edition of the journal Marketing Science. ... "  

Hospitality Innovation at Starwood

We worked with groups in the hospitality industry to brainstorm the future of business mobility.    So the industry has always fascinated me.

Now in McKinsey:  " ... Redefining service innovation at Starwood. The head of the hotel company’s loyalty program, Mark Vondrasek, describes its approach to technology, guest loyalty, and disruptive new competitors .. " .  

Notably also, learnings in loyalty:       "  ... Guest loyalty has always been important to Starwood, but a few years back we began to get far more granular in how we approached it as we studied the profitability of our guests in side-by-side comparisons. We found, for example, that the top 2 percent generated 30 percent of our organization’s profits—an incredibly high concentration. We also learned that the “platinum” members of our loyalty program—Starwood Preferred Guest (SPG)—are many, many times more profitable than guests who aren’t SPG members. Analyzing the data was eye opening, and really pushed us to reexamine how we think about loyalty and benefits.   ... " 

More granularity loyalty should be better analyzed in retail as well.

A Look at Correlation

A useful, only mildly technical tutorial by Vincent Granville.  How can you determine if a correlation is spurious?   Understanding this point is one of the most important aspects of using analytics in business.  It matches a technical measure with a real life decision. Worth a read.

Storytelling, Context and Persuasion

All data tells a story, in a context, and suggests a future.

" ...  In a world increasingly saturated with data and information, visualizations are a potent way to break through the clutter, tell your story, and persuade people to action. Raw statistics by themselves are fine. But showing in context, whether with a simple chart or more creatively in an interactive form, is the future of sharing information, and needs to be embedded in the thinking of all communications professionals.

Combining data - which can be dry - with real creativity - which isn't something humans, even creative ones, can simply turn on and off - can be challenging. Yet we live in a stream-powered mobile world that is increasingly visual, inspiring demands from media to achieve equal parts style and substance for news. This explains why unique and truly compelling visualizations are an underused, yet devastatingly effective tactic. They are equal parts rare and in demand. They beg to be shared. They are a catalyst for conversation, awareness, and action. .... ".

The Yo Store

Why?  Its a one stop shopping place for sensing online events.   In Fastcompany.   " ... Eight months after bewildering the tech world, Yo is getting a substantial revamp, according to TechCrunch. Today, the startup launched the Yo Store, a repository of branded channels that let you subscribe to notifications of your choosing. Want to know when an item on Product Hunt gets 500 upvotes? Yo will tell you. Need to know whenever there's a new listing under a certain search term on Craigslist? Don't worry about it, yo. ... " 

P&G's Strategic CIO

In CIO Mag:

by Phil Weinzimer: 

P&G's Filippo Passerini Stands Out as Stellar Example of a Strategic CIO ...  I didn't know what to expect when Procter & Gamble CIO Filippo Passerini agreed to be interviewed for my book, The Strategic CIO: Changing the Dynamics of the Business Enterprise. Passerini rose through the ranks at P&G to become not only CIO, but also president of the company's Global Business Services division. Given that P&G is an $84.2 billion consumer products company with almost 5 billion customers in 70 countries, managing one of its four main business units is of course quite a responsibility.  ... " 

Graph Databases and Modeling the Internet of Things

Have had a number of conversations on the use of graphical databases lately.  It is a good technology to understand.    It works well with anything that is interconnected.  As are people and any communicating things.  Obviously:  Its exploratory data analysis and resulting discovery.   In SiliconAngle:

" .... As a result, the Internet of Things should perhaps instead be called the “Internet of Connected Things.” This emphasizes the network itself, and the many interaction points between individual devices, people, apps, and locations; and how they can and should (or should not) interact. Understanding and managing these connections will be at least as important for businesses as understanding and managing the devices themselves.

Imagination is key to unlocking the value of connected things. For example, in a telecommunications or aviation network, the questions, “What cell tower is experiencing problems?” and “Which plane will arrive late?” can be answered much more accurately by understanding how the individual components are connected and impact one another.

Understanding connections is also key to understanding dependencies and uncovering cascading impacts. Such insight allows businesses to identify opportunities for new services and products that make the most of the IoT. To identify these opportunities, businesses need tools that can show these connections quickly and easily. ... " 

Building a Mobile Supply Chain

By their nature, supply chains are mobile and real time:

" .... Embracing mobility promises to increase visibility and responsiveness in the supply chain when properly leveraged. For these reasons, businesses are more and more interested in embracing mobile applications. In this year's North American Mobile Enterprise Applications report, Frost & Sullivan reported that companies reporting the use of at least one mobile worker application rose from 73% in 2013 to 82% this year. "Currently, 49 percent of businesses across North America adopt between one and ten applications, indicating a significant acceptance of these solutions," the report said. ... " 

Searching on Amazon vs Google

Thought about this the other day.  Where do I search?  Google,  Amazon, Wikipedia.   My own blog posts.   I typically categorize my search need quickly to make a decision.   Overlap often exists.

Technology Disruption in Health Care

Video interview: Dr. Robert Wachter, chief of hospital medicine at UCSF Medical Center and a noted expert on quality and safety, says information technology is disrupting healthcare in unexpected and sometimes unfortunate ways. ... " 

An Adaptive Dance of Drones

In NewScientist: Drones continue to advance, their cooperation, adaption and autonomy is particularly interesting.

Visualizing Sound

In the CACM:
" .... In the movies, people often discover their room is bugged when they find a tiny microphone attached to a light fixture or the underside of a table. Depending on the plot, they can feed their eavesdroppers false information, or smash the listening device and speak freely. Soon, however, such tricks may not suffice, thanks to efforts to recover speech by processing other types of information.

Researchers in the Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology (MIT), for instance, reported at last year's SIGGRAPH meeting on a method to extract sound from video images. Among other tricks, they were able to turn miniscule motions in the leaves of a potted plant into the notes of "Mary Had a Little Lamb," and to hear a man talking based on the tiny flutterings of a potato chip bag.     ... " 

Multi Sensory Computer

In Forbes, on the IBM Chip:

Meet The Brain-Inspired Computer Chip That Can Smell, Feel And Hear
By Dharmendra S. Modha, Ph.D., IBM Research

Building a computer that could match the power of the human brain has long been a goal of scientists. In August, we made a breakthrough, published in Science in collaboration with Cornell Tech, which is a significant step toward bringing cognitive computers to society. We announced that we’ve built a computer chip that functions like a brain does with the ability to sense, taste, feel, smell, hear and understand its surroundings.

The SyNAPSE chip is a neurosynaptic computer chip that is a size of a postage stamp and runs on the energy equivalent of a hearing-aid battery. This technology could transform science, technology, business, government, and society by enabling vision, audition, and multi-sensory applications ... " 

Storeless Generation

How can the 'storeless' generation be won back?   Good thoughts important to retail.

" .... But can a big-box store truly be a third space in the classic Oldenburg sense? Recent research suggests yes, but stores need to make a strategic decision about what role they play in the lives of consumers: Are you merely a fulfillment center or do you have enough permission from consumers to turn your store into a third space again? If the answer is the latter, the pleasure of shopping and the social interaction the younger generation crave can take center stage again. It’s not too late. ... " 

Japanese Hotel to Be Staffed by Robots

New robotic service application.  Here in Japan.  " ... This summer, a hotel will open in the Netherlands-themed Huis Ten Bosch amusement park in Nagasaki, Japan. It will have 72 rooms. Room fees will start at $60 per night. And it will be staffed by 10 humanoid robots. ... "   The humanoid aspect is interesting

Warning of a Digital Dark Age

Despite methods like the Internet Archive, in the BBC:

" ... Vint Cerf, a "father of the internet", says he is worried that all the images and documents we have been saving on computers will eventually be lost. Currently a Google vice-president, he believes this could occur as hardware and software become obsolete.

He fears that future generations will have little or no record of the 21st Century as we enter what he describes as a "digital Dark Age".. ... "

HBR: Data Scientist as Sexiest Job

Probably, but still on a wave of hype, as currently defined .... Is Decision Scientist next? Data Scientist; The Sexiest Job of the 21st Century. - HBR.

Watson and Jeopardy at the Brink

Jim Hendler mentions how close Ken Jenning's came to winning the Jeopardy competition over Watson - very true - check out his TED talk about after the event and his reflections  How much would this event have changed a new future for AI?    IBM is not the only player today, though it has raised interest in the topic.

Microsoft Launches Azure Machine Learning

Newly announced:

" ... Microsoft officially announced at the Strata Conference today, the general availability of the Azure Machine Learning service for big data processing in the cloud. It also announced some enhancements to the platform since its Beta release in June.

As we wrote in June, “The product is built on the machine learning capabilities already available in several Microsoft products including Xbox and Bing and using predefined templates and workflows has been built to help companies launch predictive applications much more quickly than traditional development methods, even allowing customers to publish APIs and web services on top of the Azure ML platform.” ... " 

Data Layers in Big Data

Bernard Marr on data layers that comprise big data.   Very nice, instructive and non technical, this is the essential part of defining what Big Data is.

What Does the Term Cognitive Assistance Mean?

A Linkedin Discussion started in the CSG group by Frank Stein, Director of Analytics Solution Center at IBM.  I have added some additional examples and comments

What Does the Term "Cognitive Assistance" mean? .... 

A US Government representative asked one of our team, Chuck Howell, "What does the term 'Cognitive Assistance' mean"? What is in scope for cognitive assistance systems? Chuck stitched together this working definition, after a quick back and forth, by combining excerpts from two important documents ....  "

Chuck Howell, Mitre, Scott Kordella, Mitre,
Frank Stein, IBM

Liquid Metal Electronics

In Gizmag: Possible application to 3D printing,  reminds me of a product development fabrication experiment long ago.  Probably not useful in my realm, but fascinating.

NGrams, Data and Patterns in Human Culture

A data lens to human culture.  As yet unread, but of interest.  Uncharted: Big Data as a Lens on Human Culture  Hardcover – December 26, 2013 by Erez Aiden  (Author), Jean-Baptiste Michel (Author)  See also here.

" ... Between 2007 and 2010, graduate students Erez Aiden, Jean-Baptiste Michel and Yuan Shen developed a tool for analyzing Google’s digitized books. Known as the N-gram viewer, their tool essentially just counted the number of times a word or phrase appeared in all the digitized publications for a given year.  ....

Aiden and Michel have written about their N-gram viewer and how they came to develop it in a book: Uncharted: Big Data as a Lens on Human Culture (2013). The book gives lots of interesting example of how N-grams might be used to track fame, understand nationalism, explore the birth and death of words and concepts, and follow the influence of inventions. Aiden and Michel see their tool as part of a big data revolution that will transform the way we understand human culture in general and history in particular. “Big data,” they argue, “is going to change the humanities, transform the social sciences, and renegotiate the relationship between the world of commerce and the ivory tower” (p. 8). ......   "

Simple Graph Plotting

Brought to my attention, a simple way to deliver plots.  Not perfect or anywhere near complete, but I like to collect simple means.  I have Tableau, Excel and half dozen other means at hand too.

Realtime Strategic execution

Trends in 2015, the need for real time

" ... Project teams will deliberate less and plan in real-time more. Companies will recognize a cultural difference as a point of strength, rather than a source of confusion. And, more than ever, IT will emerge as a leading influence on the critical decisions that drive organizational success. These and other developments are among the following top 10 "strategic execution trends for 2015," as adapted from a list compiled by IPS Learning. Above all, the trends speak to the increasing need-for-speed mindset that CIOs and other executives must adapt to keep up with the pace of technology and market changes. "As business continues to accelerate, flexibility and speed of strategic execution become ever more important," says Tim Wasserman, chief learning officer of IPS Learning and program director of the Stanford Advanced Project Management (SAPM) program. "Smart, high-performing organizations have already structured themselves to react more quickly to external and internal shifts. They view the future as a possibility, change as an opportunity and their strategy as the tool to get them there." IPS Learning partners with a variety of academic institutions, associations and organizations, including the Stanford Center for Professional Development (SCPD), to advise organizations on how to achieve better growth and smarter innovation

Shopping Through the Lens of IT

In the ACM    " ... Pennsylvania State University (PSU) researchers are studying how smart cameras could eventually guide visually impaired shoppers to find the items they need.

The researchers are using a $10-million U.S. National Science Foundation grant to replicate the human vision system using information technology as part of the Visual Cortex on Silicon project. "This project brings together the strengths and efforts of technical leaders in multiple disciplines," says PSU professor Vijaykrishnan Narayanan. ... " 

Watson Preps the Data

A prediction that Watson Analytics will also prepare the data.  Often a tedious and error prone activity.  There are many aspects of preparing and scrubbing data.

What Does the Term Cognitive Assistant Mean?

A Linkedin discussion on the topic, which includes some comments of my own on several systems we built that have been mentioned here.  Assistance can mean to augment someone's skills, using data or knowledge. It can also scale their capabilities broadly. It can be focused, or very broad in scope.  An assistant helps, at agreed upon levels of detail, focus and correctness.

Simulating Architectures in a Game World

From today's CSIG talk. Slides at the link. The use of simulation and game technologies to understand the operation of alternative architectures. Here specifically about cognitive (reasoning) technology, but once you present any architecture as a game, it is natural to create a simulation to understand complex interaction, side effects, and using crowd sourced interaction to tune design and alternative parameters. I like the positioning of this as 'challenging' architectures. Every architecture needs challenging.

Archiving the Internet and Searching it with the Wayback Machine

Many people think that everything that is on the net stays there.  That is not entirely true, though it is a good idea to operate as though that is the case.  A system called the Internet Archive is an attempt to archive large parts of it, so they will always be available.  Though the choice of what to archive is still partly selective and partly random.   In the January 26 New Yorker, an excellent article: The Cobweb: Can the Internet be Archived?, about what is called the WayBack Machine, and the idea of internet persistence.  Have used that capability to support some things I have written on historical topics.  It has allowed me to do things that would have been impossible otherwise.    Once you have read the New Yorker article, you can visit the Internet Archive Wayback Machine here.  It has saved 452 billion web pages over time, to date.  Check out what your internet presence is now, and what it was then.

Machine Learning Goes Mainstream

In Forbes:  Machine learning being used in our information technology environments, here a good example.  Also pointer to an academic paper on the depth of 'understanding' in learning:

" ... Machine Learning Goes Mainstream I: InboxVudu Prioritizes Your Email .... Machine learning is moving out of the backend shadows and into mainstream applications that ordinary consumers will use on a daily basis. These applications will recognize the intent behind written and spoken language and quickly classify the content of images and video. It would be a leap to say that these systems “understand” the data they process, but a recent academic paper on deep learning claims just that. ... "

Re-Threading Topics Here

At some users requests have completed re-threading a few topics here.  This usually has to be done because some of the links provided when the posts were done are no longer active.  Not all sites maintain their content.  Sometimes I can find the new link, but sometimes I can only indicate the link no longer works. I have to do that selectively, based on my estimate of interest.  Sorry for any inconvenience.

(Update)  The topic of knowledge persistence on the Web is nicely covered in this recent New Yorker Article.   Not everything on the Web is forever.  Recommended!  As long as it persists.

Internet of Things from a Consumer Perspective

In Retailwire, from the Oracle Blog:  " ... To take advantage of new data sources from sensors and machines emerging from the Internet of Things, it helps to consider what it all means to the now commonplace mobilized consumer and their journey along the path to purchase. Do you think consumer applications of the Internet of Things will facilitate collaboration between retailers and their vendor partners?  ... "    (and Discussion prompt) 

February Analytics Mag Emphasis on Data Visualization

February 2015 Informs monthly on analytics, the leading professional organization for analytics practice.  The most practical and largely non technical periodical I receive on the topic. Examples, hints and value examples.  In this issue, on  sports, marketing and Example:

‘Clumpiness’ more profitable than traditional marketing segments
Marketing managers traditionally segment customers by three summary measures (also known as the RFM model): recency (the period of time since their last visit), frequency (how often they visit) and monetary value (how much they spend on a visit). However, a new study published in the INFORMS journal Marketing Science shows that, in contrast to traditional market segmentation, one based on “binge consumption” brings a higher long-term return to business. .... " 

Stanford DeepDive

Brought to my attention.  Another example of the automation of analytics.

" .... DeepDive is a new type of system that enables developers to analyze data on a deeper level than ever before. DeepDive is a trained system: it uses machine learning techniques to leverage on domain-specific knowledge and incorporates user feedback to improve the quality of its analysis. .... 

 DeepDive is targeted to help user extract relations between entities from data and make inference about facts involving the entities. DeepDive can process structured, unstructured, clean, or noisy data and outputs the results into a database.

Users should be familiar with SQL and Python in order to build applications on top of DeepDive or to integrate DeepDive with other tools. A developer who would like to modify and improve DeepDive must have some basic background knowledge listed in the documentation below. .... 

Jobs Susceptible to Computerization

Working on a related problem.  What jobs or part of jobs are likely to be replaced vs augmented?  What is the influence of Cognitive methods?

The Future of Employment: How Susceptible are Jobs to Computerization.
Carl Benedikt Frey†and Michael A. Osborne‡ September 17, 2013
Abstract
We examine how susceptible jobs are to computerisation. To assess this, we begin by implementing a novel methodology to estimate the probability of computerisation for 702 detailed occupations, using a Gaussian process classifier. Based on these estimates, we examine expected impacts of future computerisation on US labour market outcomes, with the primary objective of analysing the number of jobs at risk and the relationship between an occupation’s probability of computerisation, wages and educational attainment. .... " 

Google Challenges Deodorant Market

CPG, beware of Google competition.  Wearable removes odors?  Patents the idea anyway.

At first this sounded like it was a very clever sensor idea, with electronic removal of odors?  Wow.   Impressive plan. But then seeing how it is done ... yawn.

" .... Google Technology Holdings was awarded a patent last week for an "odor removal device.  " that includes an activity sensor to help predict when you'll get body odor, an automatic fragrance emitter to combat it and a system that tracks people in your social networks so you can avoid them when you smell bad. ... ".  

Trusting Machines vs People

A long time interest.  To what degree and under what conditions do people trust machines?  Our early look at automated marketing engines gave a different view.  Here in Knowledge@Wharton.  A key issue for advisory systems.   Podcast and Transcript:

" ... Facts vs. intuition. Man vs. machine. Algorithms vs. emotions. When we’re given the choice of trusting another person’s conclusions, or our own guesses, or accepting facts as based on algorithmically analyzed data, most of us tend to trust the human more. But that’s not always the best choice.

In a recent interview on the Knowledge@Wharton show on Wharton Business Radio on SiriusXM channel 111, Wharton practice professor of operations and information management Cade Massey and Wharton doctoral student Berkeley Dietvorst explain what their research revealed about the biases hiding in our decision-making, and why we’re so reluctant to trust computer-generated answers if the machine has ever been less than perfect — even though our own record is even worse.  ... " 

Making Stores Matter

In McKinsey: Stores need to refine how they use their real estate.

" ... For decades, the retail industry has followed the same straightforward formula for growth: open new stores. By replicating a proven store format in a new catchment area, retailers could reliably enlarge their customer base and count on healthy increases in sales.

But the world has changed. More than half of consumers now research their retail purchases online, making purely in-store purchase decisions the shrinking minority. In many categories, e-commerce has dramatically lessened the need for physical stores. “Virtual space”—which we define as the floor space that would be required to generate the sales volume that online retail now accounts for, at a sales density equivalent to the industry average—is expanding at a staggering rate. In this new world, what is the role of the brick-and-mortar store?   ... " 

A Phase Advisor for Industrial Chemical Engineering

A recent conversation with a chemical engineer at MIT led me to remember an expert system a member of our AI group, Dan T. Davis wrote in the late 1980s.  It was meant to provide advisory consulting in the surfactant chemistry space. And was based on the expertise of  employee Bob Laughlin, author of "The Acqueous Phase Behavior of Surfactants".   It was based on methods of knowledge engineering and rule based expert systems.  I am doing some additional digging on the methods and history of this system and will post them here.    Anyone have specific examples of use and methods, contact me.

Frugal Innovation

Frugal Innovation,  by Navi Radjou and Jaideep Prabhu, The Economist, Book brought to my attention.  Today we have many more resources than ever before. Via IdeaConnection.

Strategists Guide to IOT

A Strategist’s Guide to the Internet of Things ... The digital interconnection of billions of devices is today’s most dynamic business opportunity. ... 

" ... The IoT has its technological roots in the decades-long effort to monitor and control the physical environment in which people work and play. Its most basic components are embedded devices that have existed for years: thermostats that sense ambient temperature and control heating and cooling systems, sensors that manage braking systems in automobiles, pacemakers that regulate the heart, airplane black boxes that track flight paths, and location devices that monitor the whereabouts of industrial equipment. In the past, some of these devices were wired together into more complex systems. But it wasn’t until they were provided with some intelligence, connected to the Internet, and empowered by a new wave of technological accessibility—through cloud computing, smartphones, and the prototyping capabilities of digital fabrication—that the IoT came into being. ... " 

Product Development with Virtual Prototypes

In February IEEE Computing Now:

Product Development with Virtual Prototypes
Douglass Post discusses how the use of virtual prototypes analyzed with physics-based performance prediction tools is a potential game changer for product development. ....  "

Anticipating Apple's Smart Home

My enterprise was one of the pioneers to create a smart home that could be rapidly modified to test new technology configurations.  It was also unique in that it had a next door smart store, so you could test interactions between what happened in front of home media, and then led to results in a store.  We also tested the earliest smart phones to understand how technologies like a shopping list would change the shopper's behavior.  Now, fifteen years later Apple rolls out its Smart home, to link to its broad smart ecosystem.  An early look.  Options that exist today and speculations . How will we control our homes in the near future?

Goals of Predictive Analytics Models

When I read the title of this article it said 'types' of predictive analytics models, which lead me to believe it was the kind of technical model.  But no, its about the more important thing in the model, its about the value goal of the model.  A short piece that forms a reasonable list of the typical kinds of goals you might want to achieve. Obvious perhaps, but a good reminder of what can be done.

Business Process Models meet Watson

Have for a long time thought about the natural connections between Business Process Modeling (BPM) and the augmentation of jobs and sub tasks using Cognitive systems. We experimented with this idea in the expert system era. Saw a short presentation on the topic last year.  Now IBM has software services in both spaces.  Came upon this pointer to work done underway in the area.  Worth a much closer look.

" ... IBM® Business Process Manager (BPM) is widely used to automate business processes, and the end users are often knowledge workers who make decisions based on their domain knowledge. IBM Watson™ has the capability to respond to user queries using natural language based on a corpus about any business domain. If you integrate an IBM BPM V8.5 application with the Question and Answer service on IBM Bluemix™, your team can use IBM Watson to support decision making based on a particular business corpus. ... " 

Robots and AI at an Inflection Point

At PARC Forum:   Steve Omohundro's talk:  On AI and Robotics at an inflection point.  Disruptive technologies.  Very nice look at the current status of both technologies and their directions and implications.   Social, labor, technical and beyond.  We covered robotics in our  AI group, and the changes since then are many and deep.  In particular the growing AI/Robotics connections. The approach of science fiction scenarios.  Cautions.  Implications of new levels of autonomy. Mentions of Bayesian inference.  Companies that are in play.  Well-known and new to me.  Well beyond the robot Sushi machine.    Great catch up for understanding.  Thanks to Jim Spohrer for pointer.

More on Speaker.  Previous work mentioned here.

Crash Course in Statistics

For Machine learning.   Not enough certainly, but some good points made.  With today's online resources, you don't need a room full of statisticians, just some appropriate caution.  And that know-what-you-don't-know aspect is important, have made errors myself that were of that form.

Supply Chain Visibility and RFID

Sure visibility is increased, but then how does that provide value?  Primarily by combining the information with other data, and then using analytics to feed key decisions.

iPhone Security

In Wired: Considering the security of the iPhone.  (From  September 18)  A reminder to iPhone owners cheering Apple’s latest privacy win: Just because Apple will no longer help police to turn your smartphone inside out doesn't mean it can prevent the cops from vivisecting the device on their own. ... "

Choremonster Meets Disney

I see that colleague and long time connect Chris Bergman has been out in LA to meet with Disney to talk about his gamification startup Choremonster.   What a natural connection.    Have written much about that here.  Good luck on your new connections!  Check out the Choremonster site.

Integrating Smart Devices

In CWorld: Testing Hubs, a case and physical study:

" .. Testing the Internet of Things: Can smart devices be united into an integrated whole? ...  We try out the Revolv hub to see if it can really make independent smart devices work together intelligently. The results? Mixed..... "  

Business Intelligence as Organizational Knowledge

Nine page PDF with some interesting thoughts.  No registration required.
Business Intelligence 3.0: Revolutionizing Organizational Data  ... By Panorama

This is not said in the above doc, but I like thinking about BI as a way to bring together organizational intelligence.  Integrating knowledge, structured and unstructured, corporate data (big or small) , derived cognitive knowledge and predictive analytics into a focused means to drive a business.  (Diagram at the right from Forrester 2/2011, is not based on my thought here, but takes a related view)

Predictive Regression Trees

Regression trees were a favorite method for prediction in the enterprise.  Easy to use and also easy to explain to a decision maker.   Investigating their use in open source solutions.  Here is one R library example. Technical warning,  but an indication of the variety of solutions that are available.  This was not the case a decade ago.  We are moving quickly, but the space is thus more complicated.  More tech depth here.

Polyglot Data Science

Instead of automating your data science, you could take a polyglot approach that combines the three common open source tools,  Mainly so you can get access to different libraries of solution value for your problem.  It is not that its hard to learn these languages, but the picky details can slow you down.  For the tech specialist only.  I can see the value, but like the idea of automating at least the bulk of the process.

Prezi does Creative Visual Storytelling

Prezi announces a tool for storytelling called Nutshell.  I like their original tool, this should be interesting, it is   " ...  the next new tool for creative visual storytelling. ... Nutshell is the mobile app that lets you turn everyday moments into short cinematic stories, combining the ease of photos with the spatial context of video and the fun of animated graphics. 

Just use your iPhone to snap three pictures of anything you want. Add some captions. Choose your graphics. And that's it! You've created a nutshell.  ... "   More here.

(Updated)  I tested on iPhone. Very simple idea.  Stitches a video together from three still pictures,  Then you can put text or other tags on the pictures.  The whole thing is then stored as a video which can be shared.  Quickly learnable.  Can probably use more features ... but I like the initial simplicity.  Thinking compliance delivery ideas? Other ideas?

Automating Data Science

Much in the news lately, have seen at least three other examples of automating major aspects of analytical method recently.  Can it put the data analyst out a job?   This has been much discussed for years.  The argument against was always the danger of producing analyses that would be subtly invalid and lead to business errors.  Or needed skilled statisticians and storytellers to explain them to management.   We did some things like this, but for narrow and focused domains.  But perhaps we were too cautious, the examples I have seen so far are impressive.  Warnings against problems in data can be supplied.  Domain context can be modeled into the problem.   Sponsors like Google, which works with lots of data problems, make this worth taking seriously.    Good piece, with examples of efforts underway.

See also Watson Analytics.

In Technology Review:
" ... Software that can discover patterns in data and write a report on its findings could make it easier for companies to analyze it... '     ...    By Tom Simonite

Now researchers backed by Google are developing software that could automate some of the work performed by such data scientists, in hopes of making sophisticated data skills more widely available. When fed raw data, the “automatic statistician” software spits out a report that uses words and charts to describe the mathematical trends it finds. .. "  

Cognitive Systems Learning Tasks

Excellent talk in today's CSIG presentation.  How tasks will be learned by cognitive systems.  See the demo link below for an early example.

Via Jim Spohrer:
               Prof. John Laird (U Michigan) decribes how next generation cognitive systems will be able to learn new tasks - learning to play new games, as well as other tasks - perhaps tasks from O*NET online occupation network someday?  General cognitive architecture - and learn new tasks and domain knowledge.   Slides and demonstration. ... " 

Memex is Searching the Dark Web

Those who follow the history of knowledge and the Web will remember the Memex system.   Now the term points to an effort by DARPA that looks at the Dark Web.  From ITWorld:

" ... A “Dark Web” search engine developed by U.S. defense researchers is in the spotlight this week for its use in combating human-trafficking activities, but it could play a role in business, too. 

“There’s huge potential,” said Jeff Schneider, a research professor within the Robotics Institute at Carnegie Mellon’s School of Computer Science. Earlier this year, Carnegie Mellon was awarded a $3.6 million contract to collaborate on the project at the Defense Advanced Research Projects Agency (DARPA), which is known as Memex.

The Memex program—a three-year research initiative to develop software for domain-specific indexing of open, public Web content and domain-specific search capabilities—was highlighted on Sunday by a segment on 60 Minutes and an article in Scientific American. ... "    

Data vs Decision Science

Brought up in DataScienceCentral:  What is the difference?  I support making sure we include the people that understand the decision processes involved as early as possible. Else how do we hope to understand the implications in changes we are proposing?  I am not sure this is 'science' in any sense, but it is at very least having access to decisions as they are made today.   Start at the level of Business Process Modeling, which covers the same space.

Text Signals as Sentiment

An area that a number of groups are looking at.

" ... In this burgeoning era of big data, a substantial majority of all the data is unstructured. Much of this unstructured data is textual, such as the data in reports, articles, emails, tweets, and even conversations or support calls recorded in textual transcripts. Because some of this information is perishable, the capability to process it quickly—in many cases, in real time or near-real time—is becoming quite important to enterprises. This processing requires text analytics capabilities.

What does the capability to perform text analytics mean? One simple example is processing a social media feed, such as Twitter, to extract any tweet that mentions a specific element of data such as a company name or a product. This simple approach to matching keywords can provide a quick glimpse into the presence of a product name in the public’s mind. Further, using Twitter-based metadata, for example, creates the possibility to divide this public perception into regions. ... "

Big Data Apps from Teradata

Have just had cause to go back and look to see what Teradata has done in the Big Data space.    Interesting capabilities.  Taking a deeper dive. " ... To ease companies into realizing bankable big data benefits, Teradata has developed a collection of big data apps – pre-built templates that act as time-saving short cuts to value. Limited skill sets and complexity make it challenging for analytic professionals to rapidly and consistently derive actionable insights that can be easily operationalized.  Teradata is taking the lead in offering advanced analytic apps powered by Teradata Aster AppCenter to give sophisticated results from big data analytics. ...  "

More from Teradata on Big Data Apps.

Big Data in Retail

In SiliconAngle: Obvious and not very much detail.  Lots of room for innovative approaches to using data in retail.  The opening sentence is telling:

" ... Although they haven’t necessarily figured out what to do with the data they have or think they need, retailers are moving forward with Big Data.

“We are amassing massive amounts of data,” said Rob Schmults , vice president of e-commerce at Talbots, Inc. “With all the devices and the ways that people interact with us, sometimes it just feels like we’re drowning in this sea of data.” ...  '

Business Impacts of Effective Data

Measuring the result is important, or how would you know you had any?  The means and techniques of measurement are very important. And then what is the value of the data assets being used.   In SyBase.

" ... In a study of over 150 Fortune 1000 firms from every major industry or vertical, we explored issues associated entirely with the lifeblood of today’s enterprises: data. The quality of data, the ability for that data to be accessed wherever and whenever it’s needed, and the relevance of that data in addressing a specific problem were areas of focus in the study – in essence, effective data, and the business implications of greater access to effective data.

The findings, being publicized now for the first time, definitely demonstrate the often dramatic impacts that even marginal investments in information technology can have when that technology addresses data  quality, usability, and intelligence, whether it be using mobility or remote access solutions, analytics or business intelligence solutions, or a combination of the two ... " 

Data Science at the Command Line

This is how I did analytics some time ago, with tools like SAS.  It can still be done that way.  Not sure I want to do it that way, but This book shows how.

Tesco and Big Data Loyalty Plays

Using personalization with loyalty data to better engage the consumer. Tesco and Big Data for Loyalty  Similarities to P&G's 1 Consumer Place?  Relationship to programmatic Ads?

" ... The deal, carried out by Tesco subsidiary Dunnhumby, will see Sociomantic combine its data on 700 million online shoppers with Dunnhumby’s insights into 400 million customers to create an “unprecedented” database of more than a billion people. Sociomantic runs a programmatic digital advertising service, buying and selling online ads via an automated process. Details of the deal have not been disclosed.

Simon Hay, Dunnhumby’s chief executive, says the acquisition will allow Tesco to improve its online marketing to offer a better experience for consumers and advertisers. ... " 

Workforce Analytics

Sue Lam in the APQC Blog:

" .... By now, I’m sure you’ve heard all of the buzz about analytics and the importance of it for your HR function. Though it is a newer idea for HR, social and behavioral scientists have been using predictive models for years to predict the behavior of individuals and groups. Yet it is only a recent trend that businesses are using analytics to help their workforces become more efficient and effective. Does HR really need analytics in order to survive? Business research and industry trends point to “yes.” ... " 

Commerce Anywhere

Brought to my attention: Oracle's Commerce Anywhere Blog.  " .... David Dorf, Sr Director Technology Strategy for Oracle Retail, shares news and ideas about the retail industry with a focus on innovation and emerging technologies. ... "

Smartstones: Human and Natural Technology

A message from the Smartstones Startup, to which I have provided some help.

" ... Do you believe #tech should be more #human? Are you sick of all the menus, passwords, and barriers to doing simple things like... communicating your status? saying you're on your way? or automatically letting them know when you arrive home?

The movement toward making #tech more #human and #natural has begun at http://bit.ly/199VyYn

If you believe, like we do, that it is time to transform technology. To make it adapt to our rhythms versus us adapting to it, then join us as we collectively show the world how it can be done.

Watch our manifesto video and join us bring this movement to the world ...  Are you with us?  Andreas Forsland, Founder & CEO 

On Facebook.

Turning on Twitter Analytics

This was pointed out to me, a means of turning on Twitter analytics.  Have not yet seen any associated reports.

Regression using R and a Start-up Kit

Via Vincent Granville.  Good place to get started,  since even beginning data analysts know what regression is and how to use it with other packages, Like Excel.  So this is a good place to start with R.

Free Field Guide to Data Science

This appears to be very nicely done.  By its title assumes a practical orientation with sample uses mentioned.    The Field Guide to Data Science, In several formats including PDF, and a supporting App will also eventually be available.  by Booz Allen Hamilton.    107 pages.   I see it includes mention of Bayesian networks.  Will review.

Using GMail for Business

A lengthy look at GMail, which I used during early tests in the enterprise.  It is a good system I have grown to be accustomed to, it does not solve the basic email problem.  Too much, too chaotic, and hard to turn into a knowledge base to improve understanding your business,

Winning on the Digital Shelf

A look at winning on the digital shelf.  On packaging e-content.  Some interesting thoughts, but this is a promotion.

Virtual Reality Emerges

Peter Fader, Wharton Marketing Prof, and others weigh in on the wearable-enabled, virtual reality marketplace.   Lengthy detailed piece.  Will this finally make them real as a new kind of interface?   One recommendation is: Emphasize business applications to avoid making them appear as a joke.   I agree:  Focus.  The general applications will follow if they make sense.

" ... A slew of new product introductions indicate virtual reality technology is coming into its own — but it’s a sector that is still waiting for a breakthrough product to win over consumers.

Following several product entries into the emerging field of virtual reality from companies like Samsung, Sony, Canon and Facebook’s Oculus Rift project, Microsoft recently launched the HoloLens, an augmented reality headset, and HoloStudio, a software development kit for the product, in hopes of jump-starting a market that could impact areas from gaming and entertainment to collaboration and business. ... " 

Google and Neural Nets

A look at what Geoffrey Hinton has been doing at Google with artificial neural nets.  We examined his work in the 80s.  Used it to mimic some things that had been done with statistics before.  Now it has taken off to enhance human efforts in new ways.   Will search evolve into your brain in this way?

Managing Remotely

In the HBR.  A classic problem.  Increasingly common.  " ... How do you overcome the challenges of supervising employees in different locations and time zones? What steps should you take to build trust and open lines of communication? How should you establish routines? And how do you help remote workers feel part of a team? ... " 

Data Visualization Playbook

Nicely thought out approach.  Agree that data visualization is essential, and after obtaining the data should be the first thing that is done, to start understanding the data and its form.   The data has a story to tell.  In the process of just such  a effort. " ... The Data Visualization Playbook: Telling the Data Story  .... Learn data representation strategies, techniques, and nuances in a series geared for data scientists and analysts ... "  by Jennifer Shin

Smartstones Communicate by Gesture

I see that the startup SmartStones, which I have been communicating with for some time,  has launched.  More here and participation information.   " ... Smartstones Touch is a wearable, sensorial device that let you send messages with simple gestures.  .... Smartstones Touch is the world's first nonverbal communication and sensing platform designed to enhance your daily life. It is a little stone that will empower you to connect with your world through nonverbal communication.  .. "

This Blog as Knowledge Repository

I have often used this blog, and others I have run,  as a knowledge repository.   For both things I want to point out to people, and for my personal (but still public) store of resources.  I have started, in 2015, to include placeholders in this blog that are not fully formed posts, but mostly links to other resources.   Some may be expanded on, some not.    These are marked by the #Placeholder tag.

I noticed this week in reviewing some threads on the Internet of Things, covered here for years, that a number of external links were longer working.   If I am informed of non-working links, I may try to correct them. But it is the nature of the Net that these things occur.

I was also asked about the relationship of this blog and Twitter.  These blog entries are selectively mentioned on Twitter, sometimes with additional comments.   The Tweets often include other links to related threads.  Follow me there as:  @FranzD

Automated Story Generation

Will journalism ultimately be mostly automated?  Some moves in that direction.  People still set the parameters.  Would like to see how the outlines of a story are defined.

Governance and Big Data

In SiliconAngle:  Good introductory piece on the topic.  How do you control the data and analyses you do with your big data?

" ... The past week witnessed a number of major milestones in evolution of enterprise analytics across both the open-source and proprietary camps, which came together around the fundamental challenges of processing data at scale. Hortonworks Inc. led the charge with the launch of a new industry effort aimed at addressing the need to control the massive amounts of information that are being analyzed.

The Data Governance Initiative, or DGI for short, hopes to create an “extensible framework” for enforcing data rules in Hadoop clusters. It will provide integration with Falcon, a data flow manager originally developed at InMobi Inc. that recently became a top-level project at the Apache Software foundation, and the access control technology Hortonworks obtained through the acquisition of XA Secure Inc. last May. ... " 

Gershenfeld on a New Digital Reality

Interesting Edge conversation with Neil Gershenfeld of MIT.  We connected with him there a number of times in the early days of remote manufacture.  That world is starting to mature.   " ... ...Today, you can send a design to a fab lab and you need ten different machines to turn the data into something. Twenty years from now, all of that will be in one machine that fits in your pocket. This is the sense in which it doesn't matter. You can do it today. How it works today isn't how it's going to work in the future but you don't need to wait twenty years for it. Anybody can make almost anything almost anywhere.  ... " 

Emotional Targeting Board

Mediabrix announces an emotional targeting advisory board.  From the neuromarketing buzz connections.  An interesting move.  Though I note there is no one on the board as yet to cover the big data analytics angle, which is essential for this area.  More on Mediabrix.

Big Data Predictive HR

It has been common to use data analytics in HR for years.  Here in Bloomberg, predicting when you are going to quit your job.  Some interesting details.

Machine Intelligence and Genetic Codes

Addressing the recipe's that convert codes to tasks.

Machine Intelligence Cracks Genetic Controls

Every cell in your body reads the same genome, the DNA-encoded instruction set that builds proteins. But your cells couldn’t be more different. Neurons send electrical messages, liver cells break down chemicals, muscle cells move the body. How do cells employ the same basic set of genetic instructions to carry out their own specialized tasks? The answer lies in a complex, multilayered system that controls how proteins are made.

Most genetic research to date has focused on just 1 percent of the genome — the areas that code for proteins. But new research, published today in Science, provides an initial map for the sections of the genome that orchestrate this protein-building process. “It’s one thing to have the book — the big question is how you read the book,” said Brendan Frey, a computational biologist at the University of Toronto who led the new research. ... "

We Need to Control Our Data

Artificial Intelligence Is Doomed if We Don't Control Our Data
This model is why the discussion about individuals selling their personal data is missing the point: ‘How much money is your data worth?' ‘Are you just like any data broker if you whore your information to the highest bidder?' These are questions asked within the existing paradigm of Internet advertising, where we're simply tools to generate data versus individuals in control of our identities. Artificial Intelligence in this context makes perfect sense. Our lives in code are transactional versus transformational. Why bother getting humans in the mix when invisible tracking ostensibly knows us better than we know ourselves? ... " 

Real Time Analytics is Critical

Says Cisco. I agree.  I would slightly alter it and say 'as real-time as necessary'.   But what is necessary is changing  rapidly.  Not too long ago data would available weekly, now that is not fine enough.  But we never knew what the better detail could provide us.  That can be provided by exploratory data mining today.

Tradeoff Analytics Services

Have been examining some of the Watson services announced last week.    These are available free (with limited amounts of data) with registration on the Bluemix cloud services.  One that intrigued me was tradeoff analytics.  Implementing Pareto optimization.  Have not seen that technique embedded anywhere recently, a nice idea. These can be connected via API to a mobile App,  for example.  At the link, documentation  and a demonstration video.  All the services are published as Beta.  Will try with some sample health domain data. Provide your thoughts.

" ... Helps users make better choices to best meet multiple conflicting goals, combining smart visualization and recommendations for tradeoff exploration ... 
The Tradeoff Analytics service helps people optimize their decisions while striking a balance between multiple, often conflicting, objectives. The service can be used to help make complex decisions like what mortgage to take or which laptop to purchase. Tradeoff Analytics uses Pareto filtering techniques to identify the optimal alternatives across multiple criteria. It then uses various analytical and visual approaches to help the decision maker explore the pros and cons of their alternatives. ... " 

(Update):  It was brought to my attention that Pareto optimization and its uses are not well known. The WP has a good article that starts non technically. And more broadly on Pareto Analysis.

(Update): And on Smart Pareto filters.

Crowdsourcing with the Gamification of Tasks

Notes on Choosing Tasks in a Job to Improve or Replace with Gamification (Published elsewhere)

 When doing business process modeling (BPM) you connect a group of tasks that comprise a job,  to fit together in sequence or in parallel.   Tasks also require time and other resources, like people or data to complete.   Resource availability can be estimated or simulated to model a job process.  The cost of resources can also be estimated in the job process.

If a task is boring and thus error prone for people,  it could be a candidate for replacement using a cognitive method.   But that is not yet always possible.  You can simulate how humans work by constructing a game that simulates the task.   Then crowdsource its operation to see how people perform,  while gathering information on the time they take and mistakes they make.   Such simulation can also lead to better understanding of resource use.   Which can further determine which tasks should be candidates for smart replacement.   We did this in an order placement and management application.

This was done with the game Airport Scanner,  which you can play online, which led to a study by Duke University about airport security applications.   Much more detail on that below.

http://eponymouspickle.blogspot.com/2015/01/turning-boring-task-into-game.html

Samsung Rolling out VR Demo, IoT Connection

In Engadget: Another entry into the VR wearable space.  Available for demonstration this week.  Head mounted displays that are not meant to be unobtrusive.  Focused, perhaps very low volume business applications.  Considerable gaming applications.   It also came to mind this kind of thing may become directly involved with the Internet of Things.  You look at a thing, and it provides information about itself, and you can work with that data.  Appropriate analytics are activated. We posed the same idea in our own investigation of VR and Augmented reality applications.

Teaching Data Visualization with Legos

Interesting means of teaching incremental representation of data.  I have seen lots of bad visualization in journalism lately.  I would hope they would be able to abstract this way without Legos.  But whatever it takes.

Resolving Time Travel Paradoxes

In the SciAm:  Not really a technological issue as yet, but an interesting logical point about science and time.

Forecasting the Flu

In the CACM:  We did sensor work some time ago that addressed bioterror forecasting.   This is related work that leverages social networks.   " ... A research team at the University of California, San Diego (UCSD) says it has refined and improved the predictions of Google Flu Trends (GFT).

The researchers report using social network analysis and combining the power of GFT's big data with traditional flu-monitoring data from the U.S. Centers for Disease Control and Prevention (CDC). ... " 

Scarab Sensor Array

I like the idea of having multiple sensors in a device.  It would be interesting to see a chart of what is hard to sense effectively, and what the technology obstacles are for each component.   And based on the sensors available, what additional data can be derived.   The Big data dimensions involved.  This is an interesting multi sensor example.

" .. Scarab is a wearable sensor platform that has more sensors than any other personal consumer product. These sensors continuously monitor your environment and communicate local conditions and hidden dangers to your smartphone and the web. With Scarab priced for retail at $199, you can easily protect yourself and your family from unseen health threats, add environmental data to your quantified self and help build a safer community. ... The Scarab system includes mobile apps on your smartphone, a web mapping service, such as Google Maps, and a community of users. Scarab is easy to set up and use. Just download the app, attach a Scarab to your bike, purse or backpack, and go! ... " 

A Wearable Sensor platform.

Data and the Internet of Things

In O'Reilly: Big Data and IoT.  Data and as we have been seeing, the architecture to support that data.  A four dimensional dilemma:

" ... The Internet of Things (IoT) has a data problem. Well, four data problems. Walking the halls of CES in Las Vegas last week, it’s abundantly clear that the IoT is hot. Everyone is claiming to be the world’s smartest something. But that sprawl of devices, lacking context, with fragmented user groups, is a huge challenge for the burgeoning industry.

What the IoT needs is data. Big data and the IoT are two sides of the same coin. The IoT collects data from myriad sensors; that data is classified, organized, and used to make automated decisions; and the IoT, in turn, acts on it. It’s precisely this ever-accelerating feedback loop that makes the coin as a whole so compelling.

Nowhere are the IoT’s data problems more obvious than with that darling of the connected tomorrow known as the wearable. Yet, few people seem to want to discuss these problems ... "

KNext: Knowledge in Texts

Jim Spohrer Writes about KNext:
Len Schubert (U Rochester) has developed KNEXT http://www.cs.rochester.edu/~schubert/projects/world-knowledge-mining.html -- more triples than ConceptNet, and apparently designed with inference in mind - I need to check it out more. (Len's was the next to last talk at AAAI, and I was lucky enough to share a cab ride with him to the airport and got a chance to chat with him) .... "

Bayesian Plumbing under Uncertainty

Good short piece on the potential for using Bayesian networks to simulate aspecst of the Internet of things (IOT).   The open question on many of these kinds of problems is:  What is the architecture, current or planned, of the system we are attempting to understand?   The URL made me think, well this is a kind of plumbing problem.   This is especially useful if we have much existing data to model the system.  Also good if we have a mixture of actual and expertise based data.  Then we can simulate changes in data,  from either a forecast or potential architecture design and changes.  And all systems have important elements of uncertainty, and that is also modeled by Bayesian Nets. Some of our early work looked at simple RFID systems.

AI Abstraction Methods

Instructive piece on methods in use today:

In February IEEE Computing Now:
An Anarchy of Methods: Current Trends in How Intelligence Is Abstracted in AI ... 
How should intelligence be abstracted in AI research? Which subfields, techniques, and abstractions are most promising? Why do researchers bet their careers on the particular abstractions and techniques of their chosen subfield of AI? ... " 

Cognitive Computing on Film

Another look at the recent Turing film.  Some interesting points.  Did he invent the computer?  Well not the general purpose ones we expect today.

" .....  The movie’s designers embellished the machine a bit by using bright red electrical wiring to illustrate the effect of a living being with blood-carrying arteries. The machine had to flex to the movie’s narrative and had to be viewable as a work in progress as the story unfolded. The modern-day engineers settled on Adobe Illustrator with a computer-aided design (CAD) plug-in for the design software used to create their Turing machine. This package allowed for texture, light, and visual qualities that they judged to be better than other CAD programs. .... " 

Watson Adds Services to Developer Cloud

In SiliconAngle:  Notable to me is the speech to text capability.  But the added analytics are also good directons.  I am in the midst of working with startups to evaluate Watson services.  So far these capabilities look very good, but are incomplete.

" ... The eight services available now becomes thirteen, thanks to the addition of five new services including Concept Insights, Speech-to-Text Translation, Text-to-Speech Translation, Tradeoff Analytics and Visual Recognition. IBM says each of the new services can be embedded into desktop or mobile apps via APIs, offering advanced functionality that developers would find too laborious and time-consuming to build by themselves. ... " 

Watson updated Cloud Services store.