Value of Data Wrangled

In the Trifacta Blog.  Always important element of doing the analytics.  by Joe Scheuermann  

"... Before data can be analyzed, it must be wrangled (prepped) into the requisite format for the analysis you want to run, especially if you need to blend data from various datasets or sources. This step creates a bottleneck or, worse, an impasse that prevents organizations from harnessing and using the potential of their data to enable data-driven decisions.

Data wrangling has historically been the exclusive domain of skilled data scientists—though, even for trained data scientists, this step can take as much of 80% of an analysis cycle. And because the demand for technical talent far outweighs the supply, the bottleneck in marketing data analytics is compounded.

But help is on the way. A recent influx of emerging data technologies has democratized the process of data wrangling, making it easier for marketers to get hands-on with the data they need to drive results. ... " 

Microsoft Pendleton

More on MS Data Wrangling, note Google is currently also working in this space with Trifacta.  Its a big issue in data science work process.  This is often done with tools like Excel, but adding intelligence driven methods can decrease time and remove errors.   How interactive will such an assistant be?  

Microsoft aims to take the work out of data wrangling with coming 'Pendleton' tool
Microsoft is testing privately a tool for data scientists for data preparation and cleaning that is codenamed "Pendleton."     By Mary Jo Foley

R Packages for Data Wrangling

Most of this is quite well known, but provides an excellent lengthy table of resources.  I like the extent of the information, this is an update, there will be things you don't know.  'Data Wrangling' applies to manipulating and examining your data to put it into a form that is suitable for analysis and testing.  It can be time consuming, so doing it efficiently is very useful.  In the past (and still often today) this was done with a package like Excel, or a suitable database system. Often what you know best or what is standard in your business.

 Great R packages for data import, wrangling and visualization
By Sharon Machlis, Executive Editor, Online & Data Analytics, Computerworld

The focus here is on data: from R tips to desktop tools to taking a hard look at data claims.  .... "

Streamlined Text Mining

A  time ago we used relatively simplistic methods of context analysis to understand the meaning of gathered text.  Now with focused methods and increasing amounts of text data,  new results are now possible.  In Datanami: 

Biomedical Text Mining Tool Gets the Lead Out   by George Leopold

Approximately 100 lines of Python code serve as the basis of a new predictive text-mining tool designed to accelerate the scanning online biomedical research papers for clues on everything from repurposing existing drugs to advancing stem cell treatment.

Coders from the Morgridge Institute for Research working in partnership with the University of Wisconsin at Madison reported on their “KinderMiner” algorithm during a bioinformatics conference in San Francisco this week. The researchers said the 100-line algorithm was “within hours” able to scan more than 30 million online papers to provide ranked and relevant associations based on key words and phrases.

“Most often, researchers are running manual Google searches and combing through millions of hits to find, for example, certain genes that are important to a biological process or disease,” explained Ron Stewart, associate director of bioinformatics at the Morgridge Institute. “It’s often based on hunches and intuition. We’re trying to automate and formalize that process.”

Alternative techniques require much data wrangling, added Finn Kuusisto, a postdoctoral researcher at the Morgridge Institute. “We write about 100 lines of Python code, and our users can be given answers that may significantly speed up their scientific process.”

An Advanced Beginners Guide to R

Nicely done.    So you’ve gone through the Computerworld Beginner’s Guide to R and want to take some next steps in your R journey? In this advanced beginner’s guide, you’ll learn data wrangling, best packages to use for different tasks, how to make maps with R and more    .. " 

Examining Data Preparation Tools

Better tools, likely that includes cognitive and context knowledge capabilities are needed.  Also paying more attention to metadata needs:  In O'Reilly:

Why data preparation frameworks rely on human-in-the-loop systems
The O'Reilly Data Show Podcast: Ihab Ilyas on building data wrangling and data enrichment tools in academia and industry. ... " 

Sample R Packages for Input, Wrangling, Visuals

A nice survey by Sharon Machlis, it is always good  to have examples of things you know you need to do, in a language that is relatively new to you.  This is a start.  From here you can go wherever you want:  Great R packages for data import, wrangling & visualization   ...

Data Wrangling the Ecosystem

Nice part two CW article which lays out the existing ecosystem.   Wrangling is a good word for it. This is nice to have because of the complexity of and number of players involved.  Many enterprises are deeply embedded in particular vendors, and it is not just a matter of choosing what, but also how that choice will integrate with the huge investments you have already made.  Good place to start to make sense of it.

Tools for the Data Scientist

A look at the Data Science ecosystem.  Nice to see it all in one place.  By Lukas Biewald.   Would like further to see what all the competing claims are.   Though simply stating the claims will be controversial.   And what the relative costs are.   Reviewing,    More articles are coming.  I do like this description:

" ... Since data science is so inherently cross-functional, many of these companies and tools are hard to categorize. But at the very highest level, they break down into the three main parts of a data scientist's work flow. Namely: getting data, wrangling data and analyzing data. I'll be covering them in that real-world order, starting first with getting data, or data sources. ... "