Reconstructing Hidden Movement from Video

Depending on the quality of this, consider the implications in many areas.  We did lots in the area of video understanding, could have used this.  Below description and videos.

MIT CSAIL’s AI can reconstruct hidden movement from video footage alone  By Kyle Wiggers in Venturebeat

Seeing around corners and through walls is old hat for AI and machine learning algorithms, which are at the heart of systems (some of which use lasers) that produce images outside a sight line. But what about the much more challenging task of reconstructing hidden objects without special equipment?

Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory say they’ve developed exactly that. Their system, which they lay out in a preprint paper published this week, can reconstruct hidden videos from the shadows and reflections on an observed pile of clutter. With nothing more than a video camera switched on in a room, it’s capable of “seeing” around corners even when those corners (and live-action performances) fall outside the camera’s field of view.  .... "

Talk: Sports Summary Highlight Video Construction Using AI

You can now find the recording from Stephen Hammer's excellent Oct 24th talk on improving the fan experience:    "Sports Summary Highlight Videos using AI"

Slides:    http://cognitive-science.info/wp-content/uploads/2019/10/csig_Sports__AI_Hammer_v1.120191024-2.pdf
Talk:      https://youtu.be/StDgf3mnKEU

#CSIGnews #opentechai #ISSIP #Wimbledon @usta #GRAMMYs @BaughmanAaron  @IBMWatsonMedia

Via Susan Malaika

Upcoming and past talks, given most weeks:   http://cognitive-science.info/community/weekly-update/ 

(Updated Time) Talk: Sports Highlight Video Creation

Join our talk on the creation of the IBM sports highlights Videos:

Talk by:Stephen Hammer,  Sports CTO, IBM Distinguished Engineer

#CTO & #IBMDistinguishedEngineer Oct 24 10:30-11:00am US Eastern  - "Sports Summary Highlight Videos using AI" -  #CSIGnews #opentechai #ISSIP @KarolynSchalk @ibrahimatlinux @mattganis @odbmsorg @RiyaMRoy @dzwietering @BedangSen

Zoom meeting Link: https://zoom.us/j/7371462221
Zoom Callin: (415) 762-9988 or (646) 568-7788 Meeting id 7371462221
Zoom International Numbers: https://zoom.us/zoomconference
Thu, Oct 24, 10:30am -11:00am US Eastern   @     https://zoom.us/j/7371462221
More Details and recordings will be posted Here : http://cognitive-science.info/community/weekly-update/

Sponsored by our Cognitive Systems Institute and ISSIP (International Society of Service Innovation Professionals) 

Please pass this on to people with interest in video analysis, AI, Sports marketing, etc.

Franz

AI Enhanced Editing of Sports Coverage

Watched some of the recent Wimbledon, and IBM frequently pointed out that Watson was choosing and editing and delivering the film clips based on measures like human applause.   And then writing copy based on some 20 million clips?     Well it didn't impress me, but I have never been responsible for measuring real time editing of many, many sources of tape.   So it seems it may soon become the standard thing.

IBM’s Wimbledon-watching A.I. is poised to revolutionize sports broadcasts  in DigitalTrends.

Among the most lauded essays ever written about the game of tennis is David Foster Wallace’s 2006 article “Roger Federer as Religious Experience.” Originally appearing in the New York Times, the approximately 6,000-word tribute to one of the world’s most supremely talented players reads, as its title makes clear, more like a divine celebration than a piece of sportswriting.

Wallace (and he was certainly not the first writer to do this) gushed about high-level sporting achievements as though they were more than just superb technique; as if they were, somehow, a transcendent portal to godliness. Ordinary mortals like you and I could comprehend what was happening, but only barely. In order to truly appreciate Federer’s athletic feats, we needed a member of the priesthood — a talented youth player like Wallace had been — who could make it intelligible to us.

Why mention Wallace’s almost decade-and-half old essay on a tech site? Because IBM recently unveiled the latest iteration of its impressive A.I. technology — and it’s learned to appreciate tennis on a whole new level. Well, sort of.   .... " 

Intelligent Display Scans Your Face

Is this invasive?   Saw something like this years ago,  but with no implication of seeing who it was. Advertising based on iris movement and facial analysis to extract demographic information.   Iris scanning is known to be imprecise in uncontrolled environments.

Cooler Screens' Display Cases Scan Your Face to Size Up Buying Habits
Boston Globe   By Hiawatha Bray in Boston Globe

Chicago-based Cooler Screens has developed a facial-profiling system that tries to guess what consumers will buy next based on how they appear. The doors on a Cooler Screens refrigerator are LCD video screens that display images of the items inside the case. In addition, the refrigerator doors are equipped with cameras that send images of each customer to a computer that predicts his or her sex and age. The system uses an iris tracker to detect exactly where in the case the customer is looking. The system instantly analyzes the data on each customer, then starts displaying advertisements on the screen. Drugstore chain Walgreens will test the technology at six of its U.S. locations. ... " 

Video Understanding

Very good piece, with as you might expect, good demonstration video.   This takes such methods beyond captioning and close to what we continually do as humans, visually interpret and understand a contextually changing view.  A sort of visual scene understanding.   Sensory AI at its most useful.

Helping AI master video understanding  By Dan Gutfreund
Video Analytics Scientist, IBM Research AI

I am part of the team at the MIT IBM Watson AI Lab that is carrying out fundamental AI research to push the frontiers of core technologies that will advance the state-of-the-art in AI video comprehension. This is just one example of joint research we’re pursuing together to produce innovations in AI technology that solve real business challenges.

Great progress has been made and I am excited to share that we are releasing the Moments in Time Dataset, a large-scale dataset of one million three-second annotated video clips for action recognition to accelerate the development of technologies and models that enable automatic video understanding for AI.

A lot can happen in a moment of time: a girl kicking a ball, behind her on the path a woman walks her dog, on a park bench nearby a man is reading a book and high above a bird flies in the sky. Humans constantly absorb such moments through their senses and process them swiftly and effortlessly. When asked to describe such a moment, a person can quickly identify objects (girl, ball, bird, book), the scene (park) and the actions that are taking place (kicking, walking, reading, flying). ... " 

Hinton and Capsule Networks

We followed neural networks from their earliest days, but where they failed they did not have the data to consistently work under changing context, and at high enough speed, say for analyzing real-time video.  Geoffrey Hinton's work brought a new approach forward.  Now he addresses some of the lingering problems with these methods:  speed, error rates and the need for huge amounts of data. Tech papers pointed to in the link below.

Google's AI Wizard Takes a New Twist on Neural Networks in Wired, by Tom Simonite.

" ... But Hinton now belittles the technology he helped bring to the world. “I think the way we’re doing computer vision is just wrong,” he says. “It works better than anything else at present but that doesn’t mean it’s right.”

In its place, Hinton has unveiled another “old” idea that might transform how computers see—and reshape AI. That’s important because computer vision is crucial to ideas such as self-driving cars, and having software that plays doctor.

Late last week, Hinton released two research papers that he says prove out an idea he’s been mulling for almost 40 years. “It’s made a lot of intuitive sense to me for a very long time, it just hasn’t worked well,” Hinton says. “We’ve finally got something that works well.”

Hinton’s new approach, known as capsule networks, is a twist on neural networks intended to make machines better able to understand the world through images or video. In one of the papers posted last week, Hinton’s capsule networks matched the accuracy of the best previous techniques on a standard test of how well software can learn to recognize handwritten digits.

In the second, capsule networks almost halved the best previous error rate on a test that challenges software to recognize toys such as trucks and cars from different angles. Hinton has been working on his new technique with colleagues Sara Sabour and Nicholas Frosst at Google’s Toronto office.

Capsule networks aim to remedy a weakness of today’s machine-learning systems that limits their effectiveness. Image-recognition software in use today by Google and others needs a large number of example photos to learn to reliably recognize objects in all kinds of situations. That’s because the software isn’t very good at generalizing what it learns to new scenarios, for example understanding that an object is the same when seen from a new viewpoint.   ...." 

AI Seeking Sharks

Been working recently on detection type algorithms with video feeds.  None are perfectly accurate,  so always consider a risk analysis as well.  Note its augmenting humans here.   Here the liability would seem to be high.

Drones will watch Australian beaches for sharks with AI help
They'll spot sharks with greater accuracy than humans alone.  .... "    By Jon Fingas, @jonfingas  in Engadget.

Intelligent Video Analytics

Recently asked to look at this… for applications beyond security:

What it can do for your business
IBM Intelligent Video Analytics helps security and public safety organizations develop comprehensive security, intelligence and investigative capabilities using video. You can use advanced search, redaction and facial recognition analytics to find relevant images and critical information across multiple video files from multiple camera types. Selected live-streaming cameras plus pre-recorded video ingestion from both fixed cameras and cameras in motion are supported. Augment staff and improve camera investment ROI by extracting information from captured video to uncover insights and patterns.  … "

IBM Advances PowerAI Enterprise Solution

IBM's New PowerAI Features Again Demonstrate Enterprise AI Leadership  by Patrick Moorhead  in Forbes.

What appears to be a semi automation of various areas of machine learning, to make them easier to create and maintain and work with relevant data. The claim is with minimal ML expertise.  Notable to me is one emphasis on the analysis of image and video data.

" ... Recently, IBM announced a significant revamp of PowerAI, seeking to address some of the bigger challenges facing developers and data scientists—cutting down the time required for AI system training significantly, and simplifying the development experience. .... " 

IBM writes about this, and describes the system, which requires POWER HPC hardware, you can test it free at the link:

" .. PowerAI makes deep learning, machine learning, and AI more accessible and more performant. By combining this software platform for deep learning with IBM® Power Systems™, enterprises can rapidly deploy a fully optimized and supported platform for machine learning with blazing performance. The PowerAI platform includes the most popular machine learning frameworks and their dependencies, and it is built for easy and rapid deployment. PowerAI requires installation on IBM Power Systems S822LC for HPC server infrastructure ... " 

See also their blog on OpenPower, and a description of advances.

Video Content Recognition by Microsoft

As part of Microsoft Build Meetings.  Following and looking for more details.

Microsoft Takes Aim at Google and Amazon With a New Recognition Tool  Reuters

Microsoft on Wednesday turned up the heat on other technology giants by launching new image and video recognition products, which could help it court businesses worried about running ads next to offensive content.

The Redmond, Washington-based company said its new Video Indexer can identify faces, voices, and emotions in moving pictures. Separately, its Custom Vision Search lets companies build apps that recognize images with just a few lines of code.  ... " 

More:   A preview.    Use Cases,     Identifying emotions.

Fooling Video Classification

In the Verge ...

Google’s AI video classifiers are easily fooled by subliminal images  by James Vincent   @jjvincent

" ... A recent research paper, published by the University of Washington and spotted by Quartz, makes the problem clear. It tests Google’s Cloud Video Intelligence API, which is designed to be used by clients to automatically classify the content of videos with object recognition. (The system is in private beta and not currently in use on YouTube or any other Google products.) The API, which is powered by deep neural networks, works very well against regular videos, but researchers found it was easily tricked by a determined adversary. ... " 

Jambaar for Video Content Management

Brought to my attention:

.... Jambaar is your co-creator in driving better video content results. Our subscription model makes it faster and more affordable than any other solution.   ... 

• 100% Faster and more cost effective creation
• 10x more video destinations to reach your audience
• AI predictive intel for content strategy development
• Fast implementation & integration
• Content Asset management

Our Video Intelligence tm AI Platform

Say Hello to Jambaar's advanced video analytics for business built with artificial intelligence (AI).
Predictive AI analytics and data visualization built to give you better business performance with video content. 

Get answers and new insights to make confident video content decisions in minutes. 

Make the best use of your budget by predictively targeting your audience based on key performance indicators (KPI).

Jambaar Leadership:
"As a Former P&G Executive with 25 years in the marketing and video production industry with Fortune 100 companies, I set off on a mission to solve one of the most complex and cumbersome business processes in the industry. It's time for companies to have a better video content solution. That's why Jambaar exists."

Anne Chambers

Machine Learning Images from Mobile Video

Continued interest in fast image analysis.  Has implications for the perception of images, and thus space and what exists within it.

Google acquires image recognition startup Moodstocks to bolster machine learning efforts by Maria Deutscher, 

   "... Google acquires image recognition startup Moodstocks to bolster machine learning efforts  by Maria Deutscher   " ... The Paris-based startup has developed an image recognition tool specifically optimized to process media captured via mobile devices. ... " 

Urban Intelligence

A system called Placemeter takes as input video feeds and analyzes images to determine what is going on outside your doors.  Possible retail measures?    Video content analysis (VCA).  

" .... In its first iteration, the service, which relies on real-time video feeds, was able to quantify the overall number of objects that it saw and distinguish between pedestrians and vehicles.

Now, the service is getting significantly smarter. By default, Placemeter can now distinguish between five different objects: people, bicycles, motorcycles, cars and large vehicles (think trucks, delivery vans, etc.). ... "        In TechCrunch.

Facebook Video AI

More plans to roll AI into commonly used software:

I’m going to make Facebook’s AI predict what happens in videos
.... Facebook unveiled several artificial intelligence projects. Yann Lecun, the company's director of AI, reveals what this technology can do.... "

Advanced Pedestrian Detection/Analysis

Detecting and counting pedestrians is not new, has been done since the 90s, but advanced analysis of real time video is rapidly changing.   Called VCA, or Video content analysis.  Interesting that Google is involved.  Likely for driver less car applications.  But possible for other retail applications, which we also examined.  Consider also how such methods can be connected with digital signage.

" ... Google Research has quietly made a breakthrough in the emerging area of image recognition and rapid video analysis — a breakthrough that has significant implications for pedestrian detection. Pedestrian detection refers to the analysis of statistics of pedestrian scale, occlusion and location. It gives detection systems a base of knowledge under which to operate ......  As an experiment [PDF], Joe Yue-Hei Ng, Matthew Hausknecht, Sudheendra Vijayanarasimhan, Oriol Vinyals, Rajat Monga and George Toderici — researchers from the University of Maryland at College Park, the University of Texas at Austin and Google — applied several deep neural network architectures to analyze a dataset of videos over longer time periods than previously attempted. ... " 

See a long ago item on project BlueEyes we participated in.

Deep Learning and Real Time Video Surveillance

Thought this was being done already, in fact saw a demonstration of this several years ago,  but it may well be in the details of implementation.  Examining further.

In Fast Company:
Using Deep Learning to Make Video Surveillance Smarter
Startup Camio is drawing on neural networks to better identify who—or what—is outside your door. ... " 

Cisco Managing Video

I reported on this some time ago, as it related to Video Content Analysis. (VCA) and its use for ad management in store.

Cisco introduces ‘videoscape unity’ to manage the future of video everywhere
Cisco is the world’s networking giant, and today it showed what it is doing to make sure that it can provide the plumbing for the explosion of video on live TV, paid services, and the wide-open Internet.  ... 

For the consumer, this means that Cisco will make it easier to manage video content, regardless of where it is coming from. Cisco is creating apps that allow a user to search through videos in a visual way, organized based on the user’s own tastes. You can login and your own profile will come up. The user interface will show you the videos that are relevant to you. Cisco refers to its unified approaching to managing content as Videoscape Unity.

Sighthound Computer Vision

Powered by Computer Vision

" ... Sighthound uses computer vision technology to distinguish between people and objects. By using advanced algorithms, Sighthound is able to analyze video streams to detect, track, and recognize people and objects.

Computer vision has been around for a decade or more, but only in the last few years has the science developed to the point of commercial viability. While motion detection software was useful in telling a camera when to start recording, its inability to know what is was seeing has led to billions of hours of wasted footage.  ... "