Learn from One Look

From the Google AI Blog,  Fascinating, Requiring less data for recognition.  intro is useful intro.   Link to talk,  ultimately technical.

LOLNeRF: Learn from One Look

Tuesday, September 13, 2022

Posted by Daniel Rebain, Student Researcher, and Mark Matthews, Senior Software Engineer, Google Research, Perception Team

An important aspect of human vision is our ability to comprehend 3D shape from the 2D images we observe. Achieving this kind of understanding with computer vision systems has been a fundamental challenge in the field. Many successful approaches rely on multi-view data, where two or more images of the same scene are available from different perspectives, which makes it much easier to infer the 3D shape of objects in the images.

There are, however, many situations where it would be useful to know 3D structure from a single image, but this problem is generally difficult or impossible to solve. For example, it isn’t necessarily possible to tell the difference between an image of an actual beach and an image of a flat poster of the same beach. However it is possible to estimate 3D structure based on what kind of 3D objects occur commonly and what similar structures look like from different perspectives.

In “LOLNeRF: Learn from One Look”   , presented at CVPR 2022, we propose a framework that learns to model 3D structure and appearance from collections of single-view images. LOLNeRF learns the typical 3D structure of a class of objects, such as cars, human faces or cats, but only from single views of any one object, never the same object twice. We build our approach by combining Generative Latent Optimization (GLO) and neural radiance fields (NeRF) to achieve state-of-the-art results for novel view synthesis and competitive results for depth estimation. .... ' (much detail) 

AI Based Gun Image Detection

Useful in todays world. False positives?  Easy concealment?

Oxford High Turned to AI to Detect Weapons After Shooting  By The Detroit News, June 2, 2022

Oxford High School in Michigan has been testing artificial intelligence (AI)-based gun detection software since April in the wake of a shooting last year.  The software from AI gun detection technology developer ZeroEyes connects with interior and exterior surveillance cameras, and can flag a firearm carried in plain sight in seconds.

ZeroEyes personnel monitor the system around the clock from an operations center, capturing images and recording the location if a gun is spotted.

Company co-founder Mike Lahiff said ZeroEyes trains its AI to detect guns based on hundreds of thousands of proprietary images and videos, and does not collect biometric data or use facial recognition.

The Oxford Community Schools district also is assessing other security measures, like Evolv biometric scanners for detecting weapons, and window shields...

Earlier this year, the Oxford school district started installing firearm recognition artificial intelligence software that scans security camera footage for semi- or fully brandished weapons.

Detroit News   

Microsoft says its AI can describe images 'as well as people do'

 Recall some portion of this claim being made, but had not heard much new from Microsoft,  Know sight impaired people who could use the capability.     We also worked on the general idea of  'captioning', which turns out to be tough to do generally well.  

Microsoft says its AI can Describe Images 'as Well as People Do'  By Devindra Hardawar, @devindra  in Engdget 

It’s a new milestone for AI that could genuinely help the visually impaired. 

Describing an image accurately, and not just like a clueless robot, has long been the goal of AI. In 2016, Google said its artificial intelligence could caption images almost as well as humans, with 94 percent accuracy. Now Microsoft says it’s gone even further: Its researchers have built an AI system that’s even more accurate than humans — so much so that it now sits at the top of the leaderboard for the nocaps image captioning benchmark. Microsoft claims its two times better than the image captioning model it’s been using since 2015. 

And while that’s a notable milestone on its own, Microsoft isn’t just keeping this tech to itself. It’s now offering the new captioning model as part of Azure's Cognitive Services, so any developer can bring it into their apps. It’s also available today in Seeing AI, Microsoft's app for blind and visually impaired users that can narrative the world around them. And later this year, the captioning model will also improve your presentations in PowerPoint for the web, Windows and Mac. It’ll also pop up in Word and Outlook on desktop platforms.

Classifying Images with Quantum

Sounds remarkable and instructive about how quantum can be used for some kinds of machine learning approaches.

Google researchers use quantum computing to help improve image classification  in VentureBeat by Kyle Wiggers

In a new tutorial, Google researchers demonstrate how quantum computing techniques can be used to classify 28-pixel-by-28-pixel images illuminated by a single photon. By transforming the quantum state of the said photon, they show they’re able to achieve “at least” 41.27% accuracy on the popular MINST corpus of handwritten digits — a 21.27% improvement over classical computing approaches. ... "

Technical paper from Google

Image GPT

Been looking at GPT from OpenAI, and at their site found Image GPT.   There a number of links to technical papers at the pieces below:

OpenAI first described GPT-3 in a research paper published in May. But last week it began drip-feeding the software to selected people who requested access to a private beta. For now, OpenAI wants outside developers to help it explore what GPT-3 can do, but it plans to turn the tool into a commercial product later this year, offering businesses a paid-for subscription to the AI via the cloud. ... 

Image GPT
We find that, just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. By establishing a correlation between sample quality and image classification accuracy, we show that our best generative model also contains features competitive with top convolutional nets in the unsupervised setting.

Introduction
Unsupervised and self-supervised learning,1 or learning without human-labeled data, is a longstanding challenge of machine learning. Recently, it has seen incredible success in language, as transformer2 models like BERT,3 GPT-2,4 RoBERTa,5 T5,6 and other variants78910 have achieved top performance on a wide array of language tasks. However, the same broad class of models has not been successful in producing strong features for image classification.11 Our work aims to understand and bridge this gap.

Transformer models like BERT and GPT-2 are domain agnostic, meaning that they can be directly applied to 1-D sequences of any form. When we train GPT-2 on images unrolled into long sequences of pixels, which we call iGPT, we find that the model appears to understand 2-D image characteristics such as object appearance and category. This is evidenced by the diverse range of coherent image samples it generates, even without the guidance of human provided labels. As further proof, features from the model achieve state-of-the-art performance on a number of classification datasets and near state-of-the-art unsupervised accuracy[1]  ...  " 

Google Announces Experimental Means to Detect Altered Images

A useful kind of AI, checking out the details.  Still experimental.  How often we be sure of the results?  Looks for specific means of alteration, So will have to be maintained for new ones.

How can technology strengthen fact-checking?
As the technology to create realistic fake images, video and audio becomes more sophisticated, fact-checkers and journalists need similarly advanced tools to counter this threat. Assembler is an experimental platform from Jigsaw and Google Research that hopes to make it easier to judge manipulated media and help prevent the spread of disinformation.

Assembler analyzes images using detectors — technology trained to identify specific types of manipulation — and evaluates if and where images may have been altered.  ... " 

More.

Military Long Distant Face Recognition

I would imagine will be more precise and at longer distances yet,   I have some very impressive sat images, and precise coordinates could simply be shared.   All this will change the face of war.  Likely very cheaply and fast.

US military face recognition system could work from 1 kilometre away    By David Hambling in NewScientist

The US military is developing a portable face-recognition device capable of identifying individuals from a kilometre away.

The Advanced Tactical Facial Recognition at a Distance Technology project is being carried out for US Special Operations Command (SOCOM). It commenced in 2016, and a working prototype was demonstrated in December 2019, paving the way for a production version. SOCOM says the research is ongoing, but declined to comment further.

Initially designed for hand-held use, the technology could also be used from drones. SOCOM documents …  " 

More,  

Malevolence of the Use of Evolving Images

Good-non technical view of the current state of creating and evolving images. Somewhat like the 'photo shopping' enigma still going on, but more subtle and automated.   At first this seems like its not malevolent at all,  just amusing,  but it shows how an AI can be misled, depending how its used by people.

Malevolent Machine Learning   By Chris Edwards in the CACM

Communications of the ACM, December 2019, Vol. 62 No. 12, Pages 13-15
10.1145/3365573

At the start of the decade, deep learning restored the reputation of artificial intelligence (AI) following years stuck in a technological winter. Within a few years of becoming computationally feasible, systems trained on thousands of labeled examples began to exceed the performance of humans on specific tasks. One was able to decode road signs that had been rendered almost completely unreadable by the bleaching action of the sun, for example.

It just as quickly became apparent, however, that the same systems could just as easily be misled.

In 2013, Christian Szegedy and colleagues working at Google Brain found subtle pixel-level changes, imperceptible to a human, that extended across the image would lead to a bright yellow U.S. school bus being classified by a deep neural network (DNN) as an ostrich.

Two years later, Anh Nguyen, then a Ph.D. student at the University of Wyoming, and colleagues developed what they referre3d to as "evolved images." Some were regular patterns with added noise; others looked like the static from an analog TV broadcast. Both were just abstract images to humans, but these evolved images would be classified by DNNs trained on conventional photographs as cheetahs, armadillos, motorcycles, and whatever else the system had been trained to recognize. ... "

Driving School for Computers

Image recognition and adjustment to goals.  Interesting process included.

Driving School for Computers
Ruhr-University Bochum
By Meike Drießen

Researchers at Ruhr-University Bochum (RUB)'s Neural Computation Institute in Germany have devised a technique to automatically generate traffic signs that computers can employ to practice vision. The method involves two algorithms: one is fed simple pictograms of road signs, then is tasked with rendering them into photo-like images. The second algorithm must decide if the rendered image is an actual or created photo. RUB's Sebastian Houben said, "Moreover, the second algorithm indicates to the first one in what way the selection process could be made even more difficult." After several days of this algorithmic sparring, the researchers check the resulting signs' realism, and refine the software if necessary. Both algorithms were able to outperform humans in recognizing images.  ... "

Building a Computer Vision Model

A simplified, straightforward tutoral on a computer vision model.   This is the place you can get something impressive out of neural nets,  and an intro to the general AI method along them way.       Of most use too, pointers to existing databases to get started with.  We used ImageNet and WordNet tags, for example.

From KDNuggets:

How can we build a computer vision model using CNNs? What are existing datasets? And what are approaches to train the model? This article provides an answer to these essential questions when trying to understand the most important concepts of computer vision.  

By Javier Couto, Tryolabs.

Computer vision is one of the hottest subfields of machine learning, given its wide variety of applications and tremendous potential. Its goal: to replicate the powerful capacities of human vision. But how is this achieved with algorithms?

Let's have a loot at the most important datasets and approaches.

Existing datasets
Computer vision algorithms are no magic. They need data to work, and they can only be as good as the data you feed in. These are different sources to collect the right data, depending on the task:

One of the most voluminous and well known dataset is ImageNet, a readily-available dataset of 14 million images manually annotated using WordNet concepts. Within the global dataset, 1 million images contain bounding box annotations.  .... "

Ultra Long Range Cameras

Further advances with Camera and image interpretation technology.  The AI to recompose the image from multiple components.

An Ultra Long Range Camera  Daily Mail (United Kingdom)
By James Pero

Researchers in China have developed new camera technology that can render human-sized subjects from up to 28 miles away. The camera, which relies on a combination of laser images and advanced artificial intelligence software, can cut through smog and other pollution. The software uses a technique called "gating" that helps ignore photons reflected by other objects in the camera's field of view. Since the camera uses a laser to determine the distance of a subject by measuring how long the light takes to reflect back to the machine, the new software can tell the camera to ignore everything else that falls outside of that time signature. The camera uses a new algorithm to stitch together the data collected and form a recognizable image.  .... " 

Speeding up Biological Imaging

An example of the sharing of methodology to create algorithms.  Note also the replacement and augmentation of human process.

Researchers Use Algorithm From Netflix Challenge to Speed Up Biological Imaging 
Optical Society of America

Scientists at the Ecole Normale Superieure in France have repurposed an algorithm developed for Netflix's 2009 movie preference prediction challenge for high-speed acquisition of classical Raman spectroscopy biological-tissue images. The researchers demonstrated imaging speeds of a few tens of seconds for an image that would usually take minutes to obtain, and they think sub-second speeds could be realized in the future. Said Ecole Normale Superieure's Hilton de Aguiar, "We combined compressive imaging with fast computer algorithms that provide the kind of images clinicians use to diagnose patients, but rapidly and without laborious manual post-processing." The researchers replaced costly, slow cameras used in conventional setups with a spatial light modulator, which selects groups of wavelengths identified by a single-pixel detector, compressing images as they are captured. This allowed the team to use a portion of the data typically required for non-invasive Raman spectroscopy, and employ the Netflix algorithm to fill in the missing information .... "

Resolution Limits for Image Analysis

Brought to my attention for a particular healthcare application.  Technical.  Comments?

 Resolution limit of image analysis algorithms

By Edward A. K. Cohen, Anish V. Abraham, Sreevidhya Ramakrishnan & Raimund J. Ober   in Nature Communications

Abstract:
The resolution of an imaging system is a key property that, despite many advances in optical imaging methods, remains difficult to define and apply. Rayleigh’s and Abbe’s resolution criteria were developed for observations with the human eye. However, modern imaging data is typically acquired on highly sensitive cameras and often requires complex image processing algorithms to analyze. Currently, no approaches are available for evaluating the resolving capability of such image processing algorithms that are now central to the analysis of imaging data, particularly location-based imaging data. Using methods of spatial statistics, we develop a novel algorithmic resolution limit to evaluate the resolving capabilities of location-based image processing algorithms. We show how insufficient algorithmic resolution can impact the outcome of location-based image analysis and present an approach to account for algorithmic resolution in the analysis of spatial location patterns.
...."

New Means of Seeing Objects

Closer to biomimicry, the article says a not deeply technical description:

New AI computer vision system mimics how humans visualize and identify objects
 UCLA Samueli School of Engineering

Summary:
Researchers have demonstrated a computer system that can discover and identify the real-world objects it 'sees' based on the same method of visual learning that humans use.
Researchers from UCLA Samueli School of Engineering and Stanford have demonstrated a computer system that can discover and identify the real-world objects it "sees" based on the same method of visual learning that humans use.

The system is an advance in a type of technology called "computer vision," which enables computers to read and identify visual images. It is an important step toward general artificial intelligence systems -- computers that learn on their own, are intuitive, make decisions based on reasoning and interact with humans in a more human-like way. Although current AI computer vision systems are increasingly powerful and capable, they are task-specific, meaning their ability to identify what they see is limited by how much they have been trained and programmed by humans.

Even today's best computer vision systems cannot create a full picture of an object after seeing only certain parts of it -- and the systems can be fooled by viewing the object in an unfamiliar setting. Engineers are aiming to make computer systems with those abilities -- just like humans can understand that they are looking at a dog, even if the animal is hiding behind a chair and only the paws and tail are visible. Humans, of course, can also easily intuit where the dog's head and the rest of its body are, but that ability still eludes most artificial intelligence systems. .... " 

On Google Lens

God piece on where it is and where it is going.    Impressive so far,  it gives you some remarkable results, but not always exactly what you need.   The notion of precision search using images, say from a camera,  is quite different than text search.

The era of the camera: Google Lens, one year in
By Aparna Chennapragada      VP, Google Lens and AR

There is, of course, the vacation beach pic, the kid’s winter recital, and the one--or ten--obligatory goofy selfie(s). But there’s also the book that caught my eye at a friend’s place, the screenshot of an insightful tweet and the tracking number on a package.

As our phones go everywhere with us, and storage becomes cheaper, we’re taking more photos of more types of things. We’re of course capturing sunsets and selfies, but people say 10 to 15 percent of the pictures being taken are of practical things like receipts and shopping lists.

To me, using our cameras to help us with our day-to-day activities makes sense at a fundamental human level. We are visual beings—by some estimates, 30 percent of the neurons in the cortex of our brain are for vision. Every waking moment, we rely on our vision to make sense of our surroundings, remember all sorts of information, and explore the world around us.  

The way we use our cameras is not the only thing that’s changing: the tech behind our  cameras is evolving too. As hardware, software, and AI continue to advance, I believe the camera will go well beyond taking photos—it will help you search what you see, browse the world around you, and get things done.

That’s why we started Google Lens last year as a first step in this journey. Last week, we launched a redesigned Lens experience across Android and iOS, and brought it to iOS users via the Google app.

I’ve spent the last decade leading teams that build products which use AI to help people in their daily lives, through Search, Assistant and now Google Lens. I see the camera opening up a whole new set of opportunities for information discovery and assistance. Here are just a few that we’re addressing with Lens: .... " 

Google Lens Can Detect a Billion Objects

I noted that the IOS version of Google Lens has been updated to a more stable condition..  Impressive capabilities.  Been experimenting with it to construct reference images for machine learning experiments.   Images will be captured from sources like broadcasts and other advertising resources. 

The claim is that Google Lens 'can now detect over 1 Billion objects': Quote and more from Techspot.

AI Design Trend Inspiration

Several of my students looked at startups in the fashion industry, that had design, marketing  and customer engagement angles, so this is interesting.  Have not seen to much for AI in design/creativity over the years, so that too is intriguing.   Note also the use of image recognition.

Yoox looks seeks AI’s help for design inspiration   by Matthew Stern  in Retailware  With retail expert comments.

Designers of Yoox’s latest apparel line are getting a little inspirational help; the line is based on information collected, aggregated and “powered by artificial intelligence.”

Yoox’s AI tools review content across social media and online magazines to furnish the retailer’s designers with data on what styles are being discussed and shared. Inspiration also comes from predictive indicators into emerging lifestyle and style trends, analysis of internal data from products sold on its site, customer feedback, industry purchasing trends, as well as text search and image recognition.

“We are always exploring the frontier between Human and Machine; it’s been part of our DNA since I invented Yoox in 1999,” said Federico Marchetti, CEO of Yoox Net-A-Porter Group, in a statement. .... "

Computers Recognizing and Labeling Images

We spent quite a lot of time in the late 90s addressing the labeling of photographs.   So we knew it was hard in general.    Now its become much easier.  Whether it be face recognition in real time, or the ability to label all the common elements in a photo. Not perfectly yet, it can depend on the quality of the image, just like we can be fooled by an image taken in near darkness, or some unusual juxtaposition of things in an image.  But it has become very good.  A high percentage of what we would call 'good' pictures, not meant to fool.   Why?  Good piece below on this and pointer to a paper.   And I can see why we did not figure it out then. We were then pointed in the right direction, but did not emphasize the right elements of learning,  the Data.

How computers got shockingly good at recognizing images
A landmark 2012 paper transformed how software recognizes images. By Timothy B Lee in ArsTechnica  ... "

See the technical 2012 paper.  The date of the emergence of  today's AI.  At the time we were following one of the authors,  Geoffrey Hinton.  Note the use of very large databases, a key element we never fully understood, we had examined similar architectures, but expected them to work with far less data.

And more overview in this book:

How Smart Machines Think   By Sean Gerrish   Foreword by Kevin Scott  MIT Press

Everything you've always wanted to know about self-driving cars, Netflix recommendations, IBM's Watson, and video game-playing computer programs. ... " 

Unilever Shopkick and Image reco

Unusual promo that includes image recognition.

Suave Promotion Brings Image Recognition to Walmart   By CGT Staff 

Unilever’s Suave brand is promoting a Veteran’s Day cause program at Walmart using new 2D image recognition technology shopper rewards app Shopkick.

From Oct. 14 to Nov. 18, Suave is donating $1 to the nonprofit Fisher House Foundation for every limited-edition Suave package purchased. The brand’s goal is to raise $1 million.

Shopkick members can use the app to interact with the Fisher House “Helping Military Families” logo on Suave’s promotional packaging to receive “kicks” — the Shopkick program’s rewards currency.

In addition, Shopkick will donate 10% of the overall campaign budget to Fisher House, which provides access to a home-away-from-home (at no cost) for the loved ones of hospitalized veterans and active military personnel.

“2D image recognition has benefits for both brands and consumers,” said Shopkick’s chief executive officer Adam Sand. “Shoppers learn about product and company benefits and brands bring attention to cool campaigns, new product or packaging and what makes them unique.” ... " 

Using Google Lens on IOS

Been exploring Google Lens on an Iphone via images placed in Google Photos and Google Assistant.  Google lens is a means of interpreting images.  For example it can interpret the text on a sign and show it to you.  More complexly it can take an image of a tree or plant leaf and it will tell you what it is.  It will identify a location.  If it identifies a commercial thing, it will link you to purchase.   Or a person based on your previous ID of a person in a picture.   Text Translation from signs.  Store names.  And more.   Uses AI based image recognition.   On Pixel phones it can do the recognition directly from the camera view.   Very cool thing, reminding me of Google Goggles from long ago.  I like much where this is going.

That being said, it says it is 'still learning' and does not always get the correct answer.   For example I showed it a mulberry leaf in a clear flat image, a very distinctive leaf form, and it identified it as one of four different images, including a maple leaf, which is very different.   So you can't necessarily rely on the result.  As a naturalist, will try it with other plant images.    Also having been involved with petro geology, identifying rock samples?

Learn and do more with Lens in Google Images
By Assaf Broitman    Product Manager, Google Images

On Google’s 20th anniversary last month, we announced updates to make Google Images a more useful place to explore the web through images. One of the ways we’re doing this is by bringing Lens to Google Images. We launched Lens to help you do more with what you see. People already love using it in their camera and on their photos–to find items in an outfit they like, learn more about landmarks, and identify that cute dog in the park. Lens is a natural fit for Google Images.  ... "