Google says its Lens image search can now help identify skin conditions

Recall talking to Google on such capabilities. 

Google says its Lens image search can now help identify skin conditions

‘Just take a picture or upload a photo through Lens, and you’ll find visual matches to inform your search.’

By Jon Porter, a reporter with five years of experience covering consumer tech releases, EU tech policy, online platforms, and mechanical keyboards.

Jun 15, 2023, 8:04 AM EDT 

Google Lens is now able to offer more information on that pesky rash that you’re not sure whether to worry about. In a blog post published this week, Google outlined how the Lens image search feature built into its apps on iOS and Android can “search for skin conditions” like an “odd mole or rash.” It’ll also work on other parts of your body if you want more information about a bump on your lip, line on a nail, or hair loss from your scalp. 

“Just take a picture or upload a photo through Lens, and you’ll find visual matches to inform your search,” the blog post reads. Crucially, however, Google specifically warns that results are “informational only and not a diagnosis” and says users should “consult your medical authority for advice.” 

Reading Barely Open Books

 When I first saw this I thought it was a cautionary piece about security, but no, is aimed at reading fragile books without opening them fully.   I recall our visit to a company that was scanning books for Googles' Read program.  Could have been useful there.   Been a while, have to check the status of that project.

Imaging System Captures Text From Barely Open Books, By University of Rochester, October 27, 2021

A new imaging methodology developed by the University of Rochester's Gregory Heyworth and colleagues can capture text from extremely fragile books. The technique can generate digital images of manuscripts and rare, frail tomes without opening them more than 30 degrees.

Heyworth and researchers in Rochester's text-retrieval Lazarus Project initiative built a system with a view camera featuring a twistable section, which photographs a book held open by a protective cradle.

Software corrects the resulting image's distortion, enabling a page to be read as if the book were lying flat. The system also captures multiple wavelengths of light, picking up features invisible to the eye while correcting the aged pages' coloration.

From University of Rochester

View Full Article

Apple Delays iPhone Scanning for Child Abuse

It was warned that such methods could be modified to 'scan' any kind of text content in a phone, and lead to a backdoor being established.

Apple delays plan to scan iPhones for child abuse  in the BBC

Apple has delayed plans to roll out detection technology which would have scanned US users' iPhones in search of child sexual abuse material.

It follows widespread criticism from privacy groups and others, worried that the on-device tracking set  dangerous precedent.

Apple said that it had listened to the negative feedback and was reconsidering.

There were concerns the system could be abused by authoritarian states.

The so-called NeuralHash technology would have scanned images just before they are uploaded to iCloud Photos. Then it would have matched them against known child sexual abuse material on a database maintained by the National Center for Missing and Exploited Children.

If a match was found then it would have been manually reviewed by a human and, if required, steps taken to disable a user's account and report it to law enforcement.

It was due to launch later in the year.   ... ' 

LiDAR Test Starts up Again

 Am now finally getting into my LIDAR test,  on a Apple 12 Pro phone for now.   Using the package Polycam, which looks easy to use and is free for the initial testing, then has a monthly fee.  Examples of use here:  https://sketchfab.com/polycam/collections/polycam    And allows you to share scanned models with other packages.    Anyone else doing this?  Lets share a test!

More about how to use LiDAR here.    Below is more background, repeating ... what I placed here earlier to give background.

Did you buy a new iPad Pro? How about a new iPhone 12 Pro? These are the LiDAR apps to try right now, regardless of your background or experience.   By  Brian M Wolfe

The second-generation 11-inch iPad Pro, fourth-generation 12.9-inch iPad Pro, and now the iPhone 12 Pro/Pro Max offer a LiDAR (Light Detection and Ranging) Scanner that will play a huge role in Apple's augmented reality initiatives in the years to come.

Here's more about LiDAR and apps you can use right now on your new iPad Pro to test the technology.

What is LiDAR?

LiDAR uses light to measure distance using invisible pulsating green spectrum lasers. These pulses (which occur thousands of times per minute) measure how long it takes for the light to get back to the sensor. By doing so, it paints a "picture" of the environment in front of the scanner. .... "

Scanning License Plates

The capability is well know, but the ability to save and 'mine' large libraries of the data makes some concerned. 

License-Plate Scans Aid Crime-Solving But Spur Little Privacy Debate  By The Wall Street Journal,  March 15, 2021

License-plate readers are feeding immense databases with details on Americans driving habits, helping to solve crimes despite little public awareness about the breadth of the data collected or how it is used.

Law enforcement agencies increasingly are using data gathered by the vast network of automated license-plate scanners to solve crimes.The scanners initially were placed on telephone poles, police cars, toll plazas, bridges, and in parking lots, but now can be found on tow trucks and municipal garbage trucks as well.

License-plate scans were instrumental in the arrests of several suspected rioters at the U.S. Capitol.  However, there are concerns about abuse, misidentification, and the scope of data collection, given that, for instance, some systems read a plate's number but not its state.

Electronic Frontier Foundation's Dave Maass said, "License-plate readers are mass surveillance technology. They are collecting data on everyone regardless of whether there is a connection to a crime, and they are storing that data for long periods of time."

From The Wall Street Journal

Using AI to Control a Military Jet

As described a considerable application, combining  autonomous signals and human decisions.

In a First, USAF Uses AI on Military Jet

The Washington Post  By Aaron Gregg

U.S. Air Force (USAF) officials said artificial intelligence (AI) was used for the first time on a military aircraft, when an algorithm called ARTUµ on Tuesday controlled radar and tactical navigation on a U-2 Dragon Lady spy plane during a training flight. USAF’s Josh Benedetti said ARTUµ intentionally lacked a manual override to "provoke thought and learning in the test environment." People involved in the test said the AI performed specific tasks while separated from the flight controls, which a human pilot handled. Assistant Air Force Secretary Will Roper said ARTUµ was trained against an opposing computer to scan for oncoming missile and missile launchers, and had the final say on where to orient the plane's sensors. ARTUµ is based on open source algorithms, and Roper said the purpose of test is "to shock the Air Force and the [Defense] Department as a whole into how seriously we need to treat AI teaming."

Dopamine and Motivation Scan

We spend much time trying to understand how fmri scans could be used to judge potential engagement with product. Could this new Dopamine 'scan' be used to understand this more effectively?

 How dopamine drives brain activity
A specialized MRI sensor reveals the neurotransmitter’s influence on neural activity throughout the brain.

By Anne Trafton | MIT News Office

Using a specialized magnetic resonance imaging (MRI) sensor, MIT neuroscientists have discovered how dopamine released deep within the brain influences both nearby and distant brain regions.

Dopamine plays many roles in the brain, most notably related to movement, motivation, and reinforcement of behavior. However, until now it has been difficult to study precisely how a flood of dopamine affects neural activity throughout the brain. Using their new technique, the MIT team found that dopamine appears to exert significant effects in two regions of the brain’s cortex, including the motor cortex.

“There has been a lot of work on the immediate cellular consequences of dopamine release, but here what we’re looking at are the consequences of what dopamine is doing on a more brain-wide level,” says Alan Jasanoff, an MIT professor of biological engineering, brain and cognitive sciences, and nuclear science and engineering. Jasanoff is also an associate member of MIT’s McGovern Institute for Brain Research and the senior author of the study.

The MIT team found that in addition to the motor cortex, the remote brain area most affected by dopamine is the insular cortex. This region is critical for many cognitive functions related to perception of the body’s internal states, including physical and emotional states.

MIT postdoc Nan Li is the lead author of the study, which appears today in Nature.   ... "

Brain Scans Will Identify You

Not unexpected, with that much data.   But can it be done externally?  Methods are evolving in many kinds of biometrics.

Someday a Computer May Use Brain Scans to Identify You
The New York Times   By Gina Kolata

Mayo Clinic investigators said facial recognition software could be used to match photos of people to facial reconstructions derived from magnetic resonance imaging (MRI) scans of their heads. The University of Pennsylvania's Aaron Roth warned this technique eventually will be used to compromise stored medical data. The University of California, San Francisco's Michael Weiner said the Mayo Clinic's findings represent a threat to privacy, citing the Alzheimer's Disease Neuroimaging Initiative as a potential target. The Initiative has MRI brain scans that include participants' faces, with identifying data removed; Weiner suggested attackers could match those MRIs to images of study subjects elsewhere .... "

Sams Can Scan Product as Opposed to Barcodes

Different than Amazon Go,  probably will not get the same same exposure.

Sam’s Club Files Patent for Scan & Go Technology
Retailer's members will begin testing the new app this spring     By WGB Staff

Walmart's Sam’s Club division has developed what it describes as "next-level" scan and go technology in its Dallas-based innovation lab that will enable members to recognize products from any angle without requiring them to search for a barcode on the package. The membership warehouse club retailer has filed a patent for its new mobile-first shopping experience and will soon allow members to test it beginning this spring using the Sam’s Club Now app.

In a blog post on Sam's Club's website, Eddie Garcia, VP of product and member experience, said that the retailer anticipates incorporating the same technology, which uses computer vision and machine learning, for associates to use as well to improve productivity and make it easier to serve members.

"At its core, Sam’s Club Now will be a technology lab that doubles as a live, retail club. It’s where we will incubate, test and refine technologies to help define the future of retail," Garcia said. "The future of retail is as much about people as it is about technology. ... " 

Illinois Biometric Privacy Act and Google

Notable use of legal regulation against new uses of AI.  In TheVerge.    More details at the link, see also the link to the Illinois Privacy Act, which was at issue.

Google just got an important lawsuit over facial recognition dismissed. As first reported by Bloomberg, the lawsuit has been dismissed by a state judge who found that the plaintiffs didn’t suffer “concrete injuries.” The Google lawsuit is one of three cases aimed at prominent tech companies that have allegedly violated the United States’ toughest biometric privacy law and it’s the first one to get dismissed.

The Illinois Biometric Information Privacy Act has long been a huge obstacle for tech companies working on facial recognition initiatives. The law requires companies to obtain people’s explicit permission before they can make biometric scans of their bodies. ...  "

Who Owns Precise Data Descriptions of Things?

Fascinating look at the ownership of metadata.  Here a more precise description of things that have ever been created.  And then that description allows to do analytics that we could never have done before.  Even analyses we still have not thought of yet.  Fascinating view of who is doing this and the approaches involved.  And then the ownership problem.

Who Owns 3D Scans of Historic Sites?   By Esther Shein 
Communications of the ACM, January 2019, Vol. 62 No. 1, Pages 15-17
10.1145/3290410  ... "

Wal-Mart and Bossa Nova Robotics

Much more on Wal-Mart's testing of robotics for in store tasks in 50 stores, such as scanning, auditing and stocking decision support.   The shelf can become smart through external scanning.  Also mention of a company Bossa Nova Robotics, which is being tested.  Quite more at the Bossa Nova site, including video of their systems in operation.   No indication of interactions with shoppers.   Very much retail oriented, they write: 

" ... Safe & friendly

A store is a busy place, so we know how important it is that our robot has a sense of its surroundings. That’s why it can detect objects in an aisle, like boxes, your cart, even you! It will wait for you to pass and then keep scanning the shelves when all is clear.

Here to help

Our robot takes care of the tedious shelf-auditing tasks, allowing employees to focus on more important things like… you! It also provides vital information about different departments, making your store run smoother and more efficiently. Associates say they can’t imagine life without it.

Shelf focused

The action is on the shelves side! To make sure you always find your favorite chips, our robot scans the shelves diligently and reports what is missing.  Its sole job is finding empty spaces on shelves and keeping those chips available for you.  .... 

They make in an interesting suggestion:

 ... Nice to meet you


Look out for your new neighborhood robot on your next visit to your local store. Feel free to snap a selfie and post it on Instagram. Don’t forget to copy us   @BNRobotics  .... " 

Video Example:   https://video.twimg.com/tweet_video/DNJjrCbUEAE2aPb.mp4

See the comparison to solutions like: Storeflix.  Simpler, but less automated.

Wal-Mart Robotics and Labor

 Since Wal-Mart is the largest private employer this is of considerable interest.   How will this use of robotics ultimately influence retail and other industries? Volume of labor versus quality?    Ultimately labor is a big cost that is continually examined.   Digital Trends takes a look:

Walmart expands fleet of robots to new stores, but promises it won’t cut jobs  By Eric Brackett  in Digital Trends. 

Decades after they revolutionized the manufacturing industry, robots are now making their way to the service industry. PCMag has reported that Walmart is planning on using robots to automate certain mundane tasks.

The world’s largest retailer said it hopes to use automation to “handle tasks that are repeatable, predictable and manual, like scanning shelves for out-of-stock items, incorrect prices, and wrong or missing labels.” The robots have already been deployed in multiple stores in Arkansas, Pennsylvania, and California. Now, the company is preparing to expand the project to 50 additional stores.

The company says that this technology will free up the retailer’s employees to “focus on what they tell us are the most important and exciting parts of working at Walmart — serving customers and selling merchandise.”  .... " 

Scan and Go Again

Scan and Go Again  In Retailwire.

by Matthew Stern with expert comments.

Some ideas were ahead of their time and that’s how Walmart appears to see its Scan & Go mobile app. The retailer is bringing the discontinued mobile checkout app back into the pilot phase.

Walmart began a pilot run of the reintroduced app in a single store in Rogers, Arkansas, according to Tech Times. The app, released to the Google Play store on January 9, enables shoppers to scan the barcodes of items with their smartphones as they put them in their baskets. They then check out via mobile shopping cart and need only present their receipt to leave the store. The app also keeps a running tally of the cost of groceries, so shoppers can track their spending throughout their shopping trips.

The first run of Walmart’s Scan & Go app began in September of 2012 and was eventually expanded to a few hundred stores. Walmart shut down the pilot in 2014 because customers in the test locations could not figure out how to use it.  .... " 

AI That Turns a Body into a Digital Platform

Quite fascinating approach related in engineering.com, with multiple and unexpected uses.

Meet the AI That Turns a Body into a Digital Platform
Michael Molitch-Hou posted on October 14, 2016  

However achingly slow, the bright technological future painted by science fiction is beginning to emerge. Autonomous cars have begun to hit the roads, and friendly artificial intelligence (AI) could represent humanity's first contact with alien life. Not too long from now, we may be able to have our clothes or even prosthetics customfitted through the use of 3D scanning and printing.

One company working to make this last premise a reality is Manhattan-based Body Labs, one of the few firms developing the technology for digitally and accurately representing the human form. Whether it be for designing personally-tailored clothing or realistic virtual reality avatars, Body Labs uses AI and machine learning to “collect, digitize and organize all of the data and information related to human body shape, pose and motion.”

The firm's most recent announcement is for a product called “Red,” which integrates 3D scanning hardware with Body Labs' AI software to optimize the technology for such applications as custom retail, medicine, gaming and animation, and research and development.

The possibilities of the human body as a digital platform are fascinating, but just what that concept entails could use some elucidation. For that reason, ENGINEERING.com spoke with Jon Cilley, director of marketing at Body Labs, to learn more.    .... " 

Another in Aisle Scanning Test

RetailWire contains a good piece on in aisle checkout, covering many of the impediments to its implementation. A long time retail tech subject here.   Tested it in many contexts:  by Tom Ryan:

A new Sam’s Club opening on July 21 in Columbia, SC will be one of the first Sam’s in the country to offer Scan & Go shopping.

The Scan & Go mobile app promises to make members’ checkout process faster and more convenient by allowing shoppers to scan UPC codes on items as they load them into their carts. When done shopping, customers pay directly on their phone from anywhere in the club and bypass the checkout line.

Unfortunately, Scan & Go-type technologies seem to be in continual trial mode despite the promoted advantages of walk-around budget monitoring and speedier checkout. .... "  

State of Self Checkout

In Progressive Grocer: It is a journey that has been progressing for more than 15 years.   Still controversial the article says. Some retailers have removed the systems.  Was an early look at how the average person interacted directly with a complex retail system.  Looked at a half dozen systems as they progressed. After that, and the emergence of smartphones,  came the concept of consumers scanning items in the aisle to speed the process.  Completely automating the checkout system, with the aid of embedded RFID chips, was also examined.  The work continues.

Fujitsu Data Enables LED Light

In CWorld: Fujitsu Develops method which will allow products to send data to smartphones.

" ... LED lamps lighting merchandise may soon shine invisible data that your smartphone can pick up.

Fujitsu Laboratories has developed a method to include ID data in LED light so that pointing a smartphone camera at an object it illuminates calls up linked information stored on a cloud server.

 The technology could be used to retrieve information about products on display and even trigger a mobile payments system for purchasing them, according to Fujitsu. The technology could be used by retailers as well as in museums and art galleries or trade shows.....  "

Kroger Scan, Bag and Go

Got a chance today to use the Kroger in-aisle checkout method at the Middletown,  Ohio Kroger today.   This was a concept we tested with many alternatives in the innovation center and in lab stores. Nicely done example here.

You pick up a small hand scanning device just inside the entrance.  A  sign billed it as being in test.  Only about half of maybe twenty devices were being used today.   This could be replaced with an App to simplify the process.  You scan your loyalty card barcode to identify yourself.   Then you are off and scanning.   Its easy to learn, tried the always problematic produce identification and weigh in, but as long as you followed the instructions it went fine. All barcode tagged items were identified first time.

You can add and remove items. Get total cost.  No way to upload a shopping list, which would be useful. Also no way I saw to see your whole basket at once.  A store item locator would be useful as well.  These additional features could be integrated into an App on the customers smartphone.  No obvious place to place your device when shopping,  so it takes up one hand.  Saw this solved with a handle-holster in some experiments. Did not check how this handled coupons.  No way to send any messages back to the shopper, an obvious next step.

It was a little unclear to find where to check out, but as expected this was being handled next to the other self checkout.  It works similarly to self checkout, payment is gathered by  a payment system or employee.  Any ID checks needed are signaled as well.  No wait at all occurred on a relatively busy Saturday afternoon. Overall a good experience that I would use again.

More Cheaply Done

MJ Perry describes an example where an individual has been much more efficient than the government.  Here scanning documents. Many more examples like this exist.  We have infrastructure and technology now that make it very possible.  So why not?