Chip Proccess and Classification

 Impressive speed for detecting and classifying images

Chip Processes, Classifies Nearly Two Billion Images per Second

Penn Engineering Today

Melissa Pappas, June 1, 2022

University of Pennsylvania (Penn) engineers have designed a 9.3-square-millimeter chip that can detect and classify images in less than a nanosecond. The chip directly processes light received from objects of interest using an optical deep neural network. "Our chip processes information through what we call 'computation-by-propagation,' meaning that unlike clock-based systems, computations occur as light propagates through the chip," explained Penn's Firooz Aflatouni. "We are also skipping the step of converting optical signals to electrical signals because our chip can read and process optical signals directly, and both of these changes make our chip a significantly faster technology." Penn's Farshid Ashtiani said direct processing of optical signals makes a large memory unit unnecessary. ... 

Stacking the Deck for Computer Security

 New idea: Safe Stack,  to analyze and classify security.  Useful at least to flag possible problems.

Stacking the Deck for Computer Security

Penn State News

WennersHerron Ashley J.,  June 17, 2022

An international team of researchers led by Pennsylvania State University (Penn State) has created a more reliable safeguard for data on the stack than a prior classification technique called Safe Stack. Penn State's Trent Jaeger said the DATAGUARD system "improves security through a more comprehensive and accurate safety analysis that proves a larger number of stack objects are safe from memory errors, while ensuring that no unsafe stack objects are mistakenly classified as safe." The system validates stack objects that are safe from spatial, type, and temporal memory errors, via static analysis and symbolic execution. Tests showed DATAGUARD spotted and removed 6.3% of objects wrongly labeled safe by the Safe Stack technique, and found 65% of objects labeled "unsafe" by Safe Stack actually were safe. .... '

FBI Classifies Ransomware

I mentioned the FBI recently, that given all all the resources we provide them, they should be able to 'solve', or at least make far less dangerous the curse of Ransomware.  I see that they now classified RW into about 100 types. See below.   Not sure that is very productive.   Often you can tell progress that people are making by looking at the kinds of classifications they produce.  So ...  ?

FBI Director Christopher Wray told the Wall Street Journal that the agency is investigating about 100 different types of ransomware, many of which trace back to actors in Russia.  in Reuters

In the interview published on Friday, Wray singled out Russia as harboring many of the known users of ransomware.

Each of the 100 different malicious software variants are responsible for multiple ransomware attacks in the United States, Wray told the newspaper.  ... "

Archaeological Classification via Deep Learning

 A kind of natural application.  Thought of it too watching some programs that described archaeological technique where experts had to be brought in for key finds identification.   Useful generalization.  I remember some examples of contamination classification on a packing line that could have been done similarly.

Archaeologists vs. Computers: Study Tests Who's Best at Sifting the Past

The New York Times, Heather Murphy, May 25, 2021

Computers can sort pottery shards into subtypes at least as accurately as human archaeologists, as demonstrated by Northern Arizona University researchers. The researchers pitted a deep learning neural network against four expert archaeologists in classifying thousands of images of Tusayan White Ware pottery among nine known types; the networks outperformed two experts and equaled the other two. The network also sifted through all 3,000 photos in minutes, while each expert's analysis took three to four months. The network also could more specifically communicate its reasoning for certain categorizations than its human counterparts, and offered a single answer for each classification.

Monitoring in-production ML models

Useful and detailed look at real world problems with Sagemaker Model Monitor.  Good graphical views.   Somewhat but practically technical.

Monitoring in-production ML models at large scale using Amazon SageMakerModel Monitor  | AWS Machine Learning Blog

by Sireesha Muppala, Archana Padmasenan, and David Nigenda | on 17 DEC 2020 | in Amazon SageMaker, Artificial Intelligence  

Machine learning (ML) models are impacting business decisions of organizations around the globe, from retail and financial services to autonomous vehicles and space exploration. For these organizations, training and deploying ML models into production is only one step towards achieving business goals. Model performance may degrade over time for several reasons, such as changing consumer purchase patterns in the retail industry and changing economic conditions in the financial industry. Degrading model quality has a negative impact on business outcomes. To proactively address this problem, monitoring the performance of a deployed model is a critical process. Continuous monitoring of production models allows you to identify the right time and frequency to retrain and update the model. Although retraining too frequently can be too expensive, not retraining enough could result in less-than-optimal predictions from your model.

Amazon SageMaker is a fully managed service that enables developers and data scientists to quickly and easily build, train, and deploy ML models at any scale. After you train an ML model, you can deploy it on SageMaker endpoints that are fully managed and can serve inferences in real time with low latency. After you deploy your model, you can use Amazon SageMaker Model Monitor to continuously monitor the quality of your ML model in real time. You can also configure alerts to notify and trigger actions if any drift in model performance is observed. Early and proactive detection of these deviations enables you to take corrective actions, such as collecting new ground truth training data, retraining models, and auditing upstream systems, without having to manually monitor models or build additional tooling.

In this post, we discuss monitoring the quality of a classification model through classification metrics like accuracy, precision, and more.  ... " 

Classifying Quantum Sources

An interesting kind of classification.  Perhaps  determining it origins.   And as suggested, a faster way to do the classification.  This is another example at finding patterns.  Patterns are a form of information than can be useful to leverage the type or origins of signals.

ML-Assisted Method Rapidly Classifies Quantum Sources
Purdue University School of Electrical and Computer Engineering
September 10, 2020

Purdue University engineers have invented a machine learning-assisted technique for rapid selection of solid-state quantum emitters, which could enhance the efficiency of quantum photonic circuit development. Quantum emitters generate light with non-classical characteristics, but interfacing most solid-state emitters with scalable photonic platforms requires complex integration. The Purdue researchers trained a computer to recognize promising patterns in single-photon emission within a split second, to accelerate single-photon purity-based screening. Purdue's Zhaxylyk Kudyshev said the new technique could “speed up super-resolution microscopy methods built on higher-order correlation measurements that are currently limited by long image acquisition times.”... ' 

AI Decoding Emotions?

A problem we spent much time on, depends on what you expect from such a classification.    Much like: could you hire a human 'expert' to do this?   Training and background?  Implications and risk of error?    What sensors are being used?  At what required level of accuracy and for what prescribed need?  Here a reasonable overview, but does not address the broader need.

How close is AI to decoding our emotions? Emotion AI is becoming a big business. We talked to leading researchers about how good the tech actually is.  In TechnologyReview

 years trying to crack the mystery of how we express our feelings. Pioneers in the field of emotion detection will tell you the problem is far from solved. But that hasn’t stopped a growing number of companies from claiming their algorithms have cracked the puzzle. In part one of a two-part series on emotion AI, Jennifer Strong and the team at MIT Technology Review explore what emotion AI is, where it is, and what it means.  ... " 

Classifying Images with Quantum

Sounds remarkable and instructive about how quantum can be used for some kinds of machine learning approaches.

Google researchers use quantum computing to help improve image classification  in VentureBeat by Kyle Wiggers

In a new tutorial, Google researchers demonstrate how quantum computing techniques can be used to classify 28-pixel-by-28-pixel images illuminated by a single photon. By transforming the quantum state of the said photon, they show they’re able to achieve “at least” 41.27% accuracy on the popular MINST corpus of handwritten digits — a 21.27% improvement over classical computing approaches. ... "

Technical paper from Google

Classification of Relational Data

A simplification by classification.   Would this work in any context?   Application to social networks is of interest.

Training Agents to Walk with Purpose   By KAUST Discovery

The new classification algorithm that can dramatically simplify relational data.
Researchers at King Abdullah University of Science and Technology (KAUST) in Saudi Arabia and NortoLifeLock Research Group in France have developed a new classification algorithm for relational data.

Researchers at King Abdullah University of Science and Technology (KAUST) in Saudi Arabia and NortoLifeLock Research Group in France have developed a classification algorithm for relational data that is more accurate and orders of magnitude more efficient than previous methods.

The new algorithm represents a more robust approach to classifying relational data by introducing machine learning techniques.

Classifying relational data involves a search agent taking an exploratory "walk" following the connections among nodes.

The algorithm is a graph-based classification model that trains the agent using a reinforcement learning method, which achieves a better classification result.

The new method "is also generally applicable to any kind of graph-structured data, such as social-network recommendation systems and classification of biomolecules, as well as cybersecurity," says NortonLifeLock researcher Han Yufei.

From KAUST Discovery
View Full Article

Paper:

https://discovery.kaust.edu.sa/en/article/959/training-agents-to-walk-with-purpose%E2%80%8B
Akujuobi, U., Zhang, Q., Yufei, H. & Zhang, X. (2020). Recurrent attention walk for semi-supervised classification. Proceedings of the 13th International Conference on Web Search and Data Mining Houston TX USA, January 2020, 16-24.| article

Amazon can now find Acoustic A capella Versions of Songs

This struck me as interesting,   have followed the way musical pieces are classified, combined and described online.   In fact have had much trouble with systems like Alexa and Google Home, and voice search within their systems .    Even a slight 'mistake' in the voice description, which would be easily understood by a human musician, often  stymies the search leading to more wasted attempts.   This combined with slight mispronunciation of terms adds to the complexity of search.

Amazon can now find a cappella and acoustic versions of songs   By Kyle Wiggers in Venturebeat

Just over a year after Alexa gained the ability to announce titles and artists before songs play, Amazon’s voice assistant today gained a range of new music-focused features powered by AI and machine learning. With any luck, they’ll make it easier to request specific versions of a song or album or ask for music by language.

Starting today in the U.S. in the Amazon Music app for iOS or Android and on Echo devices, Alexa users with an Amazon Music account can request a cappella, live, remastered, remix, lullaby, deluxe, acoustic, instrumental, compilation, or kids’ renditions of songs, artists, and genres. Saying commands like “Alexa, play the Con Calma remix” or “Alexa, play live J. Cole songs” kicks off the search for an alternative recording. Alternatively, while a song is playing, asking “Alexa, play the acoustic version of this” switches to the requested version.

Amazon says that Amazon Music customers with Echo devices in the U.S. will also now hear a “more natural” version of Alexa’s voice when she introduces music, including curated content like playlists and stations, or when music is requested by mood, genre, lyric, and more. Additionally, language-based requests on Amazon Music have expanded to support over 60 languages, including Vietnamese, Persian, Nigerian, Ukrainian, Romanian, Maori, Icelandic, and more, allowing listeners to combine an era or genre with their preferred language. And artists including The Weeknd, Justin Bieber, and Selena Gomez will announce their newest releases and hits on Amazon Music instead of Alexa for a limited time.

The enhancements follow the launch of Alexa’s Song ID, an opt-in feature that lets Echo smart speaker users ask Alexa to announce the title and artist name before each song plays. At the time, Amazon pitched it as a music, station, playlist, and chart discovery service for Alexa users, who it said ask Alexa devices “hundreds of thousands” of questions per day to find out more about the music they’re hearing. ... "

Classification with Naive Bayes

Good piece, behind a paywall but worth a look.  Technical.

Comparing a variety of Naive Bayes classification algorithms
Comprehensive list of formulas for text classification
Pavel Horbonos (Midvel Corp)

Naive Bayes algorithm is one of the well-known supervised classification algorithms. It bases on the Bayes theorem, it is very fast and good enough for text classification. I believe that there is no need to describe the theory behind it, nevertheless, we will cover a few concepts and after that focus on the comparing of different implementations.  .... "

Geometrically: What Should You Read Next?

Classifying and suggesting documents that point to our goals.  An old problem, but now being done with new tools.   'Geometric Data Processing'  an interesting term,  see more on that here.  "Our group studies geometric problems in computer graphics, computer vision, machine learning, optimization, and other disciplines" 

Finding a good read among billions of choices
As natural language processing techniques improve, suggestions are getting speedier and more relevant.

Kim Martineau | MIT Quest for Intelligence
With billions of books, news stories, and documents online, there’s never been a better time to be reading — if you have time to sift through all the options. “There’s a ton of text on the internet,” says Justin Solomon, an assistant professor at MIT. “Anything to help cut through all that material is extremely useful.”

With the MIT-IBM Watson AI Lab and his Geometric Data Processing Group at MIT, Solomon recently presented a new technique for cutting through massive amounts of text at the Conference on Neural Information Processing Systems (NeurIPS). Their method combines three popular text-analysis tools — topic modeling, word embeddings, and optimal transport — to deliver better, faster results than competing methods on a popular benchmark for classifying documents.

If an algorithm knows what you liked in the past, it can scan the millions of possibilities for something similar. As natural language processing techniques improve, those “you might also like” suggestions are getting speedier and more relevant.   .... " 

Customers Calling

Another example of determining the classification 'why' of behavior:

Why are your customers calling you again?  in Mckinsey.

Getting to the bottom of why customers keep calling your contact centers can generate significant savings. And result in happier customers.  .... "

Predicting Molecule Smell

This sounds a bit odd, but we were trying to exactly this some time ago, when were still in the coffee business.  Smell, taste, and other sensory aspects of products, in both their blended and growing characteristics.  And of course the fragrance industry might also be interested. 

Google is training an AI to predict a molecule’s smell
by Ivan Mehta in ThenextWeb

With plenty of mics and cameras at disposal, AI has gotten good at ‘seeing’ and ‘listening.’ But one human sense it hasn’t got around much is smell. Now, researchers at Google are trying to develop a neural network that helps an AI identify the smell characteristics of a molecule.

The company said identifying smell is a multi-label classification problem, meaning a substance can have multiple smell characteristics. For instance, Vanillin, a substance often used to create an artificial vanilla flavor, has multiple smell descriptors such as sweet, vanilla, and chocolate, with some characteristics stronger than others.   ....'

Attention for Advanced Forecasting and Classification

Interesting and quite technical view of forecasting and classification that is worth a look.  Of course accurate and timely forecasting is important for most businesses.  Considerable piece here, below an intro with much more at the link.  Have never seen it done accurately enough with these kinds of methods.

Attention for time series forecasting and classification
Harnessing the most recent advances in NLP for time series forecasting and classification  By Isaac Godfried

Transformers (specifically self-attention) have powered significant recent progress in NLP. They have enabled models like BERT, GPT-2, and XLNet to form powerful language models that can be used to generate text, translate text, answer questions, classify documents, summarize text, and much more. With their recent success in NLP one would expect widespread adaptation to problems like time series forecasting and classification. After all, both involve processing sequential data. However, to this point research on their adaptation to time series problems has remained limited. Moreover, while some results are promising, others remain more mixed. In this article, I will review current literature on applying transformers as well as attention more broadly to time series problems, discuss the current barriers/limitations, and brainstorm possible solutions to (hopefully) enable these models to achieve the same level success as in NLP. This article will assume that you have a basic understanding of soft-attention, self-attention, and transformer architecture. If you don’t please read one of the linked articles. You can also watch my video from the PyData Orono presentation night.

Attention for time series data: Review

The need to accurately forecast and classify time series data spans across just about every industry and long predates machine learning. For instance, in hospitals you may want to triage patients with the highest mortality early-on and forecast patient length of stay; in retail you may want to predict demand and forecast sales; utility companies want to forecast power usage, etc. .... " 

Etsy Uses an Algorithm for Style

Interesting example, with forthcoming details.   Note its alliance to something most everyone does today, utilize e-commerce.  Look forward to seeing the details of the paper.

 How Etsy taught style to an algorithm  By Harry McCrackenin in FastCompany

From “romantic” to “rustic,” the marketplace for handcrafted goods that express distinctive aesthetics is teaching its search engine to know what’s what.

 ... After about a year of work, Fisher says, Etsy has trained a machine-learning model to effectively suss out the styles of items on the site, based on both textual and visual cues. The company is about to start testing results based on this new algorithm on the Etsy site. But it also believes that the technology it’s developed could have applications well beyond making e-commerce more relevant. Which is why three of Etsy’s data scientists have written a paper about it—coauthored with a Twitter employee—which they’ll present at the Association for Computing Machinery’s KDD (Knowledge Discovery and Data Mining) conference in August. .... " 

How Much Knowledge Has been Created?

We explored this early on with image tagging in the enterprise.   And while we have developed lots of specific usage cases, nothing as broadly usable as we wanted.

The data that trains AI increasingly calls into question AI
After 10 years of ImageNet, AI researchers are digging into the details of test sets and some are asking just how much knowledge has really been created with machine learning.
By Tiernan Ray in ZDNet

It's been 10 years since two landmark data sets appeared in the world of machine learning, ImageNet and CIFAR10, collections of pictures that have been used to train untold numbers of models of computer vision deep learning neural networks. The venerable nature of the data has prompted some AI researchers to ask what goes on with those data sets, and what their longevity means about machine learning in the bigger picture.

As a result, 2019 may mark the year the data indicted some of the fundamental beliefs about AI.

Researchers in machine learning are getting much more specific and rigorous about understanding how the choice of data affects the success of neural networks.

And the results are somewhat unsettling. Recent work suggests at least some of the success of neural networks, including state-of-the-art deep learning models, is tied to small, idiosyncratic elements of the data used to train those networks.

Exhibit A is a study put out in February and revised in June by Benjamin Recht and colleagues at UC Berkeley, with the amusing title "Do ImageNet Classifiers Generalize to ImageNet?"

They tried to reconstruct ImageNet, in a sense, by duplicating the process of gathering images from Flickr and curating them, having people on Amazon's Mechanical Turk service look at the images and assign labels.   

The original screen from back in 2009 instructing Amazon Mechanical Turk workers to pick images that fit with labels. It kicked off a decade of development of more and more advanced computer vision neural networks.

The goal was to create a new "test" set of images, a set that's like the original group of pictures, but never seen before, to see how well all the models that have been developed on ImageNet in the past decade generalize to new data.

The results were mixed. The various deep learning image recognition models that followed one another in time, such as the classic "AlexNet" and, later, more-sophisticated networks such as "VGG" and "Inception," still showed improvement from generation to generation. In fact, on this new test set, levels of improvement were actually amplified.  .... " 

Dilemma of Scraping Facial Data

Solving deep learning problems requires considerable and varied data.   I was recently involved in such a problem which aimed to do facial demographics, learning and then do real-time classification.  It was quickly determined that this data was hard to get in sufficient volume.  Though it was also determined that services like Facebook had lots of it. Technically, scraping it online is easy.  But  this, if used, would be without direct consent.    The company involved rejected that  approach, and looked for other places to get data. 

IBM’s photo-scraping scandal shows what a weird bubble AI researchers live in

On Tuesday, NBC published a story with a gripping headline: “Facial recognition’s ‘dirty little secret’: Millions of online photos scraped without consent.” I linked to it in our last Algorithm issue, but it’s worth a revisit today.

The story highlights a recent data set released by IBM with 1 million pictures of faces, intended to help develop fairer face recognition algorithms. (I wrote about the news at the time too.) It turns out, NBC found, that those faces were scraped directly from the online photo-hosting site Flickr, without the permission of the subjects or photographers. .... "

Beyond WorstCase Analysis

A look at the worst case in performance.  This deals more with optimization methods, those that can find specific correct examples.    So for example sorting a file is given as a common problem.    There is a precise correct result that is the sorted file.  Now how long does it take to sort very large files, with many sort parameters?   This is different for deep learning methods, where the result depends on some chosen solution architecture, and is not expected to be optimally and provably best.   (But as the paper suggests, often is!)  A number of interesting problems often addressed, like classifiers, are mentioned.   But I don't often expect a classifier to produce optimal answers.

 Video intro by the author.

Paper is technical.

Beyond Worst-Case Analysis By Tim Roughgarden 

Communications of the ACM, March 2019, Vol. 62 No. 3, Pages 88-96   10.1145/3232535

Comparing different algorithms is hard. For almost any pair of algorithms and measure of algorithm performance like running time or solution quality, each algorithm will perform better than the other on some inputs.a For example, the insertion sort algorithm is faster than merge sort on already-sorted arrays but slower on many other inputs. When two algorithms have incomparable performance, how can we deem one of them "better than" the other? ....... '

The author also talks about current 'Deep learning':

" ... To illustrate some of the challenges, consider a canonical supervised learning problem, where a learning algorithm is given a dataset of object-label pairs and the goal is to produce a classifier that accurately predicts the label of as-yet-unseen objects (for example, whether or not an image contains a cat). Over the past decade, aided by massive datasets and computational power, deep neural networks have achieved impressive levels of performance across a range of prediction tasks.25 Their empirical success flies in the face of conventional wisdom in multiple ways. First, most neural network training algorithms use first-order methods (that is, variants of gradient descent) to solve nonconvex optimization problems that had been written off as computationally intractable. Why do these algorithms so often converge quickly to a local optimum, or even to a global optimum?q Second, modern neural networks are typically over-parameterized, meaning that the number of free parameters (weights and biases) is considerably larger than the size of the training dataset. Over-parameterized models are vulnerable to large generalization error (that is, overfitting), but state-of-the-art neural networks generalize shockingly well.40 How can we explain this? The answer likely hinges on special properties of both real-world datasets and the optimization algorithms used for neural network training (principally stochastic gradient descent)  .... 

 ... With algorithms increasingly dominating our world, the need to understand when and why they work has never been greater. The field of beyond worst-case analysis has already produced several striking results, but there remain many unexplained gaps between the theoretical and empirical performance of widely used algorithms. With so many opportunities for consequential research, I suspect the best work in the area is yet to come. .... " 

Further thinking the implications of this.   But it does make us think about how such algorithms should be used, and their inherent risk.

Facial Recognition for Pharma Convenience

We tested the idea of putting up different marketing messages depending on observed age/gender/demographic/weather , etc .  Which appears to be similar to this Walgreens test.    So a person is not identified, but rather their behavioral and contextual category is inferred.   Was found to have some leverage regarding improved messaging, but the implication of manipulation was quickly understood by consumers. Iris tracking takes this yet a step deeper.    Trust and privacy are brought up here as a worry.

Walgreens tests tech that sort of recognizes you in-store   by Tom Ryan in Retailwire with further expert opinion

Walgreens is piloting a line of “smart coolers” with the ability to display targeted ads to in-store shoppers.

Instead of seeing through the glass to drinks, ice cream and other items, shoppers view digitized representations of available products inside, or basically a planogram on the front of refrigerator and freezer doors.

The system involves using sensors that detect shoppers and cameras that scan their faces to estimate their gender and approximate age for delivering targeted messages. Weather, time of day and other events may also influence messaging.

Beyond messaging, iris-tracking technology can show which items are picked up or are looked at, providing insights into the effectiveness on-screen promotions and the overall display.

The digital screens on the coolers further give retailers the ability to make real-time changes to pricing and promotions. Alerts to out of stocks is touted as another benefit.  ... "