Single Photon Cameras to Peer into your Brain?

At Last, Single-Photon Cameras Could Peer Into Your Brain The tech has long been stymied on how to scale it out of the lab    By Dina Genkina  in Spectrum IEEE

Superconductor-based cameras that can detect a single photon—the smallest smidgeon of light—have existed for 20 years, but they’ve remained confined to laboratories due to the inability to scale them past a few pixels. Now, a team at the National Institute of Standards and Technology (NIST) in Boulder, Colo., has created a 0.4-megapixel single-photon camera—400 times as large as the previous biggest camera of its type. They reported their results in a preprint they submitted to arXiv on 15 June.

Single-photon cameras, made of superconducting nanowires, measure light with unrivaled sensitivity and speed, and across an unmatched frequency range. With the leap in size, the single-photon camera is poised to transition from a lab curiosity to an industrial technology. Such cameras could find a home imaging the cosmos on the next James Webb–type telescope, measuring light in photonic quantum computers and communications, and peering into the brain with noninvasive light-based techniques.

“From a scientific perspective, this is definitely opening a new avenue in optical brain imaging,” says Stefan Carp, an associate professor of radiology at the Harvard Medical School who was not involved in the work. “Other approaches for optically mapping cortical brain flow may have lower costs, but they all have shortcomings impacting signal quality that often require complex signal processing. There is no compromise with nanowires from a performance perspective.”.... 

Wearable Sensor Provides Cardiac Imaging on the Go

I have a similar implanted device, but without the imaging and other smarts detailed here.  

Wearable Sensor Provides Cardiac Imaging on the Go

By UC San Diego Today, February 3, 2023

The unique design of the sensor makes it ideal for bodies in motion.

The wearable uses ultrasound to continuously capture images of the four chambers of the heart at different angles, and analyze a clinically relevant subset of the images in real time using custom-built AI technology.

A team led by researchers at the University of California, San Diego (UC San Diego) has developed a portable ultrasound device that uses custom algorithms to measure how much blood is being pumped by the heart.

The postage-stamp-sized patch captures images of the heart at different angles using ultrasound.

The technology performs real-time analysis of a subset of the images, which have high spatial resolution, temporal resolution, and contrast.

UC San Diego's Ruixiang Qi said the device provides "accurate and continuous waveforms of key cardiac indices in different physical states, including static and after exercise, which has never been achieved before."

From UC San Diego Today   View Full Article  

2D Material May Enable Ultra-Sharp Cellphone Photos in Low Light

Most interesting, worked on a project that needed to get low light images for input to modeling.

2D Material May Enable Ultra-Sharp Cellphone Photos in Low Light

Pennsylvania State University News,   Jamie Oberdick, December 9, 2022

Researchers at Pennsylvania State University (Penn State) have developed a pixel sensor using a novel two-dimensional material that could allow mobile phones to take ultra-sharp phones with low-energy in-sensor image processing. The sensor uses a molybdenum disulfide semiconductor featuring strong signal conversion, charge-to-voltage conversion, and data transmission capabilities. The researchers arranged the sensors into a nine-square-millimeter array comprised of 900 pixels, each of which take up around 100 micrometers. Penn State's Darsith Jayachandran said, "They are much more sensitive to light than current CMOS sensors, so they do not require any additional circuitry or energy use. So, each pixel requires much less energy to operate, and this would mean a better cellphone camera that uses a lot less battery." ... ' 

Ultrasound Stickers Could Continuously Image Internal Organs for Days

Remarkable scanning idea.

Ultrasound Stickers Could Continuously Image Internal Organs for Days

New Scientist, Jeremy Hsu, July 28, 2022

Xuanhe Zhao and colleagues at the Massachusetts Institute of Technology have developed a wearable ultrasound sticker that can image the wearer's internal organs continuously for 48 hours. The researchers integrated a transducer that generates and detects ultrasound waves with a patch combining hydrogel to transmit the waves, and flexible elastomer material. The researchers stuck the stickers to volunteers who performed various actions, and were able to image changes in the size and shape of their organs during those activities. The stickers currently have to be connected by wires to a computer, which translates the ultrasound waves into images and collects the data.... ' 

Using Light, Sound to Reveal Rapid Brain Activity in Detail

Faster detail in photo microscopy.  Note also use of sound.  

Using Light, Sound to Reveal Rapid Brain Activity in Unprecedented Detail   By Duke Biomedical Engineering, May 26, 2022

Duke University researchers have developed an approach that allows for real-time scanning and imaging of blood flow and oxygen levels in a mouse brain with sufficient resolution to simultaneously observe activity of individual vessels and the entire brain.

Their method could help overcome the challenges associated with brain imaging, particularly the trade-off between speed and resolution.

The new approach, called ultrafast photoacoustic microscopy (UFF-PAM), involves a combination of a polygon scanning system that sends more laser bursts to a larger area, a new scanning mechanism that allows simultaneous operation of the laser scanner and ultrasound sensor, and machine learning algorithms to improve the resolution of the images.

Said Duke's Junjie Yao, "The resulting images looked as detailed as the high-resolution images we would usually get if we went at a much slower speed, and we didn't need to sacrifice a full field of view."   ... 

Photoacoustic microscopy uses light and sound to capture detailed images of organs, tissues, and cells throughout the body. The technique uses a laser to send light into a targeted tissue or cell; when the laser hits the cell, it heats up and expands inst ... '

From Duke Biomedical Engineering

View Full Article  

Ghost Imaging

Complexity of advanced imaging of details.

Ghost Imaging Speeds Up X-Ray Fluorescence Chemical Mapping

SciTechDaily, January 13, 2022

Sharon Shwartz and colleagues at Israel's Bar Ilan University integrated computational ghost imaging and x-ray fluorescence measurement to generate high-resolution chemical element maps. The focus-free technique reduces the need for scanning and saves measurement time, and it can be tuned to detect specific elements without seeing human tissues. The method produces two datasets for each photon energy—one with the input beam's spatial distributions and one with emitted fluorescence measurements—which an algorithm then maps out. The researchers used a compressive sensing algorithm to cut the number of scans required to chemically map an iron-cobalt object by almost 10-fold versus standard scanning-based techniques. "We expect it will allow the chemical mapping of larger objects at higher resolutions than is possible today while also enabling measurement of complex 3D [three-dimensional] objects," Shwartz said.  ... ' 

Could a Tree Signal if A Corpse is Decaying?

 We worked with satellite and other kinds of imaging to determine the growth and health of forests, as we planned their harvest and replanting for CPG products.   So in theory as mentioned below,  this is possible,  but also depends on lots of other contextual metadata.    So intriguing, but hardly seems a way to broadly hunt for bodies.    Like to see the data in real environments.

Could a Tree SIgnal if a Corpse is Decaying?

By Matt Simon in Wired

SINCE 1980, THE University of Tennessee’s Forensic Anthropology Center has plumbed the depths of the most macabre of sciences: the decomposition of human bodies. Known colloquially as the Body Farm, here scientists examine how donated cadavers decay, like how the microbiomes inside us go haywire after death. That microbial activity leads to bloat, and—eventually—a body will puncture. Out flows a rank fluid of nutrients, especially nitrogen, for plants on the Body Farm to subsume.  ... " 

Seeing Through Walls

Had seen this before, fascinating example of how more and richer signals can be leveraged into new kinds of imaging.  Here a kind of fabulous example, seeing through walls.    But in the right circumstances, with the right data.

Seeing Through Walls     by Neil Savage
Communications of the ACM, June 2020, Vol. 63 No. 6, Pages 15-16   10.1145/3392516

Machine vision coupled with artificial intelligence (AI) has made great strides toward letting computers understand images. Thanks to deep learning, which processes information in a way analogous to the human brain, machine vision is doing everything from keeping self-driving cars on the right track to improving cancer diagnosis by examining biopsy slides or x-ray images. Now some researchers are going beyond what the human eye or a camera lens can see, using machine learning to watch what people are doing on the other side of a wall.

The technique relies on low-power radio frequency (RF) signals, which reflect off living tissue and metal but pass easily through wooden or plaster interior walls. AI can decipher those signals, not only to detect the presence of people, but also to see how they are moving, and even to predict the activity they are engaged in, from talking on a phone to brushing their teeth. With RF signals, "they can see in the dark. They can see through walls or furniture," says Tianhong Li, a Ph.D. student in the Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology (MIT). He and fellow graduate student Lijie Fan helped develop a system to measure movement and, from that, to identify specific actions. "Our goal is to understand what people are doing," Li says.

Such an understanding could come in handy for, say, monitoring elderly residents of assisted living facilities to see if they are having difficulty performing the tasks of daily living, or to detect if they have fallen. It could also be used to create "smart environments," in which automated devices turn on lights or heat in a home or an office. A police force might use such a system to monitor the activity of a suspected terrorist or an armed robber.

In living situations, one advantage of monitoring activity through walls with RF signals is that they are unable to resolve faces or see what a person is wearing, for instance, so they could afford more of a sense of privacy than studding the home with cameras, says Dina Katabi, the MIT professor leading the research. Another is that it does not require people to wear monitoring devices they might forget or be uncomfortable with; they just move through their homes as they normally would.

The MIT system uses a radio transmitter operating at between 5.4 GHz and 7.2GHz, at power levels 1,000 times lower than a Wi-Fi signal, so it should not cause interference. The RF transmissions bounce strongly off people because of all the water content of our bodies, but they also bounce off other objects in the environment to varying degrees, depending on the composition of the object. "You get this mass of reflections, signals bouncing off everything," Katabi says.

So the first step is to teach the computer to identify which signals are coming from people. The team does this by recording a scene in both visible light and RF signals, and using the visual image to label the humans in the training data for a convolutional neural network (CNN), a type of deep learning algorithm that assigns weights to different aspects of an image. The signals also contain spatial information, because it takes a longer time for a signal to travel a longer distance. The CNN can capture that information and use it to separate two or more people in the same vicinity, although it can lead to errors if the people are very close or hugging, Katabi says.  .... " 

AI Improving Biomedical Imaging

It is notable how modern AI is doing best in 'vision' spaces.    As opposed to what I would call conversational interaction and process logic.   Not what we would have expected in the earlier applications of AI.  Is this because the training data is more available and concise, or because the underlying deep learning models are closer to the underlying human intelligence?   Or both?

Artificial intelligence improves biomedical imaging in TechXplore
by Fabio Bergamin, ETH Zurich

ETH researchers use artificial intelligence to improve quality of images recorded by a relatively new biomedical imaging method. This paves the way towards more accurate diagnosis and cost-effective devices.

Scientists at ETH Zurich and the University of Zurich have used machine learning methods to improve optoacoustic imaging. This relatively young medical imaging technique can be used for applications such as visualizing blood vessels, studying brain activity, characterizing skin lesions and diagnosing breast cancer. However, quality of the rendered images is very dependent on the number and distribution of sensors used by the device: the more of them, the better the image quality. The new approach developed by the ETH researchers allows for substantial reduction of the number of sensors without giving up on the resulting image quality. This makes it possible to reduce the device cost, increase imaging speed or improve diagnosis. .... "

High Quality Images from Limited data

Always looking for more information from less data.  You may just not have enough data, or it may take more effort, time, cost or exposure to get it. 

New artificial intelligence technique dramatically improves the quality of medical imaging

Source:   Massachusetts General Hospital in ScienceDaily.

Researchers have developed a new technique based on artificial intelligence and machine learning that should enable clinicians to acquire high-quality images from limited data. ... "

Reconfigurable Camera Systems

Shree Nayar discusses "Cambits: A Reconfigurable Camera System" (cacm.acm.org/magazines/2017/11/222170), a Contributed Article in the November 2017 CACM.

 .... Cambit pieces can be assembled to create a dozen different imaging systems. To celebrate this assortment, Communications has published four different covers, each one featuring a different Cambit configuration. ... 

" ... Here, we present Cambits, a set of physical blocks that can be used to build a variety of cameras with different functionalities. Blocks include sensors, actuators, lenses, optical attachments, and light sources, assembled with magnets without screws or cables. When two blocks are attached, they are connected electrically through spring-loaded pins that carry power, data, and control signals. The host computer always knows the current configuration and automatically provides a menu of imaging functionalities from which the user can choose. Cambits is a scalable system, allowing users to add new blocks and computational photography algorithms to the current set. ... " 

Demonstration video:   https://vimeo.com/236437712

Who Does Selfies?

Or perhaps the nature of derived meta data about self imagery.

Selfies: We Love How We Look and We're Here to Show You

Researchers at the Georgia Institute of Technology say they have analyzed 2.5 million selfie posts on Instagram to determine what kinds of identity statements people make by taking and sharing selfies. The researchers found nearly 52 percent of all selfies fall into the appearance category, meaning pictures of people showing off things such as their makeup, clothes, and lips. In addition, the team found that pictures about looks were twice as popular than the other 14 examined categories combined, which included social selfies, ethnicity pictures, travel, and health and fitness.

The researchers also note the prevalence of ethnicity selfies as an indication that people are proud of their backgrounds. Overall, 57 percent of selfies were posted by 18- to 35-year-olds, while the under-18 age group posted about 30 percent of selfies, and the 35-and-up group posted only 13 percent of selfies. In addition, most selfies are lone pictures, and are not taken with a group.

From Georgia Tech News Center  .... "

What Should Kodak have Done?

Fascinating piece in Innovation Excellence.  In particular because we met with Kodak to talk collaboration in innovation at the time of their long slide.   Kodak had some excellent intellectual property that could have been used for reinvention.   Article could have used some more details.

Photoshop Fix for IOS

Reported in Verge.  Most of the commonly used capabilities in Photoshop are now free with Photoshop Fix.    I had a corporate account for years, but when leaving the enterprise I dropped it, other packages could do what I needed.   Now Adobe Photoshop has changed their model with a more freemium view.  Photoship Fix released for IOS, for now at least

Vertical Focus in Big Data Applications

As solution technologies get closer to business process, it makes sense to increase focus on the direct analytical and cognitive needs of verticals involved.   Silicon Angle looks at this past week, where IBM bought a medical imaging company.  Working with health data will mean working with medical imaging and extracting and leveraging their embedded knowledge.   " ... IBM hopes that enabling Watson to examine medical issues from multiple angles will make it possible to automate much of the tedious administrative currently involved in piecing together patient information and free up healthcare professionals to focus on more pressing matters.  ... " 

On Pluto and the Nature of Seeing

What does it mean to see in a technological world?  A Wired look at visual sensors, using the Pluto flyby as an example.  At what, when and how are we seeing things?

Shopping Through the Lens of IT

In the ACM    " ... Pennsylvania State University (PSU) researchers are studying how smart cameras could eventually guide visually impaired shoppers to find the items they need.

The researchers are using a $10-million U.S. National Science Foundation grant to replicate the human vision system using information technology as part of the Visual Cortex on Silicon project. "This project brings together the strengths and efforts of technical leaders in multiple disciplines," says PSU professor Vijaykrishnan Narayanan. ... " 

Protein Imaging Data

In Nature:  Heard much about this topic in R&D when we were still in health care product development.  " ... Data bank struggles as protein imaging ups its game Hybrid methods to solve structures of molecular machines create a storage headache. ... ".    Its about the idea of curating structure.  An idea that implies human intervention, but likely with strong cognitive augmentation.

Moving Portraits

In CACM:  Augmentation to storytelling?   A Vimeo demonstration.

We present an approach for generating face animations from large image collections of the same person. Such collections, which we call photobios, are remarkable in that they summarize a person's life in photos; the photos sample the appearance of a person over changes in age, pose, facial expression, hairstyle, and other variations. Yet, browsing and exploring photobios is infeasible due to their large volume. By optimizing the quantity and order in which photos are displayed and cross dissolving between them, we can render smooth transitions between face pose (e.g., from frowning to smiling), and create moving portraits from collections of still photos. ... " 

Seeing the Invisible

Seeing through walls with Wifi.   With concern, " .... Wi-Fi makes all kinds of things possible. We can send and receive messages, make phone calls, browse the Internet, even play games with people who are miles away, all without the cords and wires to tie us down. At UC Santa Barbara, researchers are now using this versatile, everyday signal to do something different and powerful: looking through solid walls and seeing every square inch of what’s on the other side. Built into robots, the technology has far-reaching possibilities.  ... "