Single Photon Cameras to Peer into your Brain?

At Last, Single-Photon Cameras Could Peer Into Your Brain The tech has long been stymied on how to scale it out of the lab    By Dina Genkina  in Spectrum IEEE

Superconductor-based cameras that can detect a single photon—the smallest smidgeon of light—have existed for 20 years, but they’ve remained confined to laboratories due to the inability to scale them past a few pixels. Now, a team at the National Institute of Standards and Technology (NIST) in Boulder, Colo., has created a 0.4-megapixel single-photon camera—400 times as large as the previous biggest camera of its type. They reported their results in a preprint they submitted to arXiv on 15 June.

Single-photon cameras, made of superconducting nanowires, measure light with unrivaled sensitivity and speed, and across an unmatched frequency range. With the leap in size, the single-photon camera is poised to transition from a lab curiosity to an industrial technology. Such cameras could find a home imaging the cosmos on the next James Webb–type telescope, measuring light in photonic quantum computers and communications, and peering into the brain with noninvasive light-based techniques.

“From a scientific perspective, this is definitely opening a new avenue in optical brain imaging,” says Stefan Carp, an associate professor of radiology at the Harvard Medical School who was not involved in the work. “Other approaches for optically mapping cortical brain flow may have lower costs, but they all have shortcomings impacting signal quality that often require complex signal processing. There is no compromise with nanowires from a performance perspective.”.... 

Algorithm Uses Phone Camera, AI to Detect Blood Oxygen Levels

 Ai driven algorithm captures blood oxygen levels on Smartphone.  Note 'Hyperspectral Learning'.

Algorithm Uses Phone Camera to Detect Blood Oxygen Levels

By Purdue University, June 9, 2023

A smartphone camera paired with an AI-driven algorithm can capture blood oxygen data faster and more efficiently than highly specialized equipment.

The researchers used a computational approach that they described as “hyperspectral learning.”

Purdue University researchers have developed an algorithm to improve the speed and accuracy of medical diagnoses using smartphone sensors.

Although smartphone cameras capture only red, green, and blue wavelengths of light in each pixel, the researchers were able to reconstruct the full spectrum of visible light in each pixel of an image taken by a smartphone camera using deep learning, statistical techniques, and an understanding of light-tissue interactions.

The researchers found their technology produced information about blood oxygen levels in study participants' eyelids faster than commercially available hyperspectral imaging equipment, while being less expensive and just as accurate.

From Purdue University

View Full Article   

Very Tiny Cameras, Consider the Possibilities

I like the realm of the very small,  for sensors and the ability to combine multiple tiny views.  Healthcare would for example could be an interesting application. 

Researchers shrink Camera to the Size of a Salt Grain  In Princeton  Engineering  By Molly Sharlach,  November 29, 2021

Researchers at Princeton University and the University of Washington have developed an ultracompact camera the size of a coarse grain of salt. The system relies on a technology called a metasurface, which is studded with 1.6 million cylindrical posts and can be produced much like a computer chip. Image courtesy of the researchers

Micro-sized cameras have great potential to spot problems in the human body and enable sensing for super-small robots, but past approaches captured fuzzy, distorted images with limited fields of view.

Now, researchers at Princeton University and the University of Washington have overcome these obstacles with an ultracompact camera the size of a coarse grain of salt. The new system can produce crisp, full-color images on par with a conventional compound camera lens 500,000 times larger in volume, the researchers reported in a paper published Nov. 29 in Nature Communications.

Enabled by a joint design of the camera’s hardware and computational processing, the system could enable minimally invasive endoscopy with medical robots to diagnose and treat diseases, and improve imaging for other robots with size and weight constraints. Arrays of thousands of such cameras could be used for full-scene sensing, turning surfaces into cameras.

While a traditional camera uses a series of curved glass or plastic lenses to bend light rays into focus, the new optical system relies on a technology called a metasurface, which can be produced much like a computer chip. Just half a millimeter wide, the metasurface is studded with 1.6 million cylindrical posts, each roughly the size of the human immunodeficiency virus (HIV).

Each post has a unique geometry, and functions like an optical antenna. Varying the design of each post is necessary to correctly shape the entire optical wavefront. With the help of machine learning-based algorithms, the posts’ interactions with light combine to produce the highest-quality images and widest field of view for a full-color metasurface camera developed to date.  ... ' 

Micro-Sized Camera will Turn Nanorobots into Photographers

 Quite an interesting thing here.  A new way to use sensors;

This Micro-Sized Camera will Turn Nanorobots into Photographers - News WWC  By Steven

Nanorobotics, like graphene, have been trending topics in research for years but are still far from full industrial production. Everyone is aware of its potential, but the technical hurdles remain. Fortunately, research is progressing steadily. The latest invention that would allow the world’s tiniest robots to take a giant leap forward is a camera barely the size of a grain of salt.

Imagine for a moment that, instead of using a bulky CAT scan or intrusive endoscopy, an almost invisible robot could inspect your arteries or the most inaccessible corners of your heart. Those could be some of the of applications enabled by a new camera designed by scientists at Princeton University in the U.S. It is the size of a grain of salt and works in a radically different way from traditional lenses.

An alliance between metasurfaces and neural networks

The inventors of this new device claim that the image quality is similar to a camera 500,000 times larger. To achieve this, they have had to reinvent the camera concept and resort to a combination of hardware and software based on neural networks. This is how they succeeded.

Firstly, their camera looks more like a microchip than a lens. In fact, the primary material is silicon nitride, used in the production of semiconductors. Thanks to it, instead of using a lens to bend the light rays, they have squeezed 1.6 million cylinders into a space of less than half a millimeter. Each of these cylinders has a different size to emulate the effect of a lens. Thus, they operate as optical antennas. In scientific jargon, this array is known as a “metasurface.”

What is a metasurface?

To understand what a metasurface is, we must first talk about metamaterials. These are structures designed at the micro- or nanoscale that interact with light and other types of energy in ways unknown in the natural world. One of these is negative refraction, a phenomenon that opens the door to such exotic results as invisibility or superlenses of unprecedented capacities, such as the Princeton University camera. Thus, metasurfaces are films that leverage the qualities of metamaterials.     

However, the physical part is only one element of the equation. The developers’ second achievement has been to create a design that processes signals through a neural network. This processing system achieves sharp images in natural light conditions.     .... ' 

Cameras Hacked

More events and threats to doing business emerging.  Note the links to key infrastructure providers. 

Hackers Breach Thousands of Security Cameras, Exposing Tesla, Jails, Hospitals  By Bloomberg   March 10, 2021

A group of hackers say they breached a massive trove of security-camera data collected by Silicon Valley startup Verkada Inc., gaining access to live feeds of 150,000 surveillance cameras inside hospitals, companies, police departments, prisons, and schools.

Hackers say they have compromised data from as many as 150,000 surveillance cameras, including footage from electric vehicle company Tesla.

An international hacking collective executed the breach to demonstrate the ease of exposing video surveillance by targeting camera data provided by enterprise security startup Verkada.

In addition to footage from Tesla factories and warehouses, the hackers exposed footage from the offices of software provider Cloudflare, and from hospitals, schools, jails, and police stations.

Tillie Kottmann, one of the hackers claiming credit for the breach, said the collective obtained root access to cameras, enabling them to execute their own code; they exploited a Super Admin account to access the cameras, and found a username and password for an administrator account online.

A Verkada spokesperson said the company has disabled all internal administrator accounts to block unauthorized access.

From Bloomberg 

Tracking Tourists Everywhere

 Inevitable idea direction.  I recall the broad expansion of CCTV cameras in europe, this just adds more sophisticated tracking.  

Venice Is Watching Tourists' Every Move

CNN,  Julia Buckley,  January 13, 2021

Officials in Venice, Italy, are using technology to track the number of tourists in the city, and where they go. The goal is to create a more sustainable tourism plan for the city of 50,000 inhabitants, which welcomed nearly 30 million visitors annually prior to the pandemic. In September, the city launched the Venice Control Room on the island of Tronchetto, featuring a CCTV room where police monitor images from cameras around the city. A Smart Control Room across the corridor also has cameras, but images generated by those cameras, along with aggregated phone data, are used to create visitor profiles. Said Marco Bettini, whose firm Venis built the system, "We know in real time how many people are in each part [of the city], and which countries they're from." The data is important given that city has had to close its main thoroughfares to non-locals at busy times to address overcrowding.  ...'

Cyborg Eye Powered by Sunlight

A plantable sensor?  Privacy considerations?

Human-like Cyborg Eye Could Power Itself Using Sunlight
New Scientist
Donna Lu
May 20, 2020

Researchers from the Hong Kong University of Science and Technology (HKUST) and the University of California, Berkeley, have developed a spherical visual sensor that mimics the structure of the human eye. The sensor contains a lens to focus light and a hemispherical retina filled with densely packed light-sensitive nanowires made from perovskite, which is commonly used in solar cells and could enable the eye to be self-powering, said HKUST's Zhiyong Fan, although the current version of the eye requires an outside power source. When images of letters were projected onto the artificial lens to test how well it worked, a computer hooked up to the eye successfully recognized the letters E, I, and Y. The artificial eye’s low image resolution compared with commercial sensors limits its utility, but existing visual prosthetic devices use a flat object for image sensing, said Fan, limiting the possible field of view compared with a human eye.  .... "

Airports Testing Thermal Cameras

Inevitable to see this general screening, for liability and more ...

Airports are testing thermal cameras and other technology to screen travelers for COVID-19  by Hugo Martín in TechxPlore

Airports equipped with full-body scanners, metal detectors and face-recognition technology to identify potential terrorists are starting to make room for devices to target the latest global threat: travelers infected with the novel coronavirus.

Airports in the U.S. and overseas are testing thermal cameras, sanitation booths and other technologies in hopes of slowing the spread of the virus while reducing the risk of exposing airport screeners to the disease.

"This is going to be part of our normal travel system," said Richard Salisbury, managing director and founder of Thermoteknix Systems, a British company that has developed thermal cameras for airport use. "It will be part of the fabric of our travel patterns."

The pandemic has pushed demand for air travel down by as much as 95% in the U.S., and airport operators hope new screening technologies will give passengers renewed confidence to fly again.  ... "

Next Level of Police Enabling Technology

Obvious next step.   I see this is being done with our local police nearby, will look into this and see if I can get a demonstration.

Axon Rolls Out the Next Level of Police Technology: Live-Streaming Body Cameras   The Washington Post     By Tom Jackman

Police body-camera supplier Axon has deployed live-streaming cameras to the Cincinnati Police Department, allowing officers to show dispatchers or commanders crises as they unfold in real time, and helping rescuers find officers in trouble. The system automatically turns the camera on when a gun is drawn, emergency lights are activated, or a Taser is powered up. While the cameras will not include facial recognition software, they will have face-detection capabilities so police can quickly find video segments with people and react faster when footage is required for wider dissemination. Said Barry Friedman, a New York University law professor and founder of the Policing Project, “Body cameras go into sensitive places. With streaming, it won’t just be the officer, but somebody else. There have to be serious limits as to whom the video is streamed.”....'

A Billion Cameras Will be Watching

Inevitable.    But where is the data going and how is it used?  Does the installation of cameras consider future advances that will use that data in more invasive ways?  Such future use transparency does not now exist.

A World With a Billion Cameras Watching You Is Just Around the Corner
The Wall Street Journal  (With paywall) 
By Liza Lin; Newley Purnell
December 6, 2019

Industry researcher IHS Markit expects the number of cameras used for surveillance to rise above 1 billion by the end of 2021, marking an almost 30% increase from the 770 million such cameras in use today. China will continue to account for more than 50% of the total, but fast-growing, populous nations such as India, Brazil, and Indonesia also will help to drive growth in the sector. The global security camera industry has been spurred by developments in image quality and artificial intelligence (AI), technologies that allow better and faster facial recognition and video analytics. While “coverage of the surveillance market has focused heavily on China's massive deployments of cameras and AI technology,” observed IHS analyst Oliver Philippou, “future debate over mass surveillance is likely to concern America as much as China."... '

Augmenting Movie Making Using VR

Makes sense to use a visual medium to show alternative views to be chosen or combined.   Using game techniques.   No mention of a potential AR capability when combining shots with physical sets, but would be similar. How this would influence creativity remains to be seen.  It is likely that people who work with visual choices will start to see more of this kind of capability, with the further ability of seeing measures like cost, predicted audience reactions, continuity measures and more. So how might this work in other industries?  With these measures also leading to different kinds of optimization of overall goals.   On we go.

Directors Using VR on Set to Find the Perfect Shot 
New Scientist
By Andrew Rosenblum  in ACM

Technicolor has developed a virtual reality (VR) headset that allows film directors to instantly see what a scene will look like with computer-generated imagery added. Previously, directors had to send footage of human actors to render farms (groups of powerful computers) that would add graphical elements in a process that could take days to complete. The VR headset incorporates a stack of high-powered chips originally designed for gaming, which provide directors the opportunity to see immediately how a scene will look with the addition of computer-generated elements. The director can be immersed in the three-dimensional world of the scene and find the best camera angles. Independent filmmaker Kevin Margo said the technology can reduce the cost of visual effects up to 20% by trimming the expenses of lighting, and of the process to combine the elements into a single image. ... "

Ultra Long Range Cameras

Further advances with Camera and image interpretation technology.  The AI to recompose the image from multiple components.

An Ultra Long Range Camera  Daily Mail (United Kingdom)
By James Pero

Researchers in China have developed new camera technology that can render human-sized subjects from up to 28 miles away. The camera, which relies on a combination of laser images and advanced artificial intelligence software, can cut through smog and other pollution. The software uses a technique called "gating" that helps ignore photons reflected by other objects in the camera's field of view. Since the camera uses a laser to determine the distance of a subject by measuring how long the light takes to reflect back to the machine, the new software can tell the camera to ignore everything else that falls outside of that time signature. The camera uses a new algorithm to stitch together the data collected and form a recognizable image.  .... " 

Intel RealSense 3D Camera

Had heard of Intel's Realsense project some time ago, an unusual hardware play, when they were proposed for retail signage application.  Now the devices are out and deserve a look:

Intel's new cameras add human-like 3D vision to any machine
They're ready-to-use devices meant for makers and educators.
By Mariella Moon, @mariella_moon in Engadget .... "

Smartphones vs Action Cams

Why Smartphones Won't be Replacing Action Cams Anytime Soon    By Gannon Burgett

In the wake of GoPro’s Hero6 Black announcement, one question that might come up for you is, do you really need this? In an age where water-resistant smartphones are capable of capturing 4K video at 60 frames per second (fps), is there still a market for GoPro, Yi, and other action camera manufacturers?

On the surface, it would appear as though there’s little need for consumers to purchase yet another camera when most phones are capable of capturing photos and videos with similar specs. But it doesn’t take much digging to realize the benefits that come with having a separate, dedicated camera designed to get down and dirty.  ... " 

Reconfigurable Camera Systems

Shree Nayar discusses "Cambits: A Reconfigurable Camera System" (cacm.acm.org/magazines/2017/11/222170), a Contributed Article in the November 2017 CACM.

 .... Cambit pieces can be assembled to create a dozen different imaging systems. To celebrate this assortment, Communications has published four different covers, each one featuring a different Cambit configuration. ... 

" ... Here, we present Cambits, a set of physical blocks that can be used to build a variety of cameras with different functionalities. Blocks include sensors, actuators, lenses, optical attachments, and light sources, assembled with magnets without screws or cables. When two blocks are attached, they are connected electrically through spring-loaded pins that carry power, data, and control signals. The host computer always knows the current configuration and automatically provides a menu of imaging functionalities from which the user can choose. Cambits is a scalable system, allowing users to add new blocks and computational photography algorithms to the current set. ... " 

Demonstration video:   https://vimeo.com/236437712

Google Clips Gathering, Analyzing Images

Saw Clips introduced this week.  An interesting application of an always on,  AI driven camera.  With shades of Life-Logging, but with better ways to make sense of all the imagery gathered.  Creepy, I don't know since we have control of the images its far more secure than most other channels of data gathering.

Google Clips camera lays the groundwork for our AI-powered future
It's what's inside that counts.
Jessica Conditt, @JessConditt

Allow me to make a bold prediction: Google's Clips camera is going to flop.

Clips is a $250 camera powered by artificial intelligence and designed to snap images of important moments as they happen, no human input required. At best, it'll probably sell OK at launch -- there will be a handful of cute videos showing how the camera performs while attached to a dog or the top of a baby's toy mobile, and the internet will briefly swoon. Maybe a few months later, it'll catch a crime in action, and we'll be reminded that these odd, all-observant cubes exist.

But, regardless of the viral content that comes out of Clips, it's not going to be enough to convince mainstream consumers to run out and drop more than $200 on a clip-on camera. Smartphones have cameras (really good ones, even), and a lot of people have smartphones. Clips might address a real problem -- freeing up users to experience life without worrying about filming it -- but no one needs this technology right now. Besides, it's kind of a creepy concept overall.

Allow me to make another, less bold claim: Google knows all of this. And while it would be great for the company's bottom line (and its data-collection department) if Clips takes off, it doesn't need the hardware to sell well right now. Google most likely has larger plans for Clips' software .... " 

Will we all be Wearing Cams Soon?

Followed this once at the request of a retail research client.  Tested it in the aisle. Will it now be common?

Why you'll wear a body camera

As costs go down, quality goes up and ease of use improves, wearable cameras will become more compelling, even at work. Here's why the body cam revolution is coming.     ... By Mike Elgan

" ... A few years ago, a first generation of so-called "lifelogging cameras" came and went. These failed in the market because they were two early on two fronts: The technology wasn't ready. And the public wasn't ready.

Now, increasingly, both are ready. As better technology improves the quality of images, enabling even 4k video and real-time live streaming, society increasingly acclimates to people taking pictures all the time. Those 1.2 trillion photos aren't all being taken in private spaces.

Just a few years ago, nobody could have predicted or imagined what's now acceptable public behavior with a smartphone camera. People shamelessly pose and posture in public for selfies without embarrassment. They take pictures of their food and drinks in restaurants. They take selfies in the bathroom mirror. ...  " 

Camera Obscura Eclipse

91%  Eclipse, Camera Obscura, Pinhole effect,  filtered through a Maple, dramatically reflected on our front porch.

And on same Porch, closer focusing tree creates tiny suns and 3D effect:

Khan Academy on Virtual Cameras

An introduction to virtual cameras.  How they use them to teach.  With influences by Pixar and others.  See also, Pixar in a Box.

Body Cameras for All

Do Police Body Cameras Really Work?
Sometimes police body cameras accomplish their intended purpose, but other times they backfire. And nobody knows why  ....  By Barak Ariel

A considerable IEEE analysis piece on the use of body cameras by police.    Which I think does not establish well enough what 'working' means.  The conclusion, they are not as effective as we might think.    Is it because such actions and and reaction occur in extreme situations?  Worth a look.

Also relates more generally to cams that could be attached to all of us, all the time.   Or while we work, or ... ?    To help us share work and progress with minimal effort.  Record effort. Or be monitored?  Orwellian yes,  Somewhat akin to 'Life logging', covered in this space for years, but seems to gone out of fashion, except for police work.

Recalls work we did with consumers, for pay in a laboratory setting, who wore such always-on cameras in the aisle during shopping.  Later I saw this done with unobtrusive EEG caps. The 'test consumer' could engage with product which then recorded their neurological state.   Could that be added to other such cameras to indicate emotional readiness in a difficult situation, warning the wearer?