How are those Drone Swarm Light Shows Done?

I mentioned that we proposed drone and robotic swarms for industrial trials way back,   Now we are seeing the same kind of thing in shows that replace fireworks.  Such as in the opening ceremonies of the Olympics.   For spectacular entertainment.   How are these produced, effectively and technically?   Do they point to other 'swarm' activity other than entertainment?   Was pointed to some resources:

Technical Overview:  https://verge.aero/everything-about-drone-light-shows/ 

https://www.sciencedirect.com/search?qs=drone%20swarms    General articles on Drone Swarms

Intel services in creating and running Drone Swarm shows:

https://www.intel.com/content/www/us/en/technology-innovation/aerial-technology-light-show.html

Future of Logistics: Autonomous High Speed Transporting

New Efforts in the future of logistics:

An Autonomous High-Speed Transporter for Tomorrow's Logistics  By Fraunhofer-Gesellschaft (Germany)  March 9, 2021

Able to sort large quantities of packages, this high-speed transporter can serve booming online and mail-order businesses well.

A new generation of automated guided vehicles under development at the Fraunhofer Institute for Material Flow and Logistics includes "LoadRunners" that use artificial intelligence and communicate via 5G to organize themselves as a swarm and execute jobs independently.

Automated aerial vehicles guided by artificial intelligence (AI) are under development at Germany's Fraunhofer Institute for Material Flow and Logistics (IML) to enhance logistics.

LoadRunners use simulation-based AI and communicate via 5G to form swarms and sort large amounts of packages.

The 20-drone LoadRunner swarm mimics a flock of birds, with each drone taking cues from neighbors' behavior, continuously adjusting flight direction and speed to match other units' trajectory; the reciprocal action of drones' individual decisions give rise to swarm intelligence that requires no central coordination.

Fraunhofer IML's Moritz Roidl said the LoadRunner can turn in any direction without pausing to maneuver, while an onboard camera captures 400 floor images per second to navigate even at high speeds in tight formations.

AI enables the LoadRunner to assume and execute jobs independently, which can help logistics centers accommodate increasing parcel volumes.

From Fraunhofer-Gesellschaft (Germany)

Drone for Exploring Caves, and other Tight Spaces and Situations

Drones that fold and unfold to enter and explore.  Remember having to deal with that in one of my few caving efforts.  Also includes discussion of their acting as swarms.  Quite interesting detail here: 

Ridley Scott inspires Network Rail’s Cave-exploring drone

By Chris Baraniuk in the BBC     Technology of Business reporter

When Hollywood director Ridley Scott filmed astronauts looking for aliens in his 2012 movie Prometheus, little did he know that it would one day inspire a Network Rail drone project.

But, bizarrely, that's exactly what has happened. Network Rail, which owns and maintains most of the railway infrastructure in Britain - from vast stretches of track to thousands of bridges, tunnels and level crossings - has long sought cheaper and easier ways of surveying the many underground caves and abandoned mines dotted around its property.

Engineers must account for "shallow" subterranean voids when doing building work or moving heavy machinery, because the caverns might collapse and cause the ground above to shift. There are more than 5,000 shallow voids in Britain.

At an innovation workshop in 2018, Network Rail's principal mining engineer Neal Rushton found himself seated at a table with electronics experts and robot researchers. He sketched a simple drawing of the sort of device he was looking for - something that could travel down a 15cm borehole, enter an underground cave and map its interior.

Existing methods of mapping these areas are slow and sometimes hazardous. Surveyors either drill multiple boreholes into the ground, through which they poke sensors at the end of long sticks in order to scan the space below; or technicians enter the mine shafts and caverns themselves to collect the data by hand. Dangerous gases, cave-ins and even explosions are all possible causes of harm to such workers.  ... " 

Cyborg Cockroaches

Remain interested in small and very small robotic devices.    And cooperating swarms.   They don't need to be scarythough.

A Swarm of Cyborg Cockroaches That Lives in Your House

Japanese researchers say turning cockroaches into an army of insect cyborgs could be useful in a variety of ways   By Evan Ackerman

Coordinating Complex Behaviors Between Hundreds of Robots

Back to the control of swarms.  Intelligence is good, but collective intelligence is better?  The process described here is very interesting.

Coordinating Complex Behaviors Among Hundreds of Robots
Duke University Pratt School of Engineering
Ken Kingery
July 1, 2020

Duke University researchers have proposed a new approach for coordinating complex tasks between hundreds of robots while satisfying logic-based rules. The method, called STyLuS* (large-Scale optimal Temporal Logic Synthesis), bypasses the traditional requirement of building incredibly large graphs of each robot's locations or nodes by producing smaller approximations with a tree structure. At each step of the process, the algorithm randomly chooses one node from the large graph, adds it to the tree, and rewires the existing paths between tree nodes to find more direct paths from start to finish. STyLuS* also selects the next node to add based on data about the task at hand, allowing the tree to quickly approximate a good solution to the problem. The algorithm solves problems exponentially: it answered the challenge of 10 robots searching through a 50-by-50 grid space in about 20 seconds, while state-of-the-art algorithms would take 30 minutes.  ... " 

Swarms of Drones

We examined the idea of having swarms of drones performing subtasks in groups. its been much kicked about, and recently implemented, especially for military applications.

AI helps drone swarms navigate through crowded, unfamiliar spaces
It could be key to self-driving cars, not to mention search and rescue.
By Jon Fingas in Engadget

Drone swarms frequently fly outside for a reason: it’s difficult for the robotic fliers to navigate in tight spaces without hitting each other. Caltech researchers may have a way for those drones to fly indoors, however. They’ve developed a machine learning algorithm, Global-to-Local Safe Autonomy Synthesis (GLAS), that lets swarms navigate crowded, unmapped environments. The system works by giving each drone a degree of independence that lets it adapt to a changing environment.

Instead of relying on existing maps or the routes of every other drone in the swarm, GLAS has each machine learning how to navigate a given space on its own even as it coordinates with others. This decentralized model both helps the drones improvise and makes scaling the swarm easier, as the computing is spread across many robots.  ... "

Autonomous Swarming Pods

A favorite topic, distributing tasks, combining to solve tasks.

Autonomous Pods SWARM Together Like Bees in World-First Demonstration
University of Warwick
January 30, 2020

Researchers at the University of Warwick's Warwick Manufacturing Group (WMG) in the U.K., in collaboration with self-driving technology maker Aurrigo and the town of Milton Keynes, have created autonomous pods (driverless vehicles) that swarm together. The pods are engineered for pedestrian areas and shared spaces, and the researchers enhanced the pods with swarm intelligence to cluster into a "platoon" and follow each other, keeping individual vehicle movements and supervision to a minimum. The pods also can automatically optimize their behavior to satisfy future passenger demand by distributing themselves within a city to locations where rides are most likely to be requested. Aurrigo's Simon Brewerton said, "The collaborative SWARM algorithms have been developed to enable our autonomous vehicles to optimize their own trip schedules, so they deliver the optimum efficiency from a fleet of vehicles."  ... '

Samsung Patents the Drone Screen

In our innovation center we looked at many variants of the screen.  Especially as it related to in store and public retail advertising.  Here something new, a drone that resents a screen.  With lots of possibilities around scale.  The current winter Olympics showed how swarms could be impressive, why not displays integrated into such systems.  Nothing demonstrable as yet, just a patent.

Samsung patents a flying screen that could be used for hovering video
It may also be able to adjust its position based on the angle of your gaze.

By Mallory Locklear, @mallorylocklear in Engadget .... " 

Molecular Swarms

Letting Molecular Robots Swarm Like Birds   Hokkaido University  By Naoki Namba

Researchers at Hokkaido University and Kansai University in Japan have created DNA-assisted robots that autonomously swarm in response to chemical and physical signals, which could lead to the development of future nano-machines. The new "swarm robots" measure 25 nanometers in diameter and five micrometers in length, and they exhibit behavior similar to that of motile organisms such as fish, ants, and birds. "These include the formation of complex structures, distinct divisions of labor, robustness, and flexibility, all of which emerge through local interactions among the individuals without the presence of a leader," says Hokkaido University's Akira Kakugo. He notes the new system has three essential robot components--sensors, information processors, and actuators. The researchers used cellular proteins called microtubules and kinesins as the actuator, and DNA as the information processor. "The system acts as a basic computer by executing simple mathematical operations, such as AND or OR operations, leading to various structures and complex motions," Kakugo says.   ....  "

Hive Robotics

Swarming robotics to perform tasks collaboratively is something we examined in warehouse applications.  A different, but structurally similar problem came up recently and I did some searching for the state of the art. This example came up.  A way to mock up the idea at least.   See the video linked to:

In these bot hookups, the machines meld their minds    In Science News: 

New composite robots take coordination to a new level   By Maria Temming   Meet the first fleet of hive-minded robots.

These robots can latch onto one another and link up their individual “nervous systems” — the network of connections between their computer processors, cameras, wheels and other gadgetry — to create a single-minded machine. The composite robots, reported online September 12 in Nature Communications, pave the way for a new generation of machines that can change their shape on the fly.

In previous robots made up of many smaller machines, each robotic building block maintained control over its own nervous system. It was “a little bit like if we had a bunch of people joining together to do something,” explains study coauthor Marco Dorigo, an engineer at Université Libre de Bruxelles in Belgium. “They were trying to coordinate by exchanging messages with neighbors without anyone being in charge.” These multi-bodied robots could link up to create different shapes, unlike solo devices. But the lack of centralized control meant the robot collectives were clumsier and had slower reaction times than stand-alone bots.  .... " 

Swarms of Internal Drones

Precisely what we proposed at one time internally, but the tech of the time did not permit it.  Primary issues was battery life, size and sensor capability.  Linking to RFID was attractive.   Tasking these swarms to look for maintenance issues as well as taking inventory.

MIT researchers use drone fleets to track warehouse inventory
The small, safe UAVs can be used with existing RFID-tracking setups.  by David Lumb

Imagine a warehouse buzzing with tiny drones that automatically track and monitor inventory from afar, leaving workers free to manage and move material. That's the new system developed by MIT researchers, which could prevent mismatches and help employees find particular items faster on top of looking really cool with a bunch of worker drones zipping around.  ... " 

Biology of the Evolution of Cooperation

Very old problem in Biology.   Why should organisms cooperate? A solution?  Could this same technique be used when solving genetic algorithms?  Or say the collaborative genetic evolution of intelligent swarms?

New Model of Evolution Finally Reveals How Cooperation Evolves

By treating evolution as a thermodynamic process, theorists have solved one the great problems in biology.    by Emerging Technology from the arXiv  June 21, 2017

One of the great unanswered question in biology is why organisms have evolved to cooperate. The long-term benefits of cooperation are clear—look at the extraordinary structures that termites build, for example, or the complex society humans have created.

But evolution is a random process based on the short-term advantages that emerge in each generation. Of course, individuals can cooperate or act selfishly, and this allows them to accrue benefits or suffer costs, depending on the circumstances. But how this behavior can spread and lead to the long-term emergence of cooperation as the dominant behavior is a conundrum that has stumped evolutionary biologists for decades. ... " 

AI's Cooperating

Perhaps the most important question of all,  both in the relatively simplistic sense of assistants cooperating with us, and the much deeper sense of working together.  Both considerable challenges. Recall our look at approaches like swarms.

Google Just Found the One Question It Can’t Yet Answer  by Jeremy Kahn
 DeepMind explores emergence of competition, cooperation
Finds environment plays big role in fostering AI cooperation ... " 

Autonomous Collaborating Micro-Drone Swarms

The autonomy of such drones and drone swarms indicates an unprecedented depth of decision making driven by AI.  Implications are very broad.

Pentagon successfully tests micro-drone swarm  In mPhys: 
The Pentagon may soon be unleashing a 21st-century version of locusts on its adversaries after officials on Monday said it had successfully tested a swarm of 103 micro-drones. The important step in the development of new autonomous weapon systems was made possible by improvements in artificial intelligence, holding open the possibility that groups of small robots could act together under human direction.

Military strategists have high hopes for such drone swarms that would be cheap to produce and able to overwhelm opponents' defenses with their great numbers.

The test of the world's largest micro-drone swarm in California in October included 103 Perdix micro-drones measuring around six inches (16 centimeters) launched from three F/A-18 Super Hornet fighter jets, the Pentagon said in a statement.

"The micro-drones demonstrated advanced swarm behaviors such as collective decision-making, adaptive formation flying and self-healing," it said. ...." 

And in Ars Technica.

Deep Learning Solving the Complex Subtasks

Good example of how, by breaking up complex sub tasks, you can perform difficult goals.  We, as humans, don't have to 'solve' the problem of sight every time we want to see, and thus can get to more difficult goals.  Also naturally linking to the swarm solution idea.

2016: The Year that Deep Learning Took over the Internet     by Cade Metz, In Wired: 

" .... On the West coast of Australia, Amanda Hodgson is launching drones out towards the Indian Ocean so that they can photograph the water from above. The photos are a way of locating dugongs, or sea cows, in the bay near Perth—part of an effort to prevent the extinction of these endangered marine mammals. The trouble is that Hodgson and her team don’t have the time needed to examine all those aerial photos. There are too many of them—about 45,000—and spotting the dugongs is far too difficult for the untrained eye. So she’s giving the job to a deep neural network.

On the West coast of Australia, Amanda Hodgson is launching drones out towards the Indian Ocean so that they can photograph the water from above. The photos are a way of locating dugongs, or sea cows, in the bay near Perth—part of an effort to prevent the extinction of these endangered marine mammals. The trouble is that Hodgson and her team don’t have the time needed to examine all those aerial photos. There are too many of them—about 45,000—and spotting the dugongs is far too difficult for the untrained eye. So she’s giving the job to a deep neural network.  ....  " 

Robots Working Together

A topic of interest that addresses applications like cooperation and teamwork in robots.   Working with each other or with people.

From GaTech:

Team of Robots Learns to Work Together, Without Colliding
Georgia Tech News Center (09/06/16) Jason Maderer 

New algorithms created by Georgia Institute of Technology (Georgia Tech) researchers enable robots to move within inches of each other, without colliding, to complete their task. The team, led by roboticist Magnus Egerstedt, director of Georgia Tech's Institute of Robotics and Intelligent Machines, is the first to create such minimally invasive safety algorithms. A set of safe states and barrier certificates ensure each bot stays in its own safe set throughout the entire maneuver, the researchers note. In a demonstration involving four robots, the machines approach from four different areas, meet in the middle, circle counterclockwise within inches of each other, then fan out into opposite directions. In another demonstration, eight robots perform the same task, this time circling clockwise before dispersing. Instead of keeping their distance and taking the long way around their neighbors, the robots move very independently wherever they wish. "Robots are very conservative--they want to make sure they're safe," Egerstedt says. "You couldn't pack the interstate with self-driving cars with today's technology." He says a minimally invasive safety controller similar to the algorithms could be used for the next generation of air traffic control.  ... " 

Swarming, Folding Robotics

 Quite a trick to consider:

Harvard Building Swarm of Self-Folding Origami Robots

One of the challenges with swarms of robots is manufacturing and deploying the swarm itself. Even if the robots are relatively small and relatively simple, you're still dealing with a large number of them, and every step in building the robots or letting them loose is multiplied over the entire number of bots in the swarm. The dream for swarm robotics is to be able to do away with all of that, and just push a button and have your swarm somehow magically appear. We're not there yet, but we're getting close: Harvard researchers recently demonstrated a collective robot swarm that can be manufactured in a single composite sheet. On command, they'll rip themselves apart from each other, fold themselves up into origami structures, and head off on a mission en masse.  ... " 

Intelligent Home Automation

The application of local intelligence.  Smarter Home Intelligence.

A Better Paradigm

With all the hype and admiration of what these companies have achieved there must be another way that exposes the user of these exciting technologies somewhat less and makes her sleep better. And that still works with the Internet connection down …

Think Edge Computing.

Let’s have a look at Tesla, Apple Car, Uber, Google Car. All of these models of self-driving cars rely on a very powerful AI, and on machine learning, too.

But they employ local intelligence in combination with swarm learning. This means that the intelligence is on the car, and for several good reasons that I do not want to discuss here.
Apply this to the smart home!  ... " 

Waze Making us Think Like Ants in Traffic

Ants avoiding traffic. Reminds me of swarming algorithms for tracking or searching.  Considered it in warehouses.   Have used Waze from time to time but have never had it direct me 'algorithmically'.    Doubt if many people do.  I do think this might be just the thing for self driving cars.    Once people trusted the swarm.

Robots Learn from Insects

Have heard relatively little about genetic methods lately, here as a means of adapting initial descriptive solutions. Worth considering to achieve some process optimization in complex situtations.

In CACM: " ... Paris Diderot University researchers, led by researcher Jose Halloy, have developed a way to generate a robot cockroach's behavior automatically using a combination of descriptions of cockroach habits, combining models of individual movements with group activity. The researchers then used evolutionary algorithms to optimize the models.  "