Fruit Picking Robotics

 Fruit-Picking Robot Eve Ready to Harvest Apples Commercially, as Shortage of Workers Persists

By ABC News (Australia), May 25, 2023

Eve, the fruit-picking robot.

Australian startup Ripe Robotics spent four years developing its autonomous fruit-picking robot Eve, which is processing apples in Victoria-based orchards owned by agribusiness Ardmona amid a labor shortage.

Ripe Robotics' Hunter Jay said Eve picks several hundred apples at a time, using cameras on its arms to "look at the tree, measure the size and color of the fruit, and if it's ready to be picked. Then the arm will reach in with a soft suction cup, use air to suck the fruit in and twist it off a tree, and then place it on the conveyor belt."

Engineers trained the robot to identify apples using a massive neural network composed of thousands of images.

From ABC News (Australia)    

View Full Article    

Robots that Write their Own Code

Interesting general challenge, 

Robots That Write Their Own Code

WEDNESDAY, NOVEMBER 02, 2022

Posted by Jacky Liang, Research Intern, and Andy Zeng, Research Scientist, Robotics at Google

A common approach used to control robots is to program them with code to detect objects, sequencing commands to move actuators, and feedback loops to specify how the robot should perform a task. While these programs can be expressive, re-programming policies for each new task can be time consuming, and requires domain expertise.

What if when given instructions from people, robots could autonomously write their own code to interact with the world? It turns out that the latest generation of language models, such as PaLM, are capable of complex reasoning and have also been trained on millions of lines of code. Given natural language instructions, current language models are highly proficient at writing not only generic code but, as we’ve discovered, code that can control robot actions as well. When provided with several example instructions (formatted as comments) paired with corresponding code (via in-context learning), language models can take in new instructions and autonomously generate new code that re-composes API calls, synthesizes new functions, and expresses feedback loops to assemble new behaviors at runtime. More broadly, this suggests an alternative approach to using machine learning for robots that (i) pursues generalization through modularity and (ii) leverages the abundance of open-source code and data available on the Internet.

Given code for an example task (left), language models can re-compose API calls to assemble new robot behaviors for new tasks (right) that use the same functions but in different ways.

To explore this possibility, we developed Code as Policies (CaP), a robot-centric formulation of language model-generated programs executed on physical systems. CaP extends our prior work, PaLM-SayCan, by enabling language models to complete even more complex robotic tasks with the full expression of general-purpose Python code. With CaP, we propose using language models to directly write robot code through few-shot prompting. Our experiments demonstrate that outputting code led to improved generalization and task performance over directly learning robot tasks and outputting natural language actions. CaP allows a single system to perform a variety of complex and varied robotic tasks without task-specific training.  ... ' 

Unpiloted Military Helicopter

More autonomous devices. 

Unpiloted Military Helicopter Flies 134 km in Simulated Mission

By New Scientist, November 7, 2022

A Black Hawk helicopter flying autonomously.

The helicopter flew as low as 60 meters above the ground at 185 kilometers per hour between mountains, relying on onboard sensors to avoid crashing into anything.

As part of the U.S. Defense Advanced Research Projects Agency's ALIAS program, the U.S. Army flew an autonomous Black Hawk Helicopter at 185 kph (115 mph) as low as 60 meters (197 feet) above the ground for 134 kilometers (83 miles) to deliver 230 kilograms (507 lbs.) of simulated and real blood.

The helicopter flew between mountains using onboard sensors to avoid a collision.

The program is testing Sikorsky Innovations' MATRIX technology, a drop-in kit that can transform helicopters into autonomous aircraft.

Sikorsky's Igor Cherepinsky said, "It's a truly autonomous system. Once you explain what the mission is, you don't even need a data link–the aircraft makes all the decisions."

From New Scientist

View Full Article - May Require Paid Subscription   

Open-Source Fleet Management Tools for Autonomous Mobile Robots

Are I am working in, soon? 

At ROSCon 2022, NVIDIA announced the newest Isaac ROS software release, Developer Preview (DP) 2. This release includes new cloud– and edge-to-robot task...

Open-Source Fleet Management Tools for Autonomous Mobile Robots

NVIDIA Technical Blog by Gerard Andrews / October 19, 2022 at 09:12PM//keep unread//hide

task management and monitoring software for autonomous mobile robot (AMR) fleets, as well as additional features for ROS 2 developers.

NVIDIA Isaac ROS consists of individual packages (GEMs) and complete pipelines (NITROS) for hardware-accelerated performance. In addition to performance improvements, the new release adds the following functionality:

Mission Dispatch and Client: An open-source CPU package to assign and monitor tasks from a fleet management system to the robot. Mission Dispatch is a cloud-native microservice that can be integrated as part of larger fleet management systems.

FreeSpace Segmentation: A hardware-accelerated package for producing a vision AI–based occupancy grid in the proximity of the robot to be used as an input to the navigation stack.

H.264 Video Encode and Decode: Hardware-accelerated packages for compressed video data recording and playback. Video data collection is an important part of training AI perception models. The performance of these new GEMs on the NVIDIA Jetson AGX Orin platform measured at 2x 1080p stereo cameras at 30 fps (>120 fps total), reducing data footprint by ~10x.

Mission Dispatch and Client ... 

Kroger Rolls Out Belt Driven Self-Checkout Lanes

Will go to see this example of autonomy.

Kroger expanding new belted self-checkout to 20 Tri-State stores

Calls test of new high tech lanes a success  in RetailWire

Kroger says its test of new self scan checkout lanes was a success, and it is now adding them to 20 Cincinnati-area stores.

Kroger new checkout lanes   By: John Matarese    WCPO

CINCINNATI — Kroger says the test of its new high tech self-scan checkout lanes has been so well received by shoppers, it has just expanded them into 20 Cincinnati-area stores.

And it plans to add the new lanes in more stores this fall.    The checkout — if you haven't seen one in your store yet — uses a belt to speed up the process, just like the cashier uses in the standard lane.

This eliminates one of the biggest complaints about standard self-checkout lanes: The lack of space for all your items.   Many people struggle to get their purchases into the bags on the little carousel, sometimes dropping a glass jar.    .... '  

Autonomous Firefighting Robots

Recall a project about firefighting process I consulted on a number of yrs ago.  It was to early to suggest general automation then, but we had indications this was being looked at. 

Firefighting Robots Go Autonomous, By Scientific American, October 29, 2021  in CACM

The tradition-bound firefighting profession is poised for an influx of eccentric assistants. They range from contraptions the size of a toy wagon to two-ton beasts that resemble military tanks and can blast out 2,500 gallons of water per minute. Some move on rubber tires, some on steel tracks, and some fly. All are robots.

The high-tech devices can enter burning buildings too hot for human survival. They can penetrate smoke too toxic for human lungs. They are often faster, stronger, and more agile than the firefighters they work with. Most of the machines currently in use are remote-controlled, but researchers are now developing "intelligent" firefighting robots that can make decisions autonomously.

Researchers are working to change institutional reluctance to invest in devices tailored to meet varying niche needs.

From Scientific American Full Article

Autonomous Drones Attack

This will likely be a big piece of the defense and military future.

Drones May Have Attacked Humans Fully Autonomously for the First Time

New Scientist, David Hambling. May 27, 2021

  A recent report by the United Nations Security Council's Panel of Experts reveals that an incident in Libya last year may have marked the first time military drones autonomously attacked humans. Full details of the incident have not been released, but the report said retreating forces affiliated with Khalifa Haftar, commander of the Libyan National Army, were "hunted down" by Kargu-2 quadcopters during a civil war conflict in March 2020. The drones, produced by the Turkish firm STM, locate and identify targets in autonomous mode using on-board cameras with artificial intelligence, and attack by flying into the target and detonating. The report called the attack "highly effective" and said the drones did not require data connectivity with an operator.  .. " 

Future of Logistics: Autonomous High Speed Transporting

New Efforts in the future of logistics:

An Autonomous High-Speed Transporter for Tomorrow's Logistics  By Fraunhofer-Gesellschaft (Germany)  March 9, 2021

Able to sort large quantities of packages, this high-speed transporter can serve booming online and mail-order businesses well.

A new generation of automated guided vehicles under development at the Fraunhofer Institute for Material Flow and Logistics includes "LoadRunners" that use artificial intelligence and communicate via 5G to organize themselves as a swarm and execute jobs independently.

Automated aerial vehicles guided by artificial intelligence (AI) are under development at Germany's Fraunhofer Institute for Material Flow and Logistics (IML) to enhance logistics.

LoadRunners use simulation-based AI and communicate via 5G to form swarms and sort large amounts of packages.

The 20-drone LoadRunner swarm mimics a flock of birds, with each drone taking cues from neighbors' behavior, continuously adjusting flight direction and speed to match other units' trajectory; the reciprocal action of drones' individual decisions give rise to swarm intelligence that requires no central coordination.

Fraunhofer IML's Moritz Roidl said the LoadRunner can turn in any direction without pausing to maneuver, while an onboard camera captures 400 floor images per second to navigate even at high speeds in tight formations.

AI enables the LoadRunner to assume and execute jobs independently, which can help logistics centers accommodate increasing parcel volumes.

From Fraunhofer-Gesellschaft (Germany)

A Brain for Autonomous Mobile Robots

Brought to my attention with its specific applications in retail like inventory checking.

What is BrainOS?

BrainOS® is a pioneering AI software platform that powers the world’s largest fleet of autonomous mobile robots (AMRs) operating in indoor public spaces. Robots enabled by BrainOS navigate autonomously, avoid obstacles, adapt to changing environments, manage data, and seamlessly interact with end-customers.

OEM partners

Global manufacturing partners use BrainOS to successfully build, deploy, and support commercial robots at scale across industries and applications. Learn more about our partners.

End customers

We also deliver a cloud-based autonomy service that empowers end customers, including some of the largest brands in the world, to easily leverage the power of robotics to achieve mission critical tasks.

Making a machine move autonomously and safely is no easy feat. We don’t need to learn how to build them too. Our laser focus on creating the world’s best robotics AI platform means that we can deliver rapid innovation and bring more robotic applications to market than ground-up robot builders.

Best in class across the board

Combining our leading AI software technology with proven manufacturing equipment means customers get the performance they can count on day after day. This translates directly into enhanced productivity, reduced costs, and increased customer satisfaction.

Service & Support

Partnering with leading manufacturing partners means customers get access to the service and support networks that best fit their business needs. Other robotic vendors can’t match the type of comprehensive support that is available through our partner network.  ... "

Drones Doing Autonomous Acrobatics

Why, you might ask beyond artistic maneuvers?  Seems more an approach to training than anything else.  And as they say, addressing the limits of autonomy for tasks and risks. 

Algorithm Enables Quadcopters to Perform Acrobatic Maneuvers Autonomously
Daily Mail (U.K.)
Jonathan Chadwick

Researchers at Switzerland's University of Zurich and Intel have created a navigation algorithm that enables autonomous flying drones to perform challenging acrobatic maneuvers autonomously, based on onboard sensor measurements. The heart of the algorithm is an artificial neural network that integrates input from the quadcopter's onboard camera and sensors, and directly translates this data into control commands. The network is trained via simulated acrobatic maneuvers, which makes demonstrations by human pilots unnecessary, and enables training to scale to a large number of maneuvers without posing physical risks to the drone. The algorithm abstracts the sensory input from the simulations and transfers it to the physical world, and drones require just a few hours of training. Zurich's Davide Scaramuzza said, "This navigation is another step towards integrating autonomous drones in our daily lives."

Fully Autonomous AI Running a Wall Street Hedge Fund

With a number of implications/   Interesting claim here.  The emergence of legitimate DAOs Distributed Autonomous Organizations?

No humans required, the fully autonomous AI running a Wall Street hedge fund
by 7wData

I’ve been documenting the emergence of fully autonomous organisations – organisations that are able to run and scale themselves without the need for any human operators, or intervention – sometimes also known as Distributed Autonomous Organisations (DAO), for years now and I’m now starting to see the first production DAO’s appear and enter the market.

Aidyia, a new fully autonomous Hedge fund is a case in point, turned on late last year it’s a hedge fund that makes all of its stock trades using artificial intelligence (AI) – with no human intervention required – and as you’ll see Bridgewater Associates, the world’s largest hedge fund, Goldman Sachs and maybe JPMorgan Chase might not necessarily be that far behind. In fact the pace and rate of change in the industry is now becoming so rapid that it even has its own dedicated event – the Battle of the Quants, the next installment of which is in Frankfurt later this month.

“If we all die,” says Goertzel, a longtime AI guru and the company’s chief scientist, “it would keep trading.”   ... " 

Drones of all Types in the Smart City

Good general piece on the topic.  Inevitable.  Their form, security, management and capabilities are still to be worked out in sufficient detail.   Also the further specification of how autonomous these systems should be.

How drones and aerial vehicles could change cities   by Paul Cureton, The Conversation in TechxPlore

Drones, personal flying vehicles and air taxis may be part of our everyday life in the very near future. Drones and air taxis will create new means of mobility and transport routes. Drones will be used for surveillance, delivery and in the construction sector as it moves towards automation.

The introduction of these aerial craft into cities will require the built environment to change dramatically. Drones and other new aerial vehicles will require landing pads, charging points, and drone ports. They could usher in new styles of building, and lead to more sustainable design.

My research explores the impact of aerial vehicles on urban design, mapping out possible future trajectories.

Already, civilian drones can vary widely in size and complexity. They can carry a range of items from high resolution cameras, delivery mechanisms and thermal image technology to speakers and scanners. In the public sector, drones are used in disaster response and by the fire service to tackle fires which could endanger firefighters.

During the coronavirus pandemic, drones have been used by the police to enforce lockdown. Drones normally used in agriculture have sprayed disinfectant over cities. In the UK, drone delivery trials are taking place to carry medical items to the Isle of Wight.

Alongside drones, our future cities could also be populated by vertical takeoff and landing craft (VTOL), used as private vehicles and air taxis.  ... " 

Autonomous Drone piloting

Could have used this for our forestry resource measurements, which required expensive and time consuming surveys by road or helicopter.  Further doing this to determine embedded tasks for resource analysis. 

Drones can now scan terrain and excavations without human intervention  by Aarhus University in Techexplore

Drone pilots may become superfluous in the future. New research from Aarhus University has allowed artificial intelligence to take over control of drones scanning and measuring terrain.

A research project at Aarhus University (AU) in collaboration with the Technical University of Denmark (DTU) aims to make measuring and documenting gravel and limestone quarries much faster, cheaper and easier in the future.

The project has allowed artificial intelligence to take over the human-controlled drones currently being used for the task.

"We've made the entire process completely automatic. We tell the drone where to start, and the width of the wall or rock face we want to photograph, and then it flies zig-zag all the way along and lands automatically," says Associate Professor Erdal Kayacan, an expert in artificial intelligence and drones at the Department of Engineering at Aarhus University.  ... " 

Delivering Butter, Autonomously

Had no 'disengagements' that required the self-driving system needed to be interrupted.

A Self-Driving Truck Delivered Butter from California to Pennsylvania in Three Days
The Mercury News
Levi Sumagaysay in MercuryNews 

Startup Plus.ai has completed what is being called the first commercial freight cross-country trip by an autonomous truck, a 2,800-mile-run from Tulare, CA, to Quakertown, PA, that took three days to deliver 40,000 pounds of butter for Land O'Lakes. A human safety driver was aboard the autonomous 18-wheeler to take over if needed, accompanied by a safety engineer. The company said the trip was a smooth one with zero "disengagements," which occur when a self-driving system has to be suspended because of a problem. Ten to 15 companies in the U.S. are working on autonomous freight delivery, said Dan Ives of Wedbush Securities, who believes the trucking industry will be the first to adopt autonomous technology on a mass scale. “When the trucks can go long distance, that's when there will be significant ROI (return on investment)   .... "

Autonomous Entities

Also a strong interest of mine, how close are we to building such systems and trusting them in every day life?  Certainly too, in the near term such systems will contain people and other agents that will require monitoring and cooperative behavior.

Creating Autonomous  Entities: Optimizing Systems of Subsystems
Posted by Bill Schmarzo in DSC

I continue to be infatuated with the potential of autonomous entities that self-monitor, self-diagnose, self-fix and self-learn without human intervention.  The vision of leveraging new digital technologies, growing reams of customer, product and operational data, and advanced analytics to create autonomous entities – farms, oil fields, factories, airports, theme parks, vehicles, ships, trains, etc. – seems like science fiction.

Fortunately, the “3 Horizons of Digital Transformation” framework provides a roadmap leveraging for 1) today’s market share grabbing operational excellence initiatives to 2) advance to tomorrow’s IoT-induced smart entities that seek to optimize across interrelated use case decisions that lead to 3) the future world of autonomous entities that optimize their operations across a holistic and comprehensive utility function without human intervention (see Figure 1).

This blog will explore how organizations can build out their digital, data and analytic capabilities today in order to strive towards that Horizon 3 autonomous entity vision of the future.  But it won’t be a big bang approach to creating a Horizon 3 autonomous entity.  Instead, it’ll require mastering the economic power of data and analytics.  .... "

Deep Reinforcement Learning

Interesting piece, ultimately gets quite technical, but worth putting away for future consideration. Especially for real time learning and adapting of autonomous systems.

Berkeley Bair Newsletter (Introduction) 
Data-Driven Deep Reinforcement Learning
Aviral Kumar    Dec 5, 2019

One of the primary factors behind the success of machine learning approaches in open world settings, such as image recognition and natural language processing, has been the ability of high-capacity deep neural network function approximators to learn generalizable models from large amounts of data. Deep reinforcement learning methods, however, require active online data collection, where the model actively interacts with its environment. This makes such methods hard to scale to complex real-world problems, where active data collection means that large datasets of experience must be collected for every experiment – this can be expensive and, for systems such as autonomous vehicles or robots, potentially unsafe. In a number of domains of practical interest, such as autonomous driving, robotics, and games, there exist plentiful amounts of previously collected interaction data which, consists of informative behaviours that are a rich source of prior information. Deep RL algorithms that can utilize such prior datasets will not only scale to real-world problems, but will also lead to solutions that generalize substantially better. A data-driven paradigm for reinforcement learning will enable us to pre-train and deploy agents capable of sample-efficient learning in the real-world.  .... " 

Predicting Driving Personalities

Seen this mentioned before.  Would also likely be used by insurance companies. Right now only be 'predicted' by accident frequency, an example of prediction of behavior, another example that will be quickly accused of predictive bias?

Predicting People's Driving Personalities  By MIT News

Self-driving cars are coming. But for all their fancy sensors and intricate data-crunching abilities, even the most cutting-edge cars lack something that (almost) every 16-year-old with a learner's permit has: social awareness.

While autonomous technologies have improved substantially, they still ultimately view the drivers around them as obstacles made up of ones and zeros, rather than human beings with specific intentions, motivations, and personalities.

But recently a team led by researchers at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) has been exploring whether self-driving cars can be programmed to classify the social personalities of other drivers, so that they can better predict what different cars will do — and, therefore, be able to drive more safely among them.

In "Social Behavior for Autonomous Vehicles," the scientists integrate tools from social psychology to classify driving behavior with respect to how selfish or selfless a particular driver is.

Specifically, they use something called social value orientation (SVO), which represents the degree to which someone is selfish ("egoistic") versus altruistic or cooperative ("prosocial"). The system then estimates drivers' SVOs to create real-time driving trajectories for self-driving cars.  .... "

Autonomous Reasoning at Tricky Intersections

An example of the inherent complexity of driving scenarios and work being done to address them

Better Autonomous 'Reasoning' at Tricky Intersections
MIT News
By Rob Matheson
November 4, 2019

Massachusetts Institute of Technology (MIT) and Toyota researchers have developed a new model to help autonomous vehicles determine when it is safe to proceed into traffic at intersections with obstructed views. The model uses its own uncertainty to estimate the risk of potential collisions or other traffic disruptions at intersections. The system considers critical factors including nearby visual obstructions, sensor noise and errors, the speed of other cars, and the attentiveness of other drivers. Based on its measured risk, the system may instruct the car to stop, pull into traffic, or move slightly forward to gather more data. The researchers tested the system in more than 100 trials of remote-controlled cars turning left at a busy, obstructed intersection in a mock city, with other cars constantly driving through the cross street; the system helped the cars avoid collisions 70% to 100% of the time..... "

Legal Implications of Car Hacking

How much regulation is currently out there?  Don't recall signing off on responsibility when buying a car.  Implied?  See the section on legal precedents.  Was unaware of this group within RAND.   We had early connections with them.  Following. 
 
Who's Responsible When Your Car Gets Hacked?  in Rand.org by Doug Irving

In the future, when cars can drive themselves, grand theft auto might involve a few keystrokes and a well-placed patch of bad computer code. At that point, who will be liable for the damages caused by a hacker with remote control of a 3,000-pound vehicle?

Cars are becoming “fast, heavy artificial intelligences on wheels,” a recent RAND report cautioned—and that means they're becoming vulnerable. Potentially billions of dollars ride on that question of who has the legal responsibility to keep hackers from grabbing the wheel or cutting the brakes.

“These are not likely events, and there are lots of engineers working to make them even less likely,” said James Anderson, the director of the RAND Institute for Civil Justice and a coauthor of the study. “But they're not impossible. They will occur. It's at least worth some serious thought about what the legal consequences will be.”  ... '    

Database of Tomorrow: Self Driving and Autonomous

Was struck by the depth and further volume of data and implications?  The intro below, much more at the link

The Database of Tomorrow: The Self-Driving, Autonomous Database

September 18, 2019 | Artificial Intelligence Big Data Blog | Dr Mark van Rijmenam
This article is sponsored by Oracle – redefining data management with the world’s first autonomous database.

In the coming years, the amount of data we create worldwide will grow to 175 zettabytes of data per year by 2025, up from 33 zettabytes in 2018. Over half of this data will be created by the Internet of Things devices and over 60% of it will be enterprise data. By 2025, 30% of all the data created will be in real-time, offering organisations great opportunities to constantly optimise their business.

Clearly, the organisation of tomorrow is a data organisation. However, simply collecting vast amounts of data is not enough. You would also need to analyse the data for insights and change your organisational culture to benefit from it. According to McKinsey, data-driven organisations are 23x more likely to acquire customers, 6x more likely to retain customers and 19x more likely to be profitable. Being data-driven is good for business.

The Importance of Data Governance
When collecting petabytes of data, it becomes vital that this data is of high-quality. Organisations that focus on high-quality data are better able to deal with changing business environments and achieve strategic objectives. As such, in today’s data-driven world, data governance has become a necessity for organisations, and they cannot get away anymore with minimal effort. .... "