What does autonomy in marine robotics mean to you?

I have an open ended question for anyone here:

What does autonomy in marine robotics mean to you and what do you hope to get from it?

What do you hope that autonomy will allow you to do in the future? How will autonomy make marine robotics more useful to you?

The reason I ask this is because we’ve seen a striking increase in interest in autonomy and AI over our past few customer interest surveys. In 2016, 31% of respondents were interested in autonomy and AI. In 2018, it was 39% and in 2021 it was 51% or survey respondents. Interest is clearly growing.

But, “autonomy” is a huge category and can mean a lot of different things to different people. What does it mean to you and what do you hope to get from it?

7 Likes

In the World that I work in … sending ET out on its own and expecting it to do the mission package you programmed it to do with the run-time life that you have built it for and being able to get the data back to the home craft by either returning or a communications with the launching platform or others before it goes to the bottom of the ocean somewhere.

So the big show killers are Navigation, Power, Sensors and Communications. Those all are expensive hurdles to jump. If I had to rank life, I would put Power at #1 because nothing else works without it and then very accurate Navigation or homing capability so you don’t loose a platform.

Communications next because without that you might as well leave it on the kitchen table and come back after a while and stare at it. Same results without having any data from the platform and last but not least, depending on what you want to do, is the Sensor suite.

1 Like

Thanks, Harold! I agree - all of those systems are absolutely critical. You can’t really start thinking about autonomy until the basic system is working and working very reliably.

1 Like

Hey Rusty,

For us, it is driving to the data with an operator that doesn’t miss something or fall asleep. We have focused for years now on GPS Denied and -C2 (negative or no Command & Control) operations, as the logistical resources for manned-unmanned deployments are prohibitive.

And, sometimes the ‘roomba mode’, where we just wander until we find something interesting, provides the best overall picture of what is happening.

So, we began focusing on data-centric logic in our control theories. If we can pull data from our sensors into the navigational logic, then we can perform simple rule-based autonomy from there.

Example:

Have the USV conduct a surface water quality profile along the WP route (or grid). If the indicated parameters (say Chlorophyll A, temperature, 1m current direction & velocity, and DO) exceed threshold values, position keep and conduct a vertical profile. If the vertical profile indicates a thermocline with different variables, launch ROV and conduct a sinusoidal vertical profile from new WP1 to WP2 along the predominant subsurface current heading.

We can do all of this today, but it does not use AI as most people want to believe. I don’t think we are even close to that yet, as I would have no idea how to train the algorithm. We can, however, stack a bunch of rules into a flow program and conduct a mission. Think of those like macros or scripts. And, each macro can call on another - sort of like we do in CNC.

Today we can pull over 1000 parameters (water quality, Wx, physical, navigational, performance) into this logic - depending on the types of sensors installed. That was a lot of work and required a ton of code to translate sometimes proprietary vehicle CAN or sensor messages into a common format.

So, to really answer your question - why do we need autonomy in the first place? To create a force multiplier and find the data of interest so we can make new and informed regulation, compliance levels, reclamation, and restoration.

We are also very interested in marsupialism and further increasing the old ‘swarming’ concepts in autonomy.

Chandler

2 Likes

Hey Harold,

Power is indeed a huge challenge. We have been trying to slay that dragon for a while and have certainly tried all the trick technologies (hybrids, battery chemistries, fuel cell, etc.).

In the end, we are attacking the issue in the most basic and analog way possible in two ways. One, having control logic that helps the asset perform most efficiently is paramount. Why is the asset going North on its mission if that is into the wind or current? Can we take external environmental data or observation and optimize our mission grid?

Second, low power modes. Doing everything possible to determine why you use energy and optimize the heaviest loads is a main focus. Why do we have ballast weight if we could replace that with battery weight?

On the comms front, becoming communications agnostic was a huge benefit. We used to be adamant on the frequency or radio brands we used. Then, we began creating radio modules, so we could create failover.

But far and away the most beneficial was looking at every possible way to cut that link requirement. Put the smarts on the asset, and let it go. Yes, we need situational awareness and safety - but a text message from a USV miles away like “I’m on mission and conducting survey grid M01, no issues, completion with estimated 36% energy reserves” goes a long way to keeping everyone satisfied and assured things are going well…

Chandler, our company makes some crazy battery technology that has high power densities. It is a once shot use, but you can swap out the “power” area. Take a look at this link.

I have worked with these guys on some stuff for testing purposes; however, I don’t know if they sell the battery packs commercially. The other thing is I don’t think the price is going to be something in the hobby market but scaling upward there could be openings for use.

Thanks for the link - I wonder if that UUV / AUV tech was part of the OceanServer / Iver acquisition by L3?

Wow - combining smart usage and smart power certainly gives us more reach.

I don’t know for sure … I know we acquired the guys who make the IVER. Next time I see the Borg Queen I will asked if they have been assimilated yet :slight_smile:

Sorry for the looong reply…

For some time I have been working on a project (part as a work project and part as a hobby) where the main question has been:

Would it be possible to develop a cheap AUV/ROV hybrid that could perform inspections autonomously 24/7 and report back the findings over long periods of time (i.e. many months)?

Small size and smart construction methods are important to bring down the cost. E.g. 3D printing and using cheaper materials/methods.

Using easily available commodity products instead of one-off/low volume products should also bring down costs.

We either have to find these products or create large enough demand for these products to eventually bring down the cost while ensuring a healthy community of vendors and users

One way that we might easier improve the autonomous capabilities of drones is to look at the drone (AUV/ROV) mainly as a carrier of compute power and sensors to achieve autonomy.

That means:

  • Include enough low cost compute power to compensate for limited hardware capabilities
  • Use computer vision and machine learning to automate tasks
  • Improve capability and flexibility through regular software updates
  • Start with low complexity automation tasks and gradually expand complexity over time

To achieve 24/7 operations/availability it is necessary to deploy multiple units, to be able to recharge occasionally without downtime, to expand capabilities and to reduce/remove the dependency on a single point of failure.

The idea has been to deploy multiple drones that can create a dynamic acoustic mesh network for (critical) communication but would most of the time rely on operations alone or «emergent behaviour» where the units behave as a swarm, without relying on constant inter-communication).

Applications

Possible tasks for these drones could be:

Positioning & navigation

  • Map creation (visual SLAM)
  • Establish ground truth
  • Establish/locate landmarks/navigation points (man-made or natural)
  • Map/track position of assets
  • Object avoidance

Create 3D models of assets

  • Photogrammetry
  • Point clouds

Asset monitoring

  • Detect changes
  • Verify valve positions
  • Leakage detection
  • Mapping and tracking corrosion & ​cracks​

Detect and track

  • Detect man-made objects
  • Detect possible POI
  • Track pipeline and other ​infrastructure​

Software

We would like to be able to use/expand upon ArduSub or possibly ROS to control the drones autonomously (sensor fusion from several different sources would be needed) and perform smart missions, i.e. not relying on minutely planned/plotted missions by a human operator.

A lot of the features that would be needed are not implemented yet and we realize that this will not be easy. But if a group of people can find together and team up it will be easier to pull off. The resulting features should then be made available for the community.

Hardware

Some of the key hardware components we have looked into/are looking into are:

Charging station

We are currently building prototypes and have a 400w inductive charger up and running (currently artificially limited to 270w) with wireless gigabit Ethernet capabilities.

The receiver unit is connected to a 1.7kWh 10S10P 21700 battery with a smart BMS (we hope to use VESC based BMS eventually).

This charging station/solution should be available to many drone types to ensure proliferation.

Communication and positioning

We have multiple Nanomodem v3s (NM3). These are low cost and low power modems that are capable of approx. 640bps data transfer and positioning at approx. 2km range.

Software needs to be adapted/implemented for these units to enable mesh networking and positioning (needs 3 modems with known position to calculate position for other units within range).

Some of the drones will also need a DVL to navigate outside the reach of the acoustic network.

Computer vision

We plan to use multiple OAK-D (Lite) units for multiple vision based tasks. These cameras have depth capabilities and a Myriad X AI chip embedded and are cheap.

The camera sensors being used are very likely not light sensitive enough for deep water use, but Luxonis and Arducam have shown that they are capable of covering various use cases and I would also love to see official cooperation between Blue Robotics and Luxonis.

Sonar

We want to be able to use multi beam sonars and side scanning sonars both to collect data for later processing, but also real-time as part of navigation and mission decision making. This requires the data streams to be available for processing and the on-board compute power needs to be sufficient.

As of yet very little work has been done on this area except for identifying possible MB or side scanning sonars that could be used.

Compute

We plan to use a CM4 with Coral TPU units (sadly currently not supported by CM4), but we have heard from Luxonis that it might be possible to use the Myriad X processors embedded in the OAK-D units.

Hailo-8 also seems like an interesting AI accelerator that could be used, but more information regarding availability, pricing, compatibility and readiness are needed.

Alternatively Jetson Xavier NX or similar could be used. Or as a last resort a micro PC could be included.

Requirements

How to get the autonomy features that many of us likely wants?

  • We need to cooperate and build upon the great work of others.
  • We need to be willing to share competence and knowledge.
  • We need to be willing to give and not only take.

But it most likely has to be a coordinated effort.

We need to bring down costs and expand capabilities to create a community of users while ensuring that a thriving community of vendors that are able to provide the solutions can exist.

No companies or persons (or at least very few) would have the capacity/resources needed to pull this off alone, hence it is critical that this can be solved through open source solutions – either utilizing existing solutions or creating new open source solutions – both hardware and software.

3 Likes

We do research on autonomy so to us autonomy means picking out an application area that opens up some interesting new theoretical or practical direction for underwater robots and then executing on that. We’d expect that the robot is able to do some task like approaching an object, or avoiding obstacles, or stacking blocks.

More generally, the robot is loaded with some program, put into an environment, then expected to do something. This involves some basic sensor fusion, real time vision processing, and actuating things. I wrote out the hardware requirements we have right now for the kind of work we do below

To support our low cost AUV research, we need an AUV that:

  • Can be packed into a suitcase and taken on as regular checked baggage in a normal flight
  • Can be carried by one or two people over rugged terrain
  • Can be operated and debugged in situations where there is no access to internet or electricity
  • Can operate without a tether or any connection to a surface computer
  • Is safe enough to be carried to deployment locations underwater by divers
  • Provides some feedback to divers about its internal state (why isn't this thing doing anything?)
  • Provides some basic human-robot interaction capabilities so divers can tell it what to do
  • Have a decent quality, configurable, machine vision camera
  • Have computers that are fast enough to run the algorithms we are working on in real time
  • The person diving down with the robot may not be the one who wrote the code. There's gotta be good enough feed back to the operator that they can make on-the-fly decisions about the experiments they are running.
3 Likes

Autonomy to me in marine robotics is throwing an untethered AUV off the boat, that is programmed to swim out to deep water, find fish, record the depth and position, take some pictures, then swim back to the boat and download the data to my computer. If I like what the fish pictures show, I then will load the fish location into my GPS controlled boat trolling motor, which will take me where the fish are, and I start fishing. Can’t get much more autonomous than that. I think Maine lobster fisherman would like to have one of these also.