Hello, I want to use Visual-Inertial Odometry for my autonomous underwater vehicle. Due to my limited system (Jetson Nano and possibly RPi4B), I need to use a lightweight one. I am unsure which VIO system I should use. I am using ROS 2 Humble on Ubuntu 22.04. I need your help! Thank you.
@clyde can you look?
There are lots of good open source packages out there. VINS-MONO and X-VIO and a few well known ones but there are lots of others. Are you looking to run a single camera, or stereo? They are typically pretty heavy and might be a challenge running on a Jetson Nano or Pi4B, but I wouldn’t say it’s impossible. It would be dependent on the image size you’re trying to feed it and update rates you need to maintain along with other processes sharing the compute.
Hello, thank you for your response. I know this will be challenging. I am quite new to ROS2 and VIO. I found a package by @clyde — here it is: (GitHub - clydemcqueen/orb_slam3_ros: Monocular ROS2 ORB_SLAM3 driver), but I could not install it. I need some documentation to learn how to use any VIO system.
OrbSLAM is solid, but it’s even heavier as it’s a full SLAM system which is both handling odometry and mapping and looking for loop closure events which is basically when you revisit a location you’ve been to before that it will correct the odometry history to make the loop closure work (sum of odometry from a point back to itself should sum to zero).
It’s not trivial getting a system up and running – a good starting place would be a pipeline with a camera and IMU, ideally time synchronized, potentially triggered to get measurements at the same points in time.
An easier pathway could be an off-the-shelf sensor like an Intel RealSense camera rig:
This would work fine with a Jetson Nano or Pi4B since all processing is done onboard and they have ROS nodes ready to go. There are lots of options, not all will make sense. You need to have a sense of range from the sensor to your scene or surface (whatever it’s looking at), and will likely want to avoid the ones that use structured light projection (stick to stereo only) since the projector won’t work far through water. I think some research papers have demonstrated some of these sensors working in bottles underwater.
I think the widest baseline realsense rigs are effective up to ~6m.
Alternatively you can do a monocular setup which estimates scale motion with an IMU and single camera that relies on significant excitation to back out scale. This is an option but it not as consistent and reliable as direct means of estimating scale.
Thanks for the tips @jasonir129 -
It’s worth noting that only a few stereo cameras out there from Intel or other vendors don’t rely on on projection of infra-red light patterns to help with depth estimation. The Oak-D and intel 435i are cameras only, with no projectors.
This is necessary, because infrared light doesn’t go more than about 10cm through water, and doesn’t penetrate acrylic well either!
Thank you for your response. What I really need is to measure the distance I travel with high accuracy. If I had a DVL, this would be much easier, but unfortunately, it’s not possible due to financial constraints. Since I don’t have much knowledge in this area, I’m not sure what to do. Given the economic difficulties in my country, it’s also unlikely that I will be able to afford the camera you mentioned. A system that works reliably with ROS2 and provides accurate odometry data would meet my needs perfectly. As I mentioned, I browse GitHub for packages, but unfortunately, I encounter difficulties during installation.
Interesting, just took quick look at OrbSLAM3, looks interesting, could be fun to test.
Typically the way you test a ROS 1 or 2 package is you pull in their package and any dependency ROS packages into a workspace (src directory). You would then use the respective build system (catkin for ROS1 or ament if ROS2) to build things and you run the built binaries using the launch files in the launch folder. There are likely some lines you’ll need to edit in the launch folder or param files etc. to get things working on your system. You’ll also need either calibrated hardware or test data they might refer you too. For instance OrbSLAM can be tested with EuRoC data (rosbag2 in this case):
Running with data from here should be reasonably plug and play.
Agreed on this, thanks for the specific models. There are also fancy underwater stereo rigs from other vendors, but they get pretty pricey. We put together an awesome low cost rig using BR bottles with the clamps on a piece of starboard, but still this is somewhat of an effort to design.
1 Like
There are simpler routes, an RPi with a downward facing RPi camera (need to pick lens, also would recommend global shutter camera) would be a low cost setup that could work well. You could do a minimal setup with feature tracking and estimation motion scale using baro depth + bathymetry maps to estimate vehicle speed and help stabilize the autopilot EKF. This is a similar (not recommending using this) super low weight solution that uses a small range sensor for a UAV to estimate vehicle speed similar to an optical mouse:
You could do something similar but just need some way to estimate height above “ground” to help scale the feature motion.
Hi @samsam
Besides the other comments raised here, I feel compelled to note that there are some pretty big challenges no matter what software you use:
- You need a good view of the bottom / seafloor
- You need good lighting
- The seafloor needs texture that can be picked up by the feature tracker you’ve chosen
- Hopefully there is not too much other stuff moving around (particulates, vegetation, fish)
It is an interesting research topic, but I don’t know of any guaranteed solution. Search for “underwater visual monocular slam challenges”.
Feel free to raise issues on the repo if you are having install challenges on x86. I haven’t yet tried to compile it for ARM yet.
Good luck!
all great points!
it’s definitely not an easy capability to tack on and it has it’s potentially small windows of usefulness. it can also be computationally expensive to run – all of this should be considered.