we have been working on automating our BlueROV2 using a visual navigation system. Our prototype system has been tested in a pool and we are working toward some field tests soon. The hardware and software components are diagrammed below, and some videos demonstrating early trials are available at (https://www.youtube.com/watch?v=tOlrNdwt2I4, BlueROV_2 - YouTube).
The key hardware components are a Stereolabs ZED mini interfaced to an NVIDIA Jetson TX1, which communicates to the BlueROV2 Raspberry Pi via Ethernet (substituting for the topside computer). All of the visual navigation software is run on the TX1. Weâre using a slightly modified version of ORB-SLAM2 for localization and mapping (achieving ~10 fps with VGA images) and a basic trajectory controller to navigate predefined routes. The camera/computer enclosure was made by Sexton (http://www.thesextonco.com). The videos above show the ROV completing a âlawn mowerâ pattern as our ultimate goal is to use the system for mapping marine ecosystems.
A few questions for either / both @brianh and @deepsouth to get their options for my own (all be it poor coding knowledge) benefit
How well do you think this style system would work as a monocular system (with the camera down facing and assuming it is getting fed reasonable bottom images)
In your opinions is the IMU utilised in standard BlueROV2 Pixhawk (the Invensense MPU 6000 sample rate up to 8kHz angular rate ±250, ±500, ±1000, and ±2000°/sec) near suitable if not what sort of range of accuracy and sample rate do you believe is necessary to get usable data to determine the camera pose (I couldnât find what IMU is running in the ZED mini but it looks like it is reasonably up specced from a twitter quote âThe IMU of the ZED mini is running at a fixed 500Hz, both the RAW and the fused data are available from the SDKâ )
Not taking anything away from either of your fantastic work (as it makes great sense in driving the ArduSub project towards AUV navigation and is a huge plus in general) But given the BlueROV2 is a ROV and that it has a tether back to the surface and is typically driven by a laptop (and yes Andrew I understand you are running the control as a web server down on the ROV so in your case it may be lacking this topside computing grunt) Would it be a reasonable workable solution to have the ORB-SLAM2 for localization and mapping running on the topside computer and even if there was not complete loop closure these flight tracks would be of assistance to any operator
Hi Scott,
Most SLAM systems will work with monocular systems (ORBSLAM does and Iâve tested it with some underwater video sequences and it performed reasonably). Using only a monocular camera the path/map will lack an absolute scale, but depending on your needs that may not be a major issue.
The IMUs on both the BlueROV and ZEDmini are more than capable of supplying fairly accurate camera pose, and the IMU update rates far exceed the camera frame rates. More recent SLAM systems (VINS-Fusion, OKVIS, etc) make use of the IMU data to partially infer position, velocity, and acceleration changes and I imagine these systems would be more sensitive to the quality of the IMU (but we havenât tried them yet).
Finally, you could definitely run a SLAM system on the topside computer to aid navigation. youâd have to get the video, IMU, etc data up the tether but the topside computer would typically be more powerful and capable of running SLAM at high frame rates (which is critical) than an embedded computer.
thanks, Brian
Thanks for the link Andrew I have been playing around with underwater photogrammetry for 5 year (see below) and is the main driver for the BlueROV2 so as not to be as limited as a diver with depth and duration
Hi Andrew
That image was done with a standard GoPro but not from a ROV (just me as a free swimming diver)
I have done some photogrammetry from the ROV and good coverage is the hardest thing to get it working (hence the intrest in visual navigation ) I have a couple of projects in deeper water coming up very soon (110m and 145m) so outside the Waterlinked system
And some reference system of coverage at those depths would be great
Re augmenting the base unit at this stage just upgraded housing and 200m detachable tether
Looking for a ethernet hub and decent camera (considering a sony qx1 camera mainly for form factor )
Looks a nice little unit (2 x 6.2 x 7.9 cm) so easy to find a place to put it
Still tossing around different configurations a bit as I need to get more battery life into the unit hence push âstuffâ around a bit in the frame on a recent dive to 118m I used 14.8Ah of the 18Ah battery for just under 10 minutes on the bottom (seriously looking at a burn wire system (similar to Building the world's first consumer deep-sea dropcam) to add weight to the unit and jettison it when at or near the bottom (may take out about 4Ah of consumption and we can skull drag the unit by itâs tether back to the surface)
Ideally I would still love to get a decent camera (say at least greater than 12MP stills with good Iso) that would fit in a shortened 2" housing rather than a cut down 3" (think some of the Drone cameras [ZENMUSE X5 or X7 if you can get rid of the gimbals]) if I have to use a 3" housing then the Sony RX0 would also be in the mix
The biggest issue I have found is ISO and blur from motion hence better sensors and more light (sort of) as a solution
Etienne my biggest issue is $'s
Iâm not a commercial venture just an individual looking for and documenting shipwrecks so 's always come into it the need to spend money for fuel and time on the water searching for new wrecks V spending money on the SSS or the MB or the ROV they all compete for the few small 's and what will give the biggest overall bang for buck your
A second battery housing and something like your battery holder (sorry mate most likely a home build) are the most likely the 1st port of call (as well as the burn wire system)
Thanks
Scott
Finally Sorry @brianh I donât wish to hijack you thread again I really like what you are doing
. I definitely want one when testing is completed please
