Just putting this out there for peer review and input…
A recent outbreak of exotic Caulerpa (C. brachypus and C. parvifolia) in Aotearoa New Zealand has compelled us to think about how we might achieve greater surveillance productivity using ROVs to do transects vs. divers, dropcams, towcams. Note that this is all shallow coastal habitat (0-50m). A minimal viable solution might look like:
- Boat arrives at a surveillance target area (GPS location).
- ROV deployed.
- ROV manually driven along a linear transect capturing high-quality, geotagged hi-res static imagery of the seafloor (not video).
- Images reviewed and submitted as part of a surveillance dataset.
- Move to next location.
Future enhancements could then include:
- Autonomous ROV operation along the transect.
- Move beyond a simple linear transect to scanning of areas defined by a polygon, e.g. a bay or space.
- AI-based recognition of exotic Caulerpa.
- Image stitching for photogrammetry (required by other potential applications).
In terms of technical approach we’re thinking:
-
ROV and nav sensing: BlueROV2 Heavy with WaterLinked A50 for active stabilization, altitude locking and knowledge of relative positioning. A USBL + DVL combo adds cost, so was thinking DVL only. If we capture the GPS position of the ROV at the point of entry into the water, we can then apply the DVL relative position at any time to know the (approx) position of the ROV.
-
Imaging: Addition of a downward looking camera, 8Mpx + and additional area lighting (BlueRobotics 4x Lumen lights). Lowest cost/complexity would be a fixed camera manual focus and with a fixed FOV and distance to seafloor (altitude). Autofocus might be a plus. Worst case, if the platform is too unstable, we could consider a gimbal camera. USB3 or IP cameras.
-
Image triggering: With known FOV and simple camera calibration, we would (soft) trigger the camera (and lights) based on travel distance since the last picture taken. The intent would be to include some overlap to ensure 100% coverage along the transect (photogrammetry would require even more overlap).
-
Geotagging images: Would involve calculating GPS position for any given image from the GPS starting position + DVL relative position at the time that picture was taken.
For our initial testing we have two core criteria to meet: (i) image quality (resolution, clarity) sufficient to spot exotic Caulerpa at an early stage; and (ii) ability to reliably return to a place where it was found (based on the geolocation in the image).
I’d appreciate any input on the above. Some specific questions around implementation:
-
Any advantage in integrating any of this into the Ardusub/MAVLink stack? Or, is it best to just to use IP cameras, connect up to the topside computer and do it all there? (no custom code in Ardusub).
-
Does using the MAVLink Camera Manager do anything for us?
-
The best locational accuracy would mean triggering the cameras from the ROV code based on distance travelled since last trigger (as detected by the DVL). Can you think of a good way to achieve this?
-
Any advice on best approach for shipping images topside? FTP or HTTP server, GStreamer? What have people used for this. At the end of the day we want geolocated JPG images. Once again, any reason we’d consider doing this using MAVlink?
-
For the application interface: Is QGroundControl / Qt the right application to be working with here? Or, are we better off leveraging some other application technology for UX? I’ve seen some amazing aerial drone apps built using QGC and Qt fundamentally is designed for GUI development. Ideally we’d want to be able to create a simple user interface and render thumbnails on a map as images arrive top-side and get processed. In time we’ll want to be able to draw polygons and automatically structure waypoint paths…
The plan is to make this all open source and non-commercial. In the spirit of that, if you think some of this functionality should be in the core product then I’d be keen to discuss how we might help facilitate that.
