ROV-based benthic imaging - thoughts?

[EliotBR (Eliot)]

Hi @PeterM, lots of interesting things here

Unfortunately I’m quite short on time at the moment, so will have to keep this reasonably brief, but happy to discuss further as relevant and as things progress.

@PeterM

1. … If we capture the GPS position of the ROV at the point of entry into the water, we can then apply the DVL relative position at any time to know the (approx) position of the ROV.

It’s possible to manually set the current global position, in which case (if the DVL is reporting valid velocities) you should be able to run ArduSub autonomously to follow a mission (e.g. scanning along your polygon or whatnot).

@PeterM

4. Geotagging images: Would involve calculating GPS position for any given image from the GPS starting position + DVL relative position at the time that picture was taken.

If you’re triggering image capture using software on the onboard computer (e.g. using a BlueOS extension) then it should also be able to get the current position from the MAVLink stream and add it to the image metadata, which would save having to do data alignment stuff in post.

@PeterM

(i) image quality (resolution, clarity) sufficient to spot exotic Caulerpa at an early stage

This is partially dependent on the vision system, and partially dependent on the environment and vehicle state. The vision system you’ll need to tune with hardware selection/design, but you can improve your chances of getting a good image by either stopping the ROV when taking an image (and potentially monitoring the accelerometer and velocity data to only capture when “still enough”) and/or taking multiple images and (automatically) choosing the sharpest one to keep at each location.

This thread may also have some relevant advice.

@PeterM

• Any advantage in integrating any of this into the Ardusub/MAVLink stack? Or, is it best to just to use IP cameras, connect up to the topside computer and do it all there? (no custom code in Ardusub).

I doubt there’ll be much value in modifying ArduSub itself, but it will likely be worth at least monitoring the MAVLink stream for position and motion telemetry data, and potentially for notice of intended image capture points (if they’re specified in the mission).

@PeterM

• Does using the MAVLink Camera Manager do anything for us?

Not for static image capture at the moment (so probably not), but potentially if you’re wanting a video stream that’s advertised over MAVLink.

@PeterM

• The best locational accuracy would mean triggering the cameras from the ROV code based on distance travelled since last trigger (as detected by the DVL). Can you think of a good way to achieve this?

I believe ArduPilot missions support events at waypoints, so for software triggering it should be possible to send a MAVLink message that your software (on the onboard computer or the control station computer) can monitor for and run the image capture process as appropriate, or for hardware triggering it could toggle one of the flight controller’s relay outputs.

@PeterM

• Any advice on best approach for shipping images topside? FTP or HTTP server, GStreamer? What have people used for this. At the end of the day we want geolocated JPG images. Once again, any reason we’d consider doing this using MAVlink?

Do you need the images as they’re taken, or only after the dive? That may influence what makes the most sense. I don’t see how MAVLink would be relevant for image transport, but (again) data from the MAVLink stream will likely be valuable for the geolocation side of things.

@PeterM

For the application interface: Is QGroundControl / Qt the right application to be working with here? Or, are we better off leveraging some other application technology for UX?

QGroundControl has existing Survey functionality, but not all of the functionality is available for underwater vehicles, and I don’t believe it supports sending and displaying the images as they’re taken.

If you actually need to change the functionality and/or interface then I suspect Cockpit will make the most sense for this (since it’s being designed to be much easier to modify than QGC), but that depends a bit on how soon you’re wanting your interface to be “done”. Feature-wise this seems relevant to Blue Boat so I suspect we’ll be happy to help with the development.

We’re planning an initial beta release of Cockpit towards the end of this quarter, at which point there’ll be some documentation available and we’ll more meaningfully be able to discuss available features and the customisation process.