Best way to run custom code on BlueROV2

General Discussion Research
1

Hello,
I have a custom image detection model that I would like to run on the BlueROV2, that way when we are diving with it it will be able to recognize the objects and in the future do potential autonomous movement around it. What would be the best way to run this code on the BlueROV2? Thanks

2

Hi @ssicari ,

You can run your custom image detection model in surface control and forward the object coordinates bounding box to BlueROV using either pymavlink or ROS. both options have their own advantages.

I have a basic implementation of simple object detection using both pymavlink and ROS available on my Github. This repository includes a simulation setup in Gazebo, which allows you to test and refine your model in a virtual environment before deploying it to the actual BlueROV.

3

Hi @ssicari,

Do you have dedicated hardware you’re wanting to run this on, or are you planning to run it on the onboard computer (Raspberry Pi)? If the latter then the recommended approach would be to develop an Extension that runs your code, but you’ll need to be mindful of the processing requirements of your program (video decoding and computer vision can be quite intense), and will potentially need to upgrade your onboard computer to have sufficient processing capability.

Alternatively

  1. there are machine-vision cameras that allow you to run models on the camera hardware (e.g. the Luxonis OAK cameras), or
  2. you could run the model on some other intermediary hardware that sits between the camera and the onboard computer, or
  3. (as @ryan354 suggested) you could process on the (topside) control station computer and send relevant control commands back to the vehicle.
1 Like
4

This is probably the way I will end up going about it, thank you!

5

I will probably do @ryan354 suggestion and do it on my topside computer. Thank you!

6

Hello,
I think to run your custom image detection model on the BlueROV2 for object recognition and potential autonomous movement, consider using a lightweight platform like a Raspberry Pi or a Jetson Nano connected to the BlueROV2.