Running BlueROV2 in autonomous mode by coding on Rpi

[sjpanwar (Sanjay panwar)]

We are doing a a project to change the recently purchased BlueROV2 to autonomous with image capture followed processing and then according navigation.
At present, we are able to control Blue ROV2 through groundcontrol joystick and additionally we are able command ROV through mavproxy(arm throttle, rc 3 1600 etc. ) on Rpi.
Now we are planning to write python script in Rpi attached to pixhawk (ardusub) through USB. Can you send some command lines to move Blue ROV2 (Arming/disarming, forward, reverse, lateral right/left, depth hold mode etc.) in python language. We would like to run this python script on Rpi bootup.

Any help would be amazing! Thank you

[patrickelectric (Patrick José Pereira)]

Hi @sjpanwar,

Please, take a look in the forum, probably you will be able to find out a lot of related posts.

Autonomous Bluerov2

Hello, I think it is important for you to understand how your autonomous vehicle works in order to experiment safely. In order to understand it, you need to learn about it. I am happy to help you find the information that you need, and we can make some ‘getting started’ python examples in the future that people will be able to build on to make their vehicle move. Please start by looking at MAVProxy and pymavlink, as well as searching the forum for many related posts. Let us know if you have any…

And try to wait a bit for a reply, you answer was already provided in your first request:

Getting started with MAVproxy

Hi @sjpanwar, Check the QGC code to see what kind of mavlink messages are used, they sounds like exatcly whay you want, take a look in the different modes and what message is used for each one of them. If you want to send mavlink message with python, most messages follow this rules: message_send. E.g: #start connection conn = mavutil.mavlink_connection(args*) #send SET_ATTITUDE_TARGET self.conn.mav.set_attitude_target_send(args*) #send SET_POSITION_TARGET_LOCAL_NED self.conn.mav.set_positio…

[ggttkl (gtl)]

Hello, I am doing the project to autonomous ROV with image capture followed processing and then according navigation, too. And now I can send some basic commands to Ardusub(pixhawk) directly on Rpi. However, the image information is transmitted to QGC through Rpi,and I cannot find any instructions to read or process the image information directly on Rpi board.
I have both the Rpi camera and HD USB camera and I want to read both of the cameras data on Rpi board. Can they both on work at the same time?
Thank you for any possible help!

[patrickelectric (Patrick José Pereira)]

Hi,

You can check the cameras under the path /dev/video*, running the v4l-ctl (application to control video4linux drivers) you’ll have all the information needed to perform your task.

First, try to list all devices:

pi@raspberrypi:~ $ v4l2-ctl  --list-devices
mmal service 16.1 (platform:bcm2835-v4l2):
	/dev/video2

H264 USB Camera (usb-3f980000.usb-1.2):
	/dev/video0
	/dev/video1

To know the format of each video:

pi@raspberrypi:~ $ v4l2-ctl --list-formats-ext --device=/dev/video0
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'MJPG' (compressed)
	Name        : Motion-JPEG
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 800x600
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 640x360
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 352x288
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)

	Index       : 1
	Type        : Video Capture
	Pixel Format: 'YUYV'
	Name        : YUYV 4:2:2
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 800x600
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 640x360
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 352x288
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)

pi@raspberrypi:~ $ v4l2-ctl --list-formats-ext --device=/dev/video1
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'H264' (compressed)
	Name        : H.264
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 800x600
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 640x360
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 352x288
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)

pi@raspberrypi:~ $ v4l2-ctl --list-formats-ext --device=/dev/video2
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'YU12'
	Name        : Planar YUV 4:2:0
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 1
	Type        : Video Capture
	Pixel Format: 'YUYV'
	Name        : YUYV 4:2:2
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 2
	Type        : Video Capture
	Pixel Format: 'RGB3'
	Name        : 24-bit RGB 8-8-8
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 3
	Type        : Video Capture
	Pixel Format: 'JPEG' (compressed)
	Name        : JFIF JPEG
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 4
	Type        : Video Capture
	Pixel Format: 'H264' (compressed)
	Name        : H.264
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 5
	Type        : Video Capture
	Pixel Format: 'MJPG' (compressed)
	Name        : Motion-JPEG
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 6
	Type        : Video Capture
	Pixel Format: 'YVYU'
	Name        : YVYU 4:2:2
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 7
	Type        : Video Capture
	Pixel Format: 'VYUY'
	Name        : VYUY 4:2:2
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 8
	Type        : Video Capture
	Pixel Format: 'UYVY'
	Name        : UYVY 4:2:2
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 9
	Type        : Video Capture
	Pixel Format: 'NV12'
	Name        : Y/CbCr 4:2:0
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 10
	Type        : Video Capture
	Pixel Format: 'BGR3'
	Name        : 24-bit BGR 8-8-8
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 11
	Type        : Video Capture
	Pixel Format: 'YV12'
	Name        : Planar YVU 4:2:0
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 12
	Type        : Video Capture
	Pixel Format: 'NV21'
	Name        : Y/CrCb 4:2:0
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 13
	Type        : Video Capture
	Pixel Format: 'BGR4'
	Name        : 32-bit BGRA/X 8-8-8-8
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

The /dev/video1 will be the used by gstreamer to send the H264 to QGC, so it’s possible to use video0 and video2 simultaneously, take a look in this python example.

import numpy as np
import cv2

cap_usb = cv2.VideoCapture(0)
cap_rpy = cv2.VideoCapture(2)

for i in range(7):
    print(i)
    # Capture frame-by-frame
    _, frame_usb = cap_usb.read()
    _, frame_rpy = cap_rpy.read()
    # Save the frame
    cv2.imwrite('/tmp/frame_usb_{0}.png'.format(i), frame_usb)
    cv2.imwrite('/tmp/frame_rpy_{0}.png'.format(i), frame_rpy)

# When everything done, release the capture
cap_usb.release()
cap_rpy.release()
cv2.destroyAllWindows()

It’s no different from a normal camera capture.

[ggttkl (gtl)]

Hi，
Thank you for replying. I came across a situation that I cannot cd to /dev/video. What I can find under the path /dev/ are block/ bus/ char / disk/ fd/ input/ mapper/ mqueue /net /pts/ raw/ serial/ shm/ snd/ . And I cannot use command v4l2 -ctl with -bash: v4l2 command not found.
My Rpi has been updated to 0.0.15. And my CSI camera works well on QGC because I can still recieve the video. Is there any other packages or libs should I install?
EDIT: When I use my usb camera. I came across the same situation. I found that “starting device /dev/video1 with width 1980 height 1080 framerare 30 options ! h264 parse ! queue! rtph264pay config -interval =10 pt=96 ! udpsink host =192.168.2.1 port =5600” when booting Rpi.
EDIT 2nd: v4l2-ctl works. I typed with format error previously…

[patrickelectric (Patrick José Pereira)]

@ggttkl

I came across a situation that I cannot cd to /dev/video. What I can find under the path /dev/ are block/ bus/ char / disk/ fd/ input/ mapper/ mqueue /net /pts/ raw/ serial/ shm/ snd/ .

There’s no /dev/video folder, only /dev folder with video* files, where * will be 0, 1, 2. To check for this, you can run $ ls /dev/video*, it will list all video devices.

@ggttkl

And I cannot use command v4l2 -ctl with -bash: v4l2 command not found.

It’s v4l2-ctl and not v4l2 -ctl with a space before the -.

[ggttkl (gtl)]

Thank you for your help. I have installed opencv on my Rpi and run the example code you supplied successfully. I will try more image processing in the following days! Thank you!

[ruipedro (Rui Alves)]

Hi. I came across this thread and I’m having the same same problem, where I can’t connect to my Rpi camera.
My companion system has the most recent firmware, and I have single camera setup. I already tried accessing the camera by your example (I’m using C++ opencv) but gstreamer blocks the access. I’ve tried to connect to gstreamer, but I’m not receiving any video (but QGC is). I’ve been on this for a few days and can’t find a solution.

When I do ls /dev/video* I only have /dev/video0

[AKJ]

Hey @patrickelectric please see my query and give me step by step process regarding this.

Turning BlueROV2 into Autonomous one

Hi all, I am from India. Currently, we are operating BlueROV2 in XBOX controller. We want to operate it autonomously, like locomotion of bot in given waypoints avoiding the obstacles. So, please give me a step by step procedure to configure my vehicle into autonomous one.