Running BlueROV2 in autonomous mode by coding on Rpi

We are doing a a project to change the recently purchased BlueROV2 to autonomous with image capture followed processing and then according navigation.
At present, we are able to control Blue ROV2 through groundcontrol joystick and additionally we are able command ROV through mavproxy(arm throttle, rc 3 1600 etc. ) on Rpi.
Now we are planning to write python script in Rpi attached to pixhawk (ardusub) through USB. Can you send some command lines to move Blue ROV2 (Arming/disarming, forward, reverse, lateral right/left, depth hold mode etc.) in python language. We would like to run this python script on Rpi bootup.

Any help would be amazing! Thank you

Hi @sjpanwar,

Please, take a look in the forum, probably you will be able to find out a lot of related posts.

And try to wait a bit for a reply, you answer was already provided in your first request:

Hello, I am doing the project to autonomous ROV with image capture followed processing and then according navigation, too. And now I can send some basic commands to Ardusub(pixhawk) directly on Rpi. However, the image information is transmitted to QGC through Rpi,and I cannot find any instructions to read or process the image information directly on Rpi board.
I have both the Rpi camera and HD USB camera and I want to read both of the cameras data on Rpi board. Can they both on work at the same time?
Thank you for any possible help!

Hi,

You can check the cameras under the path /dev/video*, running the v4l-ctl (application to control video4linux drivers) you’ll have all the information needed to perform your task.

First, try to list all devices:

pi@raspberrypi:~ $ v4l2-ctl  --list-devices
mmal service 16.1 (platform:bcm2835-v4l2):
	/dev/video2

H264 USB Camera (usb-3f980000.usb-1.2):
	/dev/video0
	/dev/video1

To know the format of each video:

pi@raspberrypi:~ $ v4l2-ctl --list-formats-ext --device=/dev/video0
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'MJPG' (compressed)
	Name        : Motion-JPEG
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 800x600
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 640x360
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 352x288
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)

	Index       : 1
	Type        : Video Capture
	Pixel Format: 'YUYV'
	Name        : YUYV 4:2:2
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 800x600
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 640x360
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 352x288
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)

pi@raspberrypi:~ $ v4l2-ctl --list-formats-ext --device=/dev/video1
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'H264' (compressed)
	Name        : H.264
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 800x600
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 640x360
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 352x288
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)

pi@raspberrypi:~ $ v4l2-ctl --list-formats-ext --device=/dev/video2
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'YU12'
	Name        : Planar YUV 4:2:0
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 1
	Type        : Video Capture
	Pixel Format: 'YUYV'
	Name        : YUYV 4:2:2
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 2
	Type        : Video Capture
	Pixel Format: 'RGB3'
	Name        : 24-bit RGB 8-8-8
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 3
	Type        : Video Capture
	Pixel Format: 'JPEG' (compressed)
	Name        : JFIF JPEG
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 4
	Type        : Video Capture
	Pixel Format: 'H264' (compressed)
	Name        : H.264
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 5
	Type        : Video Capture
	Pixel Format: 'MJPG' (compressed)
	Name        : Motion-JPEG
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 6
	Type        : Video Capture
	Pixel Format: 'YVYU'
	Name        : YVYU 4:2:2
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 7
	Type        : Video Capture
	Pixel Format: 'VYUY'
	Name        : VYUY 4:2:2
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 8
	Type        : Video Capture
	Pixel Format: 'UYVY'
	Name        : UYVY 4:2:2
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 9
	Type        : Video Capture
	Pixel Format: 'NV12'
	Name        : Y/CbCr 4:2:0
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 10
	Type        : Video Capture
	Pixel Format: 'BGR3'
	Name        : 24-bit BGR 8-8-8
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 11
	Type        : Video Capture
	Pixel Format: 'YV12'
	Name        : Planar YVU 4:2:0
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 12
	Type        : Video Capture
	Pixel Format: 'NV21'
	Name        : Y/CrCb 4:2:0
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

	Index       : 13
	Type        : Video Capture
	Pixel Format: 'BGR4'
	Name        : 32-bit BGRA/X 8-8-8-8
		Size: Stepwise 32x32 - 2592x1944 with step 2/2

The /dev/video1 will be the used by gstreamer to send the H264 to QGC, so it’s possible to use video0 and video2 simultaneously, take a look in this python example.

import numpy as np
import cv2

cap_usb = cv2.VideoCapture(0)
cap_rpy = cv2.VideoCapture(2)

for i in range(7):
    print(i)
    # Capture frame-by-frame
    _, frame_usb = cap_usb.read()
    _, frame_rpy = cap_rpy.read()
    # Save the frame
    cv2.imwrite('/tmp/frame_usb_{0}.png'.format(i), frame_usb)
    cv2.imwrite('/tmp/frame_rpy_{0}.png'.format(i), frame_rpy)

# When everything done, release the capture
cap_usb.release()
cap_rpy.release()
cv2.destroyAllWindows()

It’s no different from a normal camera capture.

Hi,
Thank you for replying. I came across a situation that I cannot cd to /dev/video. What I can find under the path /dev/ are block/ bus/ char / disk/ fd/ input/ mapper/ mqueue /net /pts/ raw/ serial/ shm/ snd/ . And I cannot use command v4l2 -ctl with -bash: v4l2 command not found.
My Rpi has been updated to 0.0.15. And my CSI camera works well on QGC because I can still recieve the video. Is there any other packages or libs should I install?
EDIT: When I use my usb camera. I came across the same situation. I found that “starting device /dev/video1 with width 1980 height 1080 framerare 30 options ! h264 parse ! queue! rtph264pay config -interval =10 pt=96 ! udpsink host =192.168.2.1 port =5600” when booting Rpi.
EDIT 2nd: v4l2-ctl works. I typed with format error previously…

There’s no /dev/video folder, only /dev folder with video* files, where * will be 0, 1, 2. To check for this, you can run $ ls /dev/video*, it will list all video devices.

It’s v4l2-ctl and not v4l2 -ctl with a space before the -.

Thank you for your help. I have installed opencv on my Rpi and run the example code you supplied successfully. I will try more image processing in the following days! Thank you!

Hi. I came across this thread and I’m having the same same problem, where I can’t connect to my Rpi camera.
My companion system has the most recent firmware, and I have single camera setup. I already tried accessing the camera by your example (I’m using C++ opencv) but gstreamer blocks the access. I’ve tried to connect to gstreamer, but I’m not receiving any video (but QGC is). I’ve been on this for a few days and can’t find a solution.

When I do ls /dev/video* I only have /dev/video0

Hey @patrickelectric please see my query and give me step by step process regarding this.