Forwarding joystick commands to external controller

Blue Robotics Software ArduSub
1

Hello

I am using the BlueROV2 Heavy through QGC, RBP and the pixhawk, and want to add modes and controls for a robotic arm.

Currently, the arm controls are in an external controller (MyRio), but I want it to be controllable in QGC alongside with the BlueRov2. The plan is to add 3 modes (or modify exististing unused modes), that all stabilize the ROV, while one controls the robot arm with a joystick.

The pixhawk/RBP then sends a message to the MyRio about which robot arm mode is active.
The pixhawk/RBP then forwards all raw joystick button presses to the MyRio, where I can program processing of joystick controls for the robot arm in an environment, where I am more savvy.

In short I cant find out how I can send raw joystick commands out of the Pixhawk or (preferrably) the Raspberry Pi’s GPIO.

BR, Michael Hermansen

2

The communication is done with MAVLink. You can see examples of pulling data out of the stream here, as well as searching the forum. You will be interested in the MANUAL_CONTROL message. You can write some python code to sniff that message out of the stream.

3

Hey Jacob

Thanks for the link, it was helpful. How do I access the variables in MANUAL_CONTROL?

Is it like this (with a dot after manual_control):

# Import mavutil
from pyvmalink import mavutil

# Create the connection
master = mavutil.mavlink_connection('udp:0.0.0.0:14550')
# Wait a haertbeat before sending commands
master.wait_heartbeat()

int16_t x_axis
int16_t y_axis
int16_t z_axis
int16_t r_axis
uint16_t js_buttons

while True:
	msg = master.recv_match()
	if not msg:
		continue
	if msg.get_type() == 'MANUAL_CONTROL'
		x_axis = MANUAL_CONTROL.x
		y_axis = MANUAL_CONTROL.y
		z_axis = MANUAL_CONTROL.z
		r_axis = MANUAL_CONTROL.r
		js_buttons = MANUAL_CONTROL.buttons
		# gpio commands (spi or i2c) that sends variables to other controller

Also, can I access the flight modes as well? From the documentation you sent, they seem to be stored in MAV_MODE_FLAG, but when I “watch MAV_MODE_FLAG” in MAVProxy, nothing is watched.

4

You will need to substitute MANUAL_CONTROL.x with msg.x and so on.

The mode comes in the HEARTBEAT message. There is no MAV_MODE_FLAG message; MAV_MODE_FLAG is an enumeration.

5

Hello Jacob

I got around to doing this. The RBP is running a python program, where I open a connection and write serial messages to my MyRio whenever the RBP receives HEARTBEAT, MANUAL_CONTROL and SET_MODE. I found out the flight mode is stored in SET_MODE, not in MANUAL_CONTROL

When I receiver HEARTBEAT, I also send out a heartbeat back from the RBP.

However, I am never receing any MANUAL_CONTROL or SET_MODE messages. And Ive gotten confused about which connection to use. The originally suggested one, udp:0.0.0.0:14550 disables when I open QGC, so Ive tried

udpout:0.0.0.0:9000, which gives me alot of messages, but not the ones I am looking for
udpout:localhost:9000 same
udpout:192.168.2.255:14550 which gives nothing
udpout:192.168.2.1:14400, source_system=2, source_component=1, which gives only heartbeats - I took this connection from Sending Data from Raspberry Pi to QgroundControl - #21 by jwalser

I would appreciate any advice you could give on trying to pick out joystick commands and flight in the Raspberry Pi.

The code is given below

# Import mavutil
from pymavlink import mavutil
import time
import serial

# Create the connection
master = mavutil.mavlink_connection('udp:0.0.0.0:14550')
# Wait a heartbeat before sending commands
master.wait_heartbeat()

while True:
	msg = master.recv_match()
	if not msg:
		continue
	if msg.get_type() == 'MANUAL_CONTROL'
		x_axis = msg.x
		y_axis = msg.y
		z_axis = msg.z
		r_axis = msg.r
		js_buttons = msg.buttons
		ser = serial.Serial('/dev/ttyS0', 9600)
		ser.write(str(x_axis))
		ser.write(",")
		ser.write(str(y_axis))
		ser.write(",")
		ser.write(str(z_axis))
		ser.write(",")
		ser.write(str(r_axis))
		ser.write(",")
		ser.write(str(js_buttons))
		ser.write(",\n")
	if msg.get_type() == 'HEARTBEAT'
		current_mode = msg.base_mode
		ser = serial.Serial('/dev/ttyS0', 9600)
		ser.write("0,0,0,0,0,")
		ser.write(str(current_mode))
		ser.write(",\n")

BR, Michael

6

Please provide a diagram describing the system you want and what programs are running where.

7

Is attached. The ROV-thrusters and barometers are not connected, and neither is the robot arm.

The idea is that I want the MyRio below the surface with the Raspberry Pi. I am then trying to pick up joystick commands in the Raspberry Pi and send it to the MyRio.

ConnectionProgramsDiagram
ConnectionProgramsDiagram815×424 9.36 KB

8

Is what Im doing not possible? Do I need to write a python script on my computer instead of using QGC?

9

Hi,

If you want to use QGC in parallel with your python script, please follow our documentation here.

#  If you want to use QGroundControl in parallel with your python script,
#  it's possible to add a new output port in http:192.168.2.2:2770/mavproxy as a new line.
#  E.g: --out udpbcast:192.168.2.255:yourport

This messages are sended by QGC for joystick command (MANUAL_CONTROL) or to change the flight mode (SET_MODE).

If you want to change the flight mode and get the one used by the ROV, check the Change flight mode example. And, take a look in Send RC (Joystick) example for joystick commands.

1 Like
10

You need to configure mavproxy to direct mavlink traffic to your desired endpoint with the --out option. Our companion image is configured to send to localhost:9000 (port 9000 on the pi) and 192.168.2.1:14550 (port 14550 on remote host, this port is not currently in use on the local pi host. This is why you do not see any mavlink with your script running on the pi).

11

You need to configure mavproxy to direct mavlink traffic to your desired endpoint with the --out option

Thank you, I have done this now.

This messages are sended by QGC for joystick command (MANUAL_CONTROL) or to change the flight mode (SET_MODE).

If you want to change the flight mode and get the one used by the ROV, check the Change flight mode example. And, take a look in Send RC (Joystick) example for joystick commands.

I thinnk I have not expressed clearly what I want. I dont wanna send joystick and flight mode commands given in my python script to the MyRio. I want to receive these commands from my gamepad joystick going into my PC, read them in my python script and then send it to the MyRio.
I have already looked at those scripts, but the problem is that I cannot detect any message that has the commands I want. I have attached a new diagram to clarify!

SystemDiagram
SystemDiagram798×564 32.4 KB

The Command ack doesnt have anything I need, it just detects whether there is a change in flight modes. I can detect this, but it wont tell me what mode is being activated. The parameter request also doesnt have parameters that describe the messages sent by the gamepad joystick.

Is it not possible to get what I need in this way?

I used this code:
ser = serial.Serial(‘/dev/ttyS0’, 9600)

def wait_conn(master):
	msg = None
	while not msg:
		master.mav.ping_send(
			time.time(),
			0,
			0,
			0
		)
		msg = master.recv_match()
		time.sleep(0.5)

master = mavutil.mavlink_connection('udpin:192.168.2.255:10004') #the port I added to the --out in MAVPROxy

print 'Waiting for heartbeat\n'
wait_conn(master)
print 'Heartbeat!\n'

mode = 'STABILIZE'

if mode not in master.mode_mapping():
	print('Unknown mode : {}'.format(mode))
	print('Try:', list(master.mode_mapping().keys()))
	exit(1)

ack = False
while not ack:
	ack_msg = master.recv.match(type='COMMAND_ACK', blocking=True)
	ack_msg = ack_msg.to_dict()
	if ack_msg['command'] != mavutil.mavlink.MAVLINK_MSG_ID_SET_MODE:
		continue
	print ack_msg
	print(mavutil.mavlink.enums['MAV_RESULT'][ack_msg['result']].description)
	break

while True:
	msg = master.recv_match()
	if not msg:
		continue
	print(msg.get_type())
	time.sleep(0.1)
	if msg.get_type() == 'HEARTBEAT':
		ser.write("0,0,0,0,0,0,\n")
		print msg
	if msg.get_type() == 'SET_MODE':
		print msg
	if msg.get_type() == 'MANUAL_CONTROL':
		print msg
12

Hi Michael, sorry we have been busy. Have you gotten any farther with this?

13

Hi Jacob

Not not really, I have also been busy. I have been considering alternative approaches.
Do you know if my current approach is possible? Can I pull out the MANUAL_CONTROL and SET_MODE messages from the Pi somehow?

BR