Camera control with joystick

I was thinking of trying the Cockpit extension because it seemed promising.
And I wanted to set up the joystick controls.
However, I couldn’t operate the cameras.
I have one normal camera and another IP zoom camera.

  1. How can I switch between the cameras?
  2. What should I do to enable zoom control?
    For the zoom camera, I use NODE-RED to receive joystick signals via MAVLink and control it over HTTP.

  1. By switching you mean changing which camera are you watching? In that case I would add two video widgets side by side in the same view, or create two views, each one with a video widget.

But we don’t have support for IP Cameras on MavlinkCameraManager yet. Were you able to connect your IP camera to cockpit? I wonder if you could do that through the ImageViewer widget using a MJPEG url or through the UrlVideo widget.

  1. Which mavlink messages are you using to control the camera? Your idea is to send those messages from cockpit?

Hi @rafael.lehmkuhl

  1. I can use IP Camera to cockpit.

Are there instructions anywhere on how to use the video widget?
If not, it would be great if you could tell me how to use it.
What it can do is not clear to me.

The best pattern I can think of right now:

  • Normally, the sub-camera (8mm lens) is displayed in a small area to the lower left of the main camera.
  • Use the joystick to swap cameras.

2.The following is now available

I implemented it because it was easiest to control via Node-RED and HTTP.

The easiest way to modify it is to use Node-RED to receive the cockpit joystick input.
If there is an easier way to control the PTZ of the IP camera, that would be fine.

As an aside, the camera performance is about this different.
The IP camera is on the left and the standard camera is on the right.

Test in a dark room.
Left: IP camera with standard settings
right: standard camera with maximum gain

The IP camera is designed to be easily replaceable, and mounts and connectors are also made.

Cool setup!
You’re the first non-internal user to use Cockpit in a multiple camera setup as far as I’m aware, so congratulations haha

We have some good documentation on how to edit Cockpit’s interface here: Advanced Usage | ArduSub. Let me know if it helps. It’s not completely updated, as we added more functions in the last month, but I think from the docs the rest is more or less straightforward.

Regarding the screen setup, my suggestion would be for you to create a profile with two views, the first with the main camera fullscreened and the secondary on the bottom left corner, and the second view being the same, but with the video widgets reversed. You can configure each video widget by clicking the cog button within the edit-menu.

For changing between views, you can setup one of your joystick buttons to perform a “Next View” cockpit action.

Let me know if that works.


I can do it!
The cockpit extension is great!!
(I would like faster switching of view mode.)

Now if I could zoom with the joystick it would be perfect.

Two standard cameras with different lenses, it looks like this.


Thank you @rafael.lehmkuhl,
But I managed to do it before reading the manual.
The user interface is great!

Now, could you please tell me how to zoom the IP camera with a joystick?

So cool! Thanks for the feedback!

About the slow toggling view, thanks for letting us know. I found the problem and fixed it. Will release a new version in the next hours so you can test.

I have some questions here.

Which are the mavlink MANUAL_CONTROL button options that you are using to control the zoom?

My suggestion would be for you to assign “Custom 1”, “Custom 2” and “Custom 3” to buttons on your joystick (you can do that from Cockpit itself) and then read for those in NodeRed.


Just to be sure, how exactly is the HTTP request that you do to perform the zoom control?

We already had the idea of allowing users to setup HTTP requests directly from cockpit, but didn’t implement it because we didn’t have a clear use-case. If you can detail yours (e.g.: it’s a simple GET request to a specific URL, or it’s a POST request with some JSON body, etc), I can work on that. It would remove the need for a man-in-the-middle (node red in that case).

When I tried it with QGC, I could only detect whether the joystick buttons were pressed. So, I assigned it to Shift + Up or Shift + Down buttons.
I tried to assign it to Custom1-3, but I couldn’t.

If possible, could you create a sample JSON for Node-RED to receive Custom1, please?

I’m not that familiar with node-red, but I will take a look.

One idea would be for you to assign the custom1-3 and after that do a simple node-red that just displays the value of MANUAL_CONTROL.buttons. With that you can click the 3 buttons on your joystick separately to check which actions are being performed.

Remember the buttons value is a bit array, so if Custom1 is 2048 and Custom2 is 128, when they are pressed at the same time you will read 2176.


admin login

POST request with some JSON body

My code is messy, and I’m embarrassed to share it, so I’ll send you the Node-RED JSON in a message.

Thank you!
I try it.

In Cockpit joystick, MANUAL_CONTROL had no value.

Remember you need the Cockpit tab to be in focus so it can send the joystick buttons through the ManualControl message.

@Itaru forgot to mention but just to let you know, I just released v0.11.1, which makes toggling views instantaneous (there was an unintended 500ms delay).

1 Like

Hi @rafael.lehmkuhl

Thanks for everything.
I was able to do something like this!

There are so many choices for VeryGenericIndicator,
I can’t find HeartBeat temp.
I would appreciate it if you could tell me.

Others are as follows:

  1. i need a setting to change the color of the icon.
    QGC adds the following to warn me.

  2. WIFI off and on button
    sudo iwconfig wlan0 txpower off
    sudo iwconfig wlan0 txpower auto
    *Other functions that can be turned off to stabilize operation are desired to be turned off.

  3. Complete restart button
    Since becoming navigator, BlueROV2 has been unstable and I want a reboot.
    Restarting from BlueOS does not work like before with the thruster beep sound.

4.with zero tier extension
QGC can be operated remotely, but starting Cockpit from BlueOS results in an error.

Hey Itaru!
About the alerts on the VeryGenericIndicator, it’s on our plans already but not for now. We will probably work on it in the end of this year or beginning of the next.

About the Wifi on and off, I’m not planning to put any system integration directly on Cockpit, but you would be able to configure that using a Cockpit Action that makes an HTTP request to BlueOS, which has this functionality exposed on an API already.

About the restart problem, I would suggest you to open an issue on the BlueOS repository on GitHub or in the BlueOS discuss section. @EliotBR will be able to help you with that.

About the Cockpit error when using zerotier, can you give me more details? A video recording would probably help.

Hey @rafael.lehmkuhl !

Looking forward to the update!

Personally, I think reboot and WIFI off is necessary by default for the unstable ROV.
I think I can set it up, but many people can’t.
The UI should be easier for everyone to use in my opinion.

About zerotier, I tried again and was able to do it.
Last time I tried it twice from different networks and it didn’t even show up with an error.
This time it was quite easy to connect.

There was almost no delay.
I am happy to be able to do what I want to do!

But there is a problem when I think about actually doing it.
The remote loses operation when the network goes down.
So I tried to see if I could do a dual pilot, and it looks like I can!
(I tried it locally and was able to do it, so I guess I could do the remote as well if I used a regular VPN.)

The cockpit is very nice.