Development Tools for Cockpit

Blue Robotics Software Cockpit
1

Hello all,

I’m getting ready to switch over from QGC to Cockpit.

I am extremely new to this UI and its development.

I have a need to implement my own ‘widgets’ into the system.
Some are relatively complicated, but there is very little UI needed.

I also need to have the ability to add UI pins and other artifacts onto the displayed
map. I’m not sure how that would be accomplished.

Are there any tools or ‘how to’ articles that describe adding completely new functionality
into the Cockpit UI?

Thanks for the help,

Mark Kozikowski

2

Hi Mark!

There are 3 ways to add your own widgets to Cockpit:

  1. Create it using the DIY widget: this is the most straightforward way, and helps you start without any extra tools needed besides Cockpit itself. You will write it in pure HTML+CSS+JS, inside a dedicated interface, so there’s no support for external IDEs and transpilling (e.g.: React, Polyfill, etc).
  2. Create a BlueOS extension that serves a widget for Cockpit: you will be creating an extension and serve a webpage from it. This webpage will be your widget, so you have full control over its development, including tools and libraries. You will have also a special metadata endpoint, that Cockpit uses to know that the extension serves a widget and where to look for. This is the recommended way, since you can easily share your widget on other systems by simply installing the BlueOS extension there.
  3. Add your widget to the main codebase: if your extension has “a general purpose”, not being too much “system specific”, and your’e ok with having your code public, you can contribute to the project with this new widget.

Another thing you can think about, instead of creating custom widgets, is to develop your functionality with Cockpit tools. We have plenty of tools to help you do that:

You can create a Javascript action to fetch and manipulate the data, store the values into data-lake variables (think about generic variables on Cockpit that the user can see and manipulate) and show them on VeryGenericIndicators (a widget specifically designed to show any kind of information from the data-lake). If you need the user to be able to control something, we have the Input widgets, that can be consumed from your Javascript action as well.

The Map is unfortunately a place were you still can’t add custom stuff, so if you need to do something around it, your best shot for now would be to create your own widget indeed.

Let me know which one option works best for you today (if any), and we can discuss it more deeply. If none work, let me know as well, so we can discuss the subject and think about new ways to do it.

1 Like
3

Rafael,

Thanks, this is good information.

I think solution #2 would be most appropriate for our situation. I’m going to need some
coaching on creating the BlueOS extension.

The issues with the map are relatively simple. What I would like to do is add ‘pin markers’
to the map at specific Lat/Lon locations.
I would also like to provide Mouse Hover actions to those pins that can display
text and graphic information.

For the first go-around, I can work around this functionality, but I will eventually need
to add this.

Another issue I have is video. I need to add visual cues to the video stream.
Highlights, boxes, etc.
I did this previously by intercepting the video before it entered QGC, added my
requirements to the video and then passed on to QGC.

But first things first. It would be great if you could show me a link for creating a BlueOS extension so I can get started.

Mark

4

Nice to hear that!

We can definitely go by steps and make one thing work at a time.

For more information on how to write a BlueOS extension, take a look at this page, it has everything you need, including a step by step guide at the end. For development, you probably want to go with Offline Building and Manual Install.

If you’re into using LLMs, I had pretty good one-shot results with Claude 3.7 Sonnet in thinking+agent mode. Just ask for a placeholder BlueOS extension and mention the URL of the documentation page. For the build, remember to generate images for the necessary platforms.

I have a Mock extension here if you want to take a look. It does not serve a widget, but once you get your custom extension running on BlueOS, you can follow this guide on how to serve a widget from the extension.

Let me know if you need any help.

1 Like
5

Hi @mark,

Nice! :smiley:

Unfortunately we don’t yet have developer documentation or custom tools to help with Cockpit development, and my priorities for documentation at the moment are on Cockpit usage rather than development that requires custom builds of the application, as well as various other software projects that are in need of docs updates.

There is some basic development setup information in the repository README, and there’s a Discord channel for development chat, which are likely both quite helpful :slight_smile:

Rafael has already covered a bunch of this, but to add some clicky stuff and additional context:

The best way to start this is using the DIY widget, or by developing a BlueOS Extension to run some backend functionality on the vehicle and provide a widget interface to Cockpit through its automatic external iframes mechanism.

If you make a widget that you then want to contribute / integrate into Cockpit itself then we should be able to provide more directed advice on packaging that together, potentially forming the start of our development docs :sweat_smile:

It’s perhaps worth noting that the documentation I am actively working on should go into some depth on Cockpit’s API (how to make custom Actions and the like), as well as making use of its Data Lake system to use and share data around the application, and store it persistently where that’s relevant. I’m hoping to get that reviewed and release it late next week, or early the week after.

Custom map layers and the like have been considered since the start of the project, but haven’t yet been implemented in an accessible way. There’s also been some more directed discussion of the capacities of interaction, including adding landmarks and the like.

While that’s functionality you could potentially help contribute to the project, it’s also quite a fundamental feature of a control station software, so will require a fair amount of thought before a proposed implementation could be accepted. You’re very welcome to contribute your ideas and preferences to the discussion in those Issues (or a new sub-Issue), and/or in chat / forum posts as relevant :slight_smile:

This is another one that doesn’t have a strong implementation basis at this point. I’ve put some high level thoughts around a sustainable overlay system in this Issue, and some thoughts on annotation approaches in this forum thread, which you’re again welcome to contribute to :slight_smile:

As an overview, while editing video pixels can be a visually effective approach, the processing requirements, extra latency, and resulting lost frame data seem suboptimal and unsustainable when compared with just sending the annotation information separately and drawing it over the video dynamically. Having a convenient mechanism for that built into Cockpit would be valuable, and with the right design that information could even be saved in subtitle files that could be played back over the video (like is already done with telemetry data), and turned off when it’s preferable to view/analyse just the video frames :smiley:

1 Like