Communication between FlightController and BlueOS Extension

Hey everyone!

I’m currently working on a project where I need to establish communication between a flight controller and my BlueOS extension. The goal is to create a system where the flight controller and the extension work hand-in-hand during the rover’s waypoint navigation.

Here’s the flow I’m aiming for:

  1. The flight controller navigates along a predefined path (created in ArduPilot).
  2. When the rover reaches a waypoint, the flight controller signals the BlueOS extension to pause.
  3. The rover waits while the extension executes specific tasks.
  4. Once the extension finishes, it sends a signal back to the flight controller, allowing the rover to resume its journey to the next waypoint.
  5. This process repeats at every waypoint.

After some research, I found that MAVLink2Rest could be a promising way to implement this two-way communication between the extension and the flight controller.


Is MAVLink2Rest the best tool for this kind of two-way communication, or are there better methods to implement this functionality? I’d love to hear about alternative approaches or any insights on the best way to achieve this.


Any insights, suggestions, or examples would be incredibly appreciated! :raised_hands:
Thanks so much for your time and help!

Hi @niklasebner -
Checkout this thread - I think a LUA script might be better suited to your application. You could create an extension that the LUA extension reaches via HTTP request, to trigger whatever functionality you need - and the LUA script itself makes controlling GPIO and general vehicle functionality much easier!

Both that winch lua script, and the lua script in the top level of this repository control a vehicle. The HAUV.lua is a work in progress, but shows how you can use an http request to trigger video recording in an extension running on BlueOS! I’ve got some issues in the Dockerfile at the moment, but it has passed an initial field test - more to come with a write up eventually!

Mavlink2Rest technically does allow for control and not just receipt of telemetry, but we don’t have much documentation on the POST requests to do that yet…

I’d be curious what task you’d like the vehicle to perform!

Hey @tony-white!

Thank you so much for your response and for pointing me to your thread — it’s incredibly insightful and aligns with many of the things we’re aiming to achieve. It’s great to see that we’re not the only ones who need functionality like this!


The vehicle we’re working on is a weeding robot. I know it sounds a bit unconventional, especially since BlueOS wasn’t designed with land rovers in mind, but we wanted to experiment with it and believe it can be adapted to fit our needs.

Current workflow goals:

  1. Drive forward approximately one meter and come to a full stop.
  2. While stopped, the robot will capture an image, which is sent to our server for processing.
  3. The server runs an AI model to detect weeds and pinpoint their locations.
  4. Once the server completes the processing, the robot should resume driving to the next waypoint, repeating this cycle.

In the future, we plan to integrate an XYZ table to zap weeds with high voltage based on the processed images, but for now, we’re focused on getting the core workflow up and running.


What’s working so far:

  • The server is fully set up and functional.
  • The BlueOS extension can successfully capture images and send them to the server, then receive the processed coordinates.

What we’re currently working on:

  • Establishing communication between the flight controller and the BlueOS extension to implement the “pause, process, and resume” workflow.

Thoughts on your suggested approach:

  • The idea of using a LUA script makes sense, especially for managing the vehicle’s control flow, but we do have a couple of considerations:

    1. We’re currently using Mission Planner instead of QGC. From my research, I believe it’s still possible to integrate LUA scripts, though there might be minor differences.
    2. Our team doesn’t have prior experience with LUA scripting, and the existing extension is entirely written in Python. So, we’ll need to ensure that the LUA script can effectively communicate with the Python-based extension — likely through HTTP requests, as you suggested.

A couple of quick questions:

  1. Do you still think using LUA scripts is the best approach for this setup? Or would there be another method you’d recommend?
  2. Is there a way to connect BlueOS to a VPN? This might be something we need down the line for data transmission.
  3. If you have any other ideas or suggestions, we’d be incredibly grateful!

Thanks again for your time and help — your insights are invaluable! :slight_smile:

Hi @niklasebner -
Happy to help! What a cool application!
BlueOS is definitely not marine vehicle specific, nor is the Navigator Autopilot. We’ve seen folks make both ground vehicles and quadcopters with it - it’s just meant to enable robotics!

I do think Lua scripts are the best approach for your setup. You would plan the mission in Mission Planner (although we generally find that software a real pain!) or QGC, and then the LUA script would take care of pausing and communicating with your extension. The Winch.lua script illustrates how to take action at waypoints, and the HAUV.lua has an example of the http request from a lua script. I’m still learning LUA myself, but it’s quite easy with LLM AI and some resources like this and this more comprehensive one, and this list of supported functions.

What flight controller are you using? I believe BlueOS requires the Navigator for LUA scripts - you simply enable the scripting parameter, and put them in the right folder as detailed in the previous winch on BlueBoat post. The scripts run on boot of the autopilot, so I test with SITL and restarting the autopilot until no errors come from the script.

As for VPN, we highly recommend ZeroTier - it has an extension that makes accessing BlueOS and controlling your vehicle from anywhere in the world super easy! Some docs on setup are available here.

Hey @tony-white!

Thank you so much for your help! :raised_hands:
Using BlueOS has been a great experience overall — aside from a few things here and there, it’s been working really well. It’s such a cool software with a ton of potential! While there might not be a vast amount of information available online for more specific questions, the forum and the incredible support from you and the community have been invaluable. I really appreciate it!


Flight Controller & Setup:

We’re using the Kakute F4 flight controller. We’ve been working with Mission Planner mainly because we’re more familiar with it, but we definitely get why it can be a pain at times.


Setting up the communication:

I’d really appreciate it if you could give me a short walkthrough of how you would approach setting up communication between the flight controller and the BlueOS extension in our case.

The extension so far is fairly simple — it has a small user interface and a Python script that takes pictures, sends them to the server, and retrieves the coordinates.

I’m just not entirely sure where to start. Should the flight controller be the one directly sending the HTTP requests to the extension? Can I handle everything on the extension side using only a Python script , or will I need to implement a LUA script as well? And when it comes to setting up the HTTP endpoint on the extension, what is the best way to do that?

I’m mainly trying to figure out how to tie everything together, especially the communication flow between the LUA script on the flight controller and the Python-based extension. Since my colleague is handling the flight controller and ArduPilot setup, while I’m focused on the BlueOS extension and the server side, my knowledge about flight controllers, ArduPilot, and Mission Planner is still a bit limited — so any guidance would be super helpful!


About VPN & ZeroTier:

Thank you for suggesting ZeroTier — it looks like an awesome solution, and I’ll definitely explore it further. That said, we might still need a more traditional VPN setup in parallel since our server is currently running on a virtual machine within a local network, and I need to be connected to that network to access it. I’ve tried a few methods to set it up via the terminal but haven’t been able to establish a connection yet.


Once again, thank you so much for all your help — I genuinely believe our project wouldn’t be where it is today without your guidance. Quick sidenote: you’ll definitely get a mention in the “Danksagung” (acknowledgments) section of our thesis! :wink:

Hi @niklasebner -
I would recommend against using the Kakute F4 flight controller. We’ve not tested it with ArduSub or ArduRover, and when used with BlueOS it won’t support LUA scripts. That said, you could get things working without lua scripts using Pymavlink, but that is a less elegant solution in my opinion.

LUA scripts are convenient because they are run by the autopilot, but don’t require expanding the autopilot firmware with more functionality. The specific code used in my HAUV script to send an HTTP request to my extension is

function start_video_recording()
    local sock = Socket(0)
    if not sock:bind("0.0.0.0", 9988) then
        gcs:send_text(6, "Failed to bind socket")
        sock:close()
        return false
    end

    if not sock:connect(HTTP_HOST, HTTP_PORT) then
        gcs:send_text(6, string.format("Failed to connect to %s:%d", HTTP_HOST, HTTP_PORT))
        sock:close()
        return false
    end

    local request = "GET /start?split_duration=2 HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n"
    gcs:send_text(6, string.format("Sending request to http://%s:%d/start?split_duration=2", HTTP_HOST, HTTP_PORT))
    sock:send(request, string.len(request))
    sock:close()
    gcs:send_text(6, "Video recording started")
    return true
end

This is received by this code in the extension:

@app.route('/start', methods=['GET'])
def start():
    global process, recording, start_time
    try:
        if recording:
            return jsonify({"success": False, "message": "Already recording"}), 400
            
        # Ensure the video directory exists
        os.makedirs("/app/videorecordings", exist_ok=True)
            
        timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
        filename = f"video_{timestamp}.mp4"
        filepath = os.path.join("/app/videorecordings", filename)
        
        pipeline = ("v4l2src device=/dev/video2 ! "
            "video/x-h264,width=1920,height=1080,framerate=30/1 ! "
            f"h264parse ! mp4mux ! filesink location={filepath}")

        command = ["gst-launch-1.0", "-e"] + shlex.split(pipeline)

        process = subprocess.Popen(command,
                           stdout=subprocess.PIPE,
                           stderr=subprocess.PIPE)
        
        logger.info(f"Starting recording with command: {' '.join(command)}")
        
        if process.poll() is not None:
            stdout, stderr = process.communicate()
            logger.error(f"Process failed to start. stdout: {stdout.decode()}, stderr: {stderr.decode()}")
            raise Exception(f"Failed to start recording: {stderr.decode()}")
            
        recording = True
        start_time = datetime.now()
        
        return jsonify({"success": True})
    except Exception as e:
        logger.error(f"Error in start endpoint: {str(e)}")
        recording = False
        start_time = None
        if process:
            try:
                process.kill()
            except:
                pass
        process = None
        return jsonify({"success": False, "message": str(e)}), 500

If you don’t want to switch to a Navigator, your extension would not only need to handle your image capture goals, but also communicate with the autopilot to change modes and arm/disarm the system. You can read about that here, but it has been primarily been tested with ArduSub, so some challenges may be lurking…

I’d also strongly recommend against a “traditional” VPN - they are very difficult to setup correctly, unreliable, and limited in bandwidth in most circumstances - and BlueRobotics has not tested other solutions for compatibility with BlueOS. ZeroTier is essentially a VPN, and if you need to access other devices on the network, you can do so. . Essentially, a ZeroTier device has a ZeroTier IP that canbe accessed from any device on the same virtual ethernet switch that is represented by your ZeroTier network.