I’m trying to deploy YOLOv11 (NCNN lightweight version) on a Blue Robotics Navigator stack running BlueOS on a Raspberry Pi, and I’ve hit a wall with installation errors. I’m hoping someone here has experience with computer vision models on BlueOS.
Hardware / Software Setup
Vehicle: Blue Robotics Navigator stack
Companion Computer: Raspberry Pi (Navigator-integrated)
OS: BlueOS
Access Method: BlueOS Jupyter Notebook
Goal: Run YOLOv11 lightweight (NCNN) for onboard object detection
What I’ve Tried
Attempted to install YOLOv11 via Jupyter Notebook inside BlueOS
Followed standard YOLO installation steps (pip installs / model setup)
Intended to use the NCNN lightweight version due to Pi compute limits
Issue Encountered
Installation repeatedly fails with PyTorch-related errors
Errors occur during dependency installation (Torch / TorchVision)
BlueOS does not seem to allow PyTorch to install correctly on this setup
I’m unable to get far enough to even convert/export the model to NCNN
What I Suspect
PyTorch may not be supported or installable in the BlueOS environment
My approach (installing YOLO directly via Jupyter in BlueOS) may be fundamentally flawed
There may be a better workflow
What I’m Looking For
Has anyone successfully run YOLO (any version) on BlueOS + Raspberry Pi?
Is PyTorch supported in BlueOS, or should it be avoided entirely?
Recommended workflow for NCNN-based object detection on Navigator
Whether this should be done via:
A BlueOS extension
Docker container
Offboard model conversion + onboard inference only
If helpful, I can post:
Full error logs
Python version
BlueOS version
Exact commands used
Thanks in advance, any guidance or examples would be greatly appreciated. I’m also not the most proficient when it comes to software so including “basic” information would be appreciated. Thanks!
Hi @Kevin12 -
Welcome to the forums!
You may not be running into any issues due to BlueOS, but general support for PyToch and Yolov11 on the Raspberry Pi 4’s ARM cpu? Simply using pip to install is likely the reason for your issues.
Even if you do get something running, it will be limited performance - don’t expect more than 1-5 FPS?
Your best bet is to install a 64 bit version of BlueOS, potentially on a Pi5, for a bit more performance. You’d probably benefit from getting your environment setup in an Extension, its own Docker container, rather than trying to piggyback on jupyter notebook’s? Your dockerfile would have all the install steps, and if it builds and image with the github action you’ll be off to the races once you install the extension image in BlueOS. You can use the terminal to get into this container, and continue development from there, or fully build out your application and test the whole “chain.”
For most applications using machine vision, it makes more sense to run your model on the topside computer, and send commands to the ROV based on the results - unless you’re working on a project that will eventually be untethered? Can you share more about your application and goals?
Thanks for the response, I really appreciate you taking the time to explain this. To be honest, a lot of what you mentioned (ARM CPUs, PyTorch support, etc.) is still pretty new to us, so we’re likely missing some fundamental understanding here.
This is a senior design project for a team of four Mechanical Engineering students at UW–Madison, and our coding background is fairly basic (mostly Python experience). Because of that, we’re trying to make sure we’re not going down a path that’s unrealistic for our skill level or timeline.
For more context on the project:
We’re modifying a Blackjack 42” 8S catamaran-style RC boat
The goal is to autonomously collect floating buoys (we’re using ping pong balls) in a pool environment
The collection method is intentionally simple: using the catamaran layout, we plan to mount a net in the bridgedeck clearance, and the boat will just drive forward and “scoop” the ball between the hulls
The vehicle can be manually driven to the general area, then switch to autonomous mode for detection and collection
We were hoping to use a forward-facing camera for object detection, then command the boat to drive straight toward the detected ball
We also plan to use the Navigator IMU mainly to help keep the boat pointed forward during the approach (not complex path planning)
Given all of this, we’re trying to understand a few things at a more fundamental level:
Is the Raspberry Pi 4 + Navigator stack even a realistic platform for running any form of onboard object detection for a student project like this?
Are we misunderstanding what’s feasible by trying to install YOLO (or PyTorch-based tools) directly on the Pi?
Would it make more sense for us to:
Run vision on a topside laptop and send simple commands to the boat, or
Use a very lightweight onboard approach (e.g., NCNN-only inference, or even simpler vision methods)?
From a “student project” perspective, what approach would you recommend?
We’re absolutely open to simplifying the vision approach if YOLO is overkill for detecting something as basic as a ping pong ball. Our priority is a robust, demonstrable system rather than advanced perception.
Any advice on whether this hardware/software stack makes sense at all, and how you’d recommend a student team approach this, would be hugely appreciated. Thanks again for your patience and help.
Hi @Kevin12 -
Thanks or the context.
While using pytorch and some YOLO model on a pi is possible, it may be simpler to develop that software on a completely different computer. If you’ll be in close-range, processing the video on the topside may be easiest?
If you use ArduRover on the Pi/Navigator, it already has Obstacle avoidance features that may be useful? You can probably start by loading the default BlueBoat parameters, since the catamaran probably handles similarly to a BlueBoat.
A boat setup with this system will automatically hold heading when in Acro mode. Auto mode lets it follow GPS waypoints. For your application, you’d likely want to command heading and simulated “pilot input” in Acro mode to navigate the vehicle based on the output of your machine vision software.
A dedicated machine vision camera could potentially simplify things, and give you a serial data stream that the autopilot could directly consume. I’m pretty sure people have used an openMV camera to control Ardupilot drones for position holding and target tracking applications before! You would need to stick it a waterproo housing - something from Polycase would work (IP68 or better) and you could use an appropriate WetLink Penetrator with the USB cable to get things connected. An adapter like this can be helpul when re-terminating cable after passing it through the bolt.
Thanks again for the detailed guidance, this is really helpful.
To clarify our setup: we already have the Blue Robotics ExploreHD USB camera, and we’ll be operating in close range in a pool, so running vision on a topside laptop is feasible for us. Our team’s software experience is fairly basic, so we’re trying to understand what the full end-to-end setup would look like in practice.
Based on your suggestion, running YOLO (or other vision processing) topside and then commanding the vehicle in Acro mode sounds like the right approach. Where we’re unsure is how to actually connect the pieces:
How is the ExploreHD camera video typically accessed topside through BlueOS?
What’s the recommended way to send commands back to the Navigator (e.g., MAVLink messages, simulated pilot input, heading/speed commands)?
At a high level, what does the control loop usually look like from detection → vehicle motion?
If possible, we’d really appreciate either:
A brief outline of the steps to get a basic “detect buoy → drive forward” behavior working, or
Pointers to documentation, examples, or repos we should study.
Thanks again for your help, we’ve been a little stuck on this part of the project and any detailed guidance or references would be hugely appreciated.
Hi @Kevin12 -
The exploreHD camera can be accessed as a UDP (default) or RTSP stream. You can install the DWE OS extension, and lower the bitrate for better performance over your limited bandwidth connection - typically 2-5 Mbit is good or a WiFi link. This information is in the guides linked on the product page!
I’d recommend using pymavlink to control the vehicle, but a lua script can also be used.
For your case, I’d imagine your machien vision software would recognize the ping pong ball, and give it an X coordinate relative to the center of the camera image. If the ball is to the left, you’d send a command to adjust the heading of the vehicle, iterating until the ball is in the center of the camera stream. You’d then drive forward? Your control loop only needs to focus on adjusting vehicle heading based on ball location.
I’d recommend chatting with a LLM AI to get things going, providing it detailed pymavlink and machine vision examples!