I’m losing heartbeats between my ROV and topside roughly once every 10 minutes, at random — sometimes more frequently othertimes less often. It happens both when the vehicle is idle and when operating.
To try debig I have the current setup:
Raspberry Pi 4 + Blue Robotics Navigator, BlueOS
Direct Ethernet cabele from Pi to laptop with power from battery
Topside: [QGroundControl on Windows laptop]
What I’ve already ruled out / tried:
Power: swapped to a different 5V 6A supply. vcgencmd get_throttled returns 0x0 even after a session with dropouts (checked without rebooting), so no undervoltage has been detected since boot.
Swapped to a different flight controller board
Fresh BlueOS install on a new SD card
Connected laptop directly via Ethernet to rule out tether/network gear
Key observation: with nothing plugged into the Pi, it runs fine with no heartbeat loss. As soon as all peripherals are connected, the dropouts start. This makes me suspect one of the connected devices, but I’m not sure how to pin down which as its so inconsistent that when I plug them in one by one it may work fine but on other instances it may loose heartbeats again? I’ve also tried another windows laptop I had and the issue still occured so rules out my topside.
Any recommendations to help me me figure out the cause?
What version of BlueOS are you running? You should be on the latest stable, 1.4.3?
You can check out your tether connection quality with the tether diagnostics app - if you install this in BlueOS and share a screenshot of it, it may show that your connection has issues that cause your disconnects.
Can you share what you have plugged into the Pi? It is suspicious that dropouts start with all peripherals connected, I would suspect one of them is pulling more power or cpu usage than it should…
I have the same issue on my rov aswell. I have the latest stable BlueOS 1.4.3 and the latest firmware for the navigator. I also just installed a brand new navigator and SD card, since my Navigator suddendly died. I could not load any of the vehicle parameters, so i swapped to a new one and that fixed one issue.
I just had a test dive since i was happy with all the bench testing. I run a network test and got around 35-40mbit/s on both download and upload. When the rov went in the water it was fine for a few minute, but then started to get the MYGC 255 Heartbeat Lost alarm, lost manual control very frequently.
I used the network test function in BlueOS many times and I could see sometimes that it dropped below 10mbit/s. And next time it could be 35mbit/s.
Same answer! The tether diagnostic extension should shed some light on your issue, as well understanding what payloads you have connected .
In general, no firmware is actually running on the Navigator! The autopilot runs on the pi, and communicates to output devices on the PCB - so if replacing your Navigator fixed your issues in the past that was likely due to either a hardware failure or software issue…
I’ve also noticed that if I leave a vehicle for a very long period on Deck, it can struggle initially went entering the water. I theorize this is something to do with the black magic that the lx200 does to send ethernet over two wires, and the RF situation changing dramatically from what was stable when when the tether begins to unspool and be in a isolated RF environment. Generally it stabilizes pretty quickly and is normal in my experience, but maybe it’s continuing for you?
These two first pictures is with the blue-blue/white pair.
I swapped to the orange pair, and got a much mote stable connection. Just had a complete wet test just now and it works perfectly. So the fix for me was to swap to another twisted pair on both FXTI.
Last two pictures is with the orange pair that works flawless.
The tether diagnostic tool is great! Thanks @tony-white
Leak Sensor, barometer, relay (for dvl, pin 15), lights, servo for camera, using 5V from servo rail to power an inverter chip, using serial 1 to read a signal using GPIO 14 to read a 3.3V high low signal.
Had an issue today when driving th ROV i was loosing connection but still had video feed and could control the ROV in test tank.
Is this a connection issue as its causing connection loss but doesn’t loose video feed / control sometimes? It’s very intermitient and difficult to replicate as it’s so inconsistent.
You should not be on that “latest” version of BlueOS! I recommend you update to 1.4.3, the latest stable. There have been many more (beta) versions of BlueOs in the past 5 months! You’ll need to connect the vehicle to WiFi with internet via the icon in the upper right to update.
The graph when you were having issues shows seperation between blue and pink lines in the upper left - this indicates that one of the wires in the fathom-x tether wire does not have a good electrical connection. Those dropouts also indicate complete disconnections!
I’d recommend removing elements of the connection chain until you find the culprit - start by connecting the tether directly to your topside, not going through the slip ring or deck cable of a tether spoool? That’s after you get on 1.4.3 BlueOS stable of course!
