We have a communication loss problem every time we start a mission. When we have the boat on the shore, we set the mode to AUTO, and after a few seconds, the communication stops and it switches to HOLD mode (I have it set that way).
Then, after a few seconds, communication is restored and I can put it back in AUTO MODE and it completes the full one-hour mission correctly. But always, always, this happens to us at the beginning. We communicate via the internet with Zerotier. What could be causing this?
Another question I have is about the batteries in QGroundControl. If I put 4 batteries in the USV, is there a way to know the voltage of each one, or since they are all in parallel, can I only set one battery? Same with the consumption, I mean, if I put 4 batteries of 18000 mAh, do I have to multiply that capacity by 4 to see the remaining percentage?
Finally, what happens if I disable all the Kalman EKF filters? I’m using Injector and sometimes the program I use to do bathymetries sends a zero in lat/long/heading via UDP to the USV and gives me EKF variance. I’ve also thought about raising the “FS_EKF_THRESH” parameter to 1.00 to see if it prevents these errors during the mission.
To clarify, are you saying you’ve set it to disable communication, or you’ve configured an autopilot failsafe that switches to HOLD mode if it loses communication for a few seconds?
From your description it sounds like you’re enabling AUTO mode while the boat is out of the water, so maybe it’s trying to spin up the motors and drawing enough current that the voltage drops and your network hardware browns out? Beyond power issues I’m unsure what would cause a loss of communication when AUTO mode is engaged.
Have you compared control station telemetry logs with the autopilot bin logs, to see whether both sides lose connection with each other? It may also help to check the relevant service logs of your network manager. You haven’t mentioned whether you’re running BlueOS on a BlueBoat, or some kind of custom setup.
It’s not possible to measure parallel voltages independently without disconnecting them. Parallel batteries will also charge each other to maintain the same voltage.
We generally recommend using voltage values instead of ArduPilot’s battery capacity estimates, because the estimates currently assume the battery is fully charged whenever the autopilot is powered on, which is not always correct.
If you do always start your vehicle with fully charged batteries (or want to use the estimates regardless, with some understanding of how they work) then yes, you would need to specify the full capacity of your battery setup for it to provide accurate remaining capacity estimates. It doesn’t care how many cells or sub-sections you have in your batteries, it just needs to know how much total energy is stored/available to use.
Note that it’s a good idea to set up voltage-based battery failsafes, so even if you’re visually monitoring an incorrect capacity value the autopilot still has some warning mechanisms for when the battery is low.
That depends what you mean. If you disable the latest version (EKF3), and the previous version (EKF2), then I suppose it has to switch back to DCM, which is a different kind of filter.
An autopilot cannot function without some kind of understanding/estimate of the state in between measurements - even “remember/use the last measurement value” is technically a kind of filter (just not a very good one). The EKF is also used for sensor fusion, to blend different sensor values into estimates of state that aren’t directly measured, and/or to compare the consistency of complementary measurements to indicate the health of the sensors (e.g. if the compass indicates a rapid rotation, but the gyroscope and accelerometer do not, then the variance will go up because the sensors disagree with each other).
The NMEA Injector built into BlueOS, or something else?
EKF variance in such a situation is expected - the EKF is receiving conflicting information. What kind of variance values are you talking about, and how are you monitoring that currently?
It seems problematic that your program is sending zero values - do you know why it’s doing that, and whether it’s documented behaviour, and/or whether it’s trying to flag somehow that those values shouldn’t be used without that being respected in whatever is passing them on or receiving them?
Relaxing a failsafe threshold makes sense if you’re expecting your sensors to disagree with each other (e.g. you’re operating in strong magnetic fields, or getting dodgy estimates from a sensor). It’s of course preferable to have better data for the autopilot to work with, and reducing warnings may mean it takes longer to identify when genuine issues are occurring, but if low quality / noisy data is “normal” for your operations then relaxing warning levels makes practical sense.
I have it set in the parameters so that if it triggers a “failsafe” it goes into “HOLD” mode, meaning it doesn’t return home.
No, we activate AUTO mode when we have already put it in the water floating.
How do you look at the telemetry log from the control station? I have only looked at the LOG files that I download from the USV.
Ok, regarding the batteries, it’s better to use and rely on the voltage.
I use the injector by sending from my bathymetry program the 4 NMEA signals (RMC, GGA, HDT, etc.) which are then received by the USV through Serial 2 with “udpin:0.0.0.0:27000”. I think the zeros are sent by the bathymetry program, because if the GPS cannot provide a valid reading it returns, for example: $GPHDT,# and leaves the heading value blank, but it seems my bathymetry program interprets this blank as 0 and sends it.
Ok, I’ll try raising the “FS_EKF_THRESH” to see how it goes.
The issue with communication loss really has me intrigued since it happens to us almost all the time. Explain to me how I can check the log from the operator to see if I can find something, I use QGroundControl.
Here are the relevant QGC docs - feel free to follow up if something’s missing or unclear
Hmm, that’s not great. Are you able to modify that program at all?
It’s probably also possible to ignore zero values in the autopilot, but that kind of modification is unlikely to be accepted for general use in the official firmware repository unless there’s a defined specification for what to do in that kind of scenario.
Alternatively it may be possible to achieve using the BlueOS NMEA Injector, though it would need to be updated to support HDT messages.
Does it come up as an autocomplete option if you click “Add Expression” and then start typing HEARTBEAT? I believe there are multiple heartbeat sources involved, so some of the field values should alternate:
Sorry, I didn’t notice you’re still using a DataFlash (.bin) log pulled from the autopilot. QGC should be recording its own Telemetry (.tlog) files, which is where the HEARTBEAT and other MAVLink messages are recorded.
Thank you very much Eliot, of course, I thought that from the LOG of the Blue Boat you could also see the HEARTBEAT.
Okay, I found it in the QTC .tlog, and I see that yes, there is a moment when there are no HEARTBEATs, but how do I interpret this? A momentary internet connection loss?
Hi @Pulsar79 -
A loss of heartbeat means a loss of connection to the vehicle via the Base Station. This would trigger your Fail-Safe action- does it happen as frequently as you suspected? If so, one of your radios may be damaged (was it ever turned on with antenna not connected?) or similarly one of your antennas could be bad?
Hi @Pulsar79 -
What is likely happening is on launch, as the vehicle drives away the route it was using to reach you via zerotier changes - it no longer can access you via WiFi or some connection it had locally, and can only reach you via the 4G connection. It can take zerotier 1 to 5 minutes to find this new route, but then you regain connection and everything works fine!
To avoid, make sure the vehicle is not connected to your system via the BaseStation or the Pi WiFi - only the 4G connection before launch!
Thanks, Toni. The computer I use with QGroundControl is connected to my phone’s internet, and the USV has a router with a SIM card that provides it with internet.
Can the log show if the internet was lost or only if it stopped sending HEARTBEAT?
