GPS problems on vanilla BlueBoat

Blue Robotics Vehicles BlueBoat
1

Well, I’m out of ideas. I simply cannot get the stock GPS on my shiny new BlueBoat to behave. It WILL NOT achieve a GPS fix, despite being out in my back yard with clear sky view, for hours at a time. The only time it got a fix was last night for about 15 seconds, down in my basement, with the starboard module sitting almost upside down on the ottoman in front of my couch. Its position showed it a couple blocks down the street, but at least it was close. Before then, and since then, nothing.

I resorted to carefully taking the GPS board out of the BB electronics rack, and plugged it into a laptop via USB then opened up u-center to see if that could shed any light on things. It clearly shows the GPS “seeing” a number of SVs from the different constellations, and if I click the “debug messages” button (which appears to start sending a number of UBX messages down the USB connection, in addition to the NMEA sentences), most of the signal columns turn cyan, which I believe means u-center considers those channels to be “ready” (ie, locked?) – but they never turn green/“used”, and the “Fix Mode” value never changes from “No Fix”. I’ve read other posts where people suggest configuring the GPS to output NMEA and turn off auto_config, etc, which I’ve tried – nothing has worked for me. I simply cannot understand why the GPS won’t acquire a Fix. Or even more confusing, why it DID briefly acquire a fix with the unit upside down in my basement – but under the clear blue skies of my backyard, nothing.

u-Center Status
u-Center Status1920×1140 239 KB

Just to be clear, I followed the “congratulations, you just received your new BB!” instructions to the letter – paired radios, updated ArduRover, updated BlueOS, calibrated the accelerometers and compass, and all of that went smoothly. As far as I know, it never got a GPS fix during any of that, but I wasn’t really paying attention to GPS status at that point.

Anyways, any suggestions would be really appreciated! If nothing else, I’ve become a u-center expert in the past 24hrs. I’ve also got a fair amount of previous experience with Mission Planner and ArduRover/Plane/Copter, and GPS technology in general, so I think I have a reasonably decent idea of what I’m doing. But I’m out of ideas on this one.

2

Hi @rjmickle
So sorry for the hassle! We’ve seen this with a very small number of units in the wild. Please contact us via our support form (report a problem with a product) and we can send you a replacement! Thanks

3

No worries Tony – thanks so much for getting back to me, I really appreciate it. I was worried you guys would be taking an extra long holiday this week for the 4th of July, so I’m relieved to hear from you!

I’ve submitted a support form request.

Thanks again!

4

@rjmickle did you manage to get your GPS problem solved. we have the same issue with multiple GNSS that all work plugged into a Pixhawk with same ardupilot firmware and settings but slow or no fix with fewer sats when fixed.

5

Sorry for the long post below – I’ve never been accused of being concise…

So, some progress today. I actually bought a separate NEO-M9N board last week with an external SMA patch antenna and bench tested that with u-center via USB cable to make sure that it was working nicely before installing it in my blueboat. Worked beautifully, got extremely fast fixes, even from a cold start. I then designed and 3D printed a nice little bracket for the Taoglas MagmaX antenna and mounted it on the inside of the port module, similar to how the mRo card is mounted on the starboard side, and again tested this out in my backyard with u-center via USB cable to my laptop — again, it worked beautifully.

I then hooked it into the Navigator via its built-in UART, with power coming from the 5V side of the fuse block on the port side – and lo-and-behold, it started behaving exactly like my mRo unit within a minute of turning on the boat: tracking lots of satellites, but refusing to flip to “3D Fix”, which it had done within seconds during my earlier testing. I panicked a bit and cursed myself for not having backed up the parameters to a file before doing this, but discovered that I could “revert to default configuration” within u-center, to “rescue” it: within u-center, go to View → Configuration View, then near the top of the list on the left, select the CFG (Configuration) page, and then select the “Revert to default configuration” item. Then down below the list on the left, click Send, and it should immediately flip to 3D Fix mode (assuming it’s tracking lots of satellites).

With the non-mRo GPS now working, I started doing some testing. The GPS board I bought has a little PPS LED that blinks at 1Hz when it has a proper fix (the mRo card has this too, it blinks pink when it’s got a fix, I believe – not to be confused with the other LED on there that’s controlled by the Navigator and can blink one of several different colours to indicate the state of ArduRover’s happiness). I again turned on the boat, the GPS acquired a hot start fix within seconds (as indicated by the blinking PPS LED) while the boat booted up, and again, once ArduRover was up and running, the PPS LED quit blinking, indicating no fix. I again connected using u-center and confirmed that it was back to the way my original mRo card was: tracking, but no Fix. I was able to reproduce this behaviour several times.

Having now conclusively determined that something in the Navigator was mis-configuring the GPS (when I have some time, I aim to determine exactly what setting is causing this behaviour), I went about turning off GPS_AUTOCONFIG in ArduRover, and manually setting various things in both the ArduRover parameters and on the GPS card. For example, as others have done in other threads, I configured the GPS card to only output NMEA on UART1, then set the ArduRover GPS_TYPE to NMEA (I first tried manually setting it to the ublox protocol on both sides, but couldn’t get it to work, but may try again now that I have things more or less working), as well as playing with baud rates, etc. But having “rescued” my second GPS card by restoring default settings, I held my breath and did the same thing to the stock mRo card that came with my Blueboat – and guess what, it worked! The instant I hit “restore default settings” and clicked Send, the mRo card immediately got a 3D Fix. So I then configured it the same way as the other board, and now both are working beautifully.

So, in a nutshell: it looks to me like something in ArduRover’s autoconfig function is doing something that these M9N chips don’t like. I’d prefer to use the ublox protocol instead of NMEA, but it’s relatively easy to switch to that once things are confirmed to be working in a future update. For now, I’ll live with the less-efficient NMEA data stream. (a small tip: you can safely turn off the GxGSV messages, the boat seems quite happy without them, and they fill up about half of the NMEA ascii stream being sent – in other words, disabling GSV messages cut the total message block size roughly in half)

So, give all of that a try. It’s a pain taking the mRo card out from underneath the wifi modem – it’d be sweet if you could easily plug in a microUSB cable from the side or something. But mine is now working and back in its original spot, with a redundant unit now installed in the other hull.

Hope this helps!

EDIT: I’ve recently discovered that you can remove the GLL and GSA messages too, ArduRover doesn’t parse those. GSA in particular is a pretty large message, so removing that will cut down on your message block size considerably.

1 Like
6

Hi all,

@rjmickle I might be able to make things slightly easier (at least if your hardware is hard to disassemble

I have seen two different issues with these GPS that we couldn’t get to the bottom off (the units worked fine once returned to us).

  • One of them plainly didn’t get a fix, despite seemingly communicating perfectly fine. IIRC it saw something like 3 satellites, but once returned, worked perfectly fine at our facilities. The replacement one fixed the issue for the customer.

  • One of them had erratic behavior, no comms after ardupilot attempted to configure it. manually setting it up to NMEA worked perfectly fine, so that is what we went with, and it has been operating fine like that ever since.

For helping diagnosing, I suggest the following:
Detach the serial port from the autopilot:
Click the (x) to remove it

image
image753×845 63.9 KB

and click “Save and restart”
Screenshot 2024-07-12 at 14.04.20

(it was pointed below that the power cycle is not required)
~Now power-cycle your system so that the gps looses any changes Ardupilot might have made.~

Now there’s a tricky part. You need to find out (or remember) the baud rate.
I think the standards are 38400, 57600, and 230400.

  • One option is going to the terminal and using “screen” to try one by one:
    screen /dev/ttyAMA3 57600
    Try all of them until it looks like you have a constant stream of data coming in.
    remember to end each session by pressing “ctrl+a”, then “k” (kill) then “y”(yes).

  • The other options is to just guess them in the next step until it works.

Go to Bridges, in BlueOS, click the plus button at the lower right corner, and put theses settings in (with the baudrate you found/guessed).

Screenshot 2024-07-12 at 14.13.46

This will expose the serial port on an UDP server on port 15000 at the vehicle.

Next download PyGPSClient. Unfortunately U-Center doesn’t play nicely with our network options and there’s no documentation available for us to try to fix it.

python -m pip install pygpsclient

Put these settings in and click the TCP/UDP button to connect:

Screenshot 2024-07-12 at 14.17.02

From there on, you can use pygpsclient to do most things u-center does, such as viewing the received messages, configuring serial ports, messages sents and rates.

1 Like
7

Haha, your definition of “slightly easier” is definitely different than mine! I’m more of a “direct connection” kind of guy, I guess… Thanks nonetheless for the helpful writeup, I’m sure it’ll come in handy at some point.

What would be awesome is to determine exactly which UBX messages ArduRover wants, then manually configure the GPS to output those, rather than relying on gps_autoconfig, which consistently seems to be confusing the GPS chip. I have to imagine that autoconfig worked at one point in the past, so I’m also curious what version of ArduRover introduced the issue, and what exactly that issue is.

If the GPS gets configured in the way that I’ve described above (ie, tracks satellites, but won’t actually use them), I’m not sure PyGPSclient would be able to “rescue” the GPS, as the only way I was able to achieve this was yesterday via u-center’s “Revert to default configuration”. No amount of message/baud/rate configuration was able to make the GPS behave, at least, not that I was able to achieve with hours of poking around in u-center.

“Now power-cycle your system so that the gps looses any changes Ardupilot might have made.” – To be clear, whatever ArduRover 4.5.4 does to the GPS apparently gets saved to its flash memory, and essentially becomes permanent, until reverted by u-center. I’m 100% certain of this, and have reliably and repeatably observed this on 2 different NEO-M9N boards (one, the mRo unit that came with the boat, the other a Sparkfun board). Now that I know how to “rescue” the GPS, I may try rolling back ArduRover to see if autoconfig works in a past version. I tried this last week when I first was having problems, but it didn’t help.

Either way, I’d like to use the UBX protocol instead of NMEA, so hopefully this issue gets fixed at some point in ArduRover!

Thanks again, Willian, appreciate the input.

8

Thanks, I’ll edit my message to take this out, it has been a while since that happened.

I’m taking a look at the code, it is somewhat confusing. the first step taken seems to be asking for an ubx message and setting the baudrate.

Then it really depends on what actualy GPS is connected, but I think these are the desired message rates or a “generic” u-blox gps.

1 Like
9

Another important thing to note (probably not quite @rjmickle’s issue):

This PR Fixed an issue where we could have multiple access on the serial ports. That means that the both Ardupilot and Bridges or Ping services could access the port at the same time, which could also cause communication issues and baudrate mismatches.

I someone else faces a similar issue, I recommend trying the latest BlueOS 1.3.0-beta.7

10

I’m taking a look at the code, it is somewhat confusing. the first step taken seems to be asking for an ubx message and setting the baudrate.

Yeah, I definitely noticed that ArduRover was re-configuring the port to 230400, regardless of what I had it set to before that.

Then it really depends on what actualy GPS is connected, but I think these are the desired message rates or a “generic” u-blox gps.

ooo, interesting, thanks for digging that up! I’ll play around with the ublox message output settings on my second GPS board (which is way easier to access than the BB mRo unit), and see if these are all message types that I can manually enable, then test to see if things work via ublox protocol. Thanks Willian!

2 Likes
11

Ok, so after a few weeks off, a little more information:

I looked at the ArduPilot UBX config code that Willian kindly linked to above, specifically, the UBX message rates that are apparently being set by ArduPilot:

#define RATE_POSLLH 1
#define RATE_STATUS 1
#define RATE_SOL 1
#define RATE_TIMEGPS 5
#define RATE_PVT 1
#define RATE_VELNED 1
#define RATE_DOP 1
#define RATE_HW 5
#define RATE_HW2 5
#define RATE_TIM_TM2 1

Note, I used my secondary GPS unit installed in the port side for this testing. It’s not an mRo unit like the one that came with the boat, but it’s a Sparkfun board with a NEO-M9N GPS unit, the same GPS chip that’s on the BB mRo board. As I mentioned above, it behaves exactly the same as the mRo unit when being being configured by AP – it’s just much easier for me to access the USB port on it while testing, since the BB GPS board is “hidden” above the wifi radio. Both GPS units are running fw version 4.04, which is the fw version they shipped with.

Anyways, I used u-center to configure it to output the above UBX messages on uart1 (after turning off all NMEA messages on uart1). This was relatively easy to do – however, when I tried to select the NAV_SOL message in the dropdown box, it wouldn’t let me, it kept reverting the dropdown box to my previous selection. I thought this was odd, so did a bit more poking, and it turns out u-blox removed the SOL message from the Gen 9 chips – as shown in section 5.2.5 of this u-blox document. It says to use the PVT message instead, which is also on the list above.

I haven’t yet tested whether the above manually-configured UBX messages make AP happy, I’ll do that tonight. But I’m wondering whether AP trying to set the SOL message is what’s making the M9N units go into the weird mode I described in my original post above, where they’re seeing a whole bunch of satellites, but not actually using them? But why isn’t this affecting everyone? The fact that it does exactly the same thing to 2 different M9N GPS units from 2 different manufacturers suggests to me that I don’t just have a flaky mRo GPS unit. I received the replacement mRo unit right before I left on vacation, so haven’t had a chance to test it, but I expect exactly the same behavior from it.

Anyways, if the above UBX message list makes AP happy, I’ll dig the mRo unit out from under the wifi radio and configure it the same way. But just thought I’d post this little “SOL message removal” nugget in the meantime, to see if anyone has any comment.

1 Like