Eth0 bouncing on newer blueos releases

Hello we have encountered a problem upgrading a bluerov heavy to 1.4-beta.11 and beta.12 whereby the eth0 link is constantly re-established. There are no physical errors accruing on the interface and we have tried using an alternate tether, tether patch lead, fathom-x interface and topside control station.

Rolling back to 1.3 fails, leaving the ROV on the “factory” image which does not exhibit the network problem however we have not been able upgrade to 1.3 releases

root@blueos:/var/log# tail -n 100 /var/log/syslog
Jan 31 15:35:23 blueos kernel: [ 2347.495079] bcmgenet fd580000.ethernet eth0: Link is Down
Jan 31 15:35:23 blueos avahi-daemon[321]: Withdrawing address record for 192.168.2.2 on eth0.
Jan 31 15:35:24 blueos avahi-daemon[321]: Joining mDNS multicast group on interface eth0.IPv4 with address 192.168.2.2.
Jan 31 15:35:24 blueos avahi-daemon[321]: New relevant interface eth0.IPv4 for mDNS.
Jan 31 15:35:24 blueos avahi-daemon[321]: Registering new address record for 192.168.2.2 on eth0.IPv4.
Jan 31 15:35:24 blueos dhcpcd[715]: eth0: carrier acquired
Jan 31 15:35:24 blueos kernel: [ 2348.523065] bcmgenet fd580000.ethernet: configuring instance for external RGMII (RX delay)
Jan 31 15:35:24 blueos kernel: [ 2348.523397] bcmgenet fd580000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx
Jan 31 15:35:24 blueos kernel: [ 2348.524398] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Jan 31 15:35:24 blueos dhcpcd[715]: eth0: IAID dd:2e:35:dd
Jan 31 15:35:24 blueos dhcpcd[715]: eth0: adding address fe80::cacc:36ff:26b6:d05c
Jan 31 15:35:24 blueos avahi-daemon[321]: Joining mDNS multicast group on interface eth0.IPv6 with address fe80::cacc:36ff:26b6:d05c.
Jan 31 15:35:24 blueos avahi-daemon[321]: New relevant interface eth0.IPv6 for mDNS.
Jan 31 15:35:24 blueos avahi-daemon[321]: Registering new address record for fe80::cacc:36ff:26b6:d05c on eth0.*.
Jan 31 15:35:24 blueos dhcpcd[715]: eth0: soliciting an IPv6 router
Jan 31 15:35:25 blueos dhcpcd[715]: eth0: soliciting a DHCP lease
Jan 31 15:35:30 blueos avahi-daemon[321]: Withdrawing address record for 192.168.2.2 on eth0.
Jan 31 15:35:30 blueos avahi-daemon[321]: Leaving mDNS multicast group on interface eth0.IPv4 with address 192.168.2.2.
Jan 31 15:35:30 blueos avahi-daemon[321]: Interface eth0.IPv4 no longer relevant for mDNS.
Jan 31 15:35:30 blueos avahi-daemon[321]: Withdrawing address record for fe80::cacc:36ff:26b6:d05c on eth0.
Jan 31 15:35:30 blueos avahi-daemon[321]: Leaving mDNS multicast group on interface eth0.IPv6 with address fe80::cacc:36ff:26b6:d05c.
Jan 31 15:35:30 blueos avahi-daemon[321]: Interface eth0.IPv6 no longer relevant for mDNS.
Jan 31 15:35:30 blueos avahi-daemon[321]: Joining mDNS multicast group on interface eth0.IPv4 with address 192.168.2.2.
Jan 31 15:35:30 blueos avahi-daemon[321]: New relevant interface eth0.IPv4 for mDNS.
Jan 31 15:35:30 blueos avahi-daemon[321]: Registering new address record for 192.168.2.2 on eth0.IPv4.
Jan 31 15:35:30 blueos dhcpcd[715]: eth0: carrier lost
Jan 31 15:35:30 blueos avahi-daemon[321]: Interface eth0.IPv4 no longer relevant for mDNS.
Jan 31 15:35:30 blueos avahi-daemon[321]: Leaving mDNS multicast group on interface eth0.IPv4 with address 192.168.2.2.
Jan 31 15:35:30 blueos kernel: [ 2353.958546] bcmgenet fd580000.ethernet eth0: Link is Down
Jan 31 15:35:30 blueos avahi-daemon[321]: Withdrawing address record for 192.168.2.2 on eth0.
Jan 31 15:35:31 blueos avahi-daemon[321]: Joining mDNS multicast group on interface eth0.IPv4 with address 192.168.2.2.
Jan 31 15:35:31 blueos avahi-daemon[321]: New relevant interface eth0.IPv4 for mDNS.
Jan 31 15:35:31 blueos avahi-daemon[321]: Registering new address record for 192.168.2.2 on eth0.IPv4.
Jan 31 15:35:31 blueos dhcpcd[715]: eth0: carrier acquired
Jan 31 15:35:31 blueos kernel: [ 2354.986428] bcmgenet fd580000.ethernet: configuring instance for external RGMII (RX delay)
Jan 31 15:35:31 blueos kernel: [ 2354.986896] bcmgenet fd580000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx
Jan 31 15:35:31 blueos kernel: [ 2354.987247] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Jan 31 15:35:31 blueos dhcpcd[715]: eth0: IAID dd:2e:35:dd
Jan 31 15:35:31 blueos dhcpcd[715]: eth0: adding address fe80::cacc:36ff:26b6:d05c
Jan 31 15:35:31 blueos avahi-daemon[321]: Joining mDNS multicast group on interface eth0.IPv6 with address fe80::cacc:36ff:26b6:d05c.
Jan 31 15:35:31 blueos avahi-daemon[321]: New relevant interface eth0.IPv6 for mDNS.
Jan 31 15:35:31 blueos avahi-daemon[321]: Registering new address record for fe80::cacc:36ff:26b6:d05c on eth0.*.
Jan 31 15:35:31 blueos dhcpcd[715]: eth0: soliciting an IPv6 router
Jan 31 15:35:32 blueos dhcpcd[715]: eth0: soliciting a DHCP lease
Jan 31 15:35:36 blueos avahi-daemon[321]: Withdrawing address record for 192.168.2.2 on eth0.
Jan 31 15:35:36 blueos avahi-daemon[321]: Leaving mDNS multicast group on interface eth0.IPv4 with address 192.168.2.2.
Jan 31 15:35:36 blueos avahi-daemon[321]: Interface eth0.IPv4 no longer relevant for mDNS.
Jan 31 15:35:36 blueos avahi-daemon[321]: Withdrawing address record for fe80::cacc:36ff:26b6:d05c on eth0.
Jan 31 15:35:36 blueos avahi-daemon[321]: Leaving mDNS multicast group on interface eth0.IPv6 with address fe80::cacc:36ff:26b6:d05c.
Jan 31 15:35:36 blueos avahi-daemon[321]: Interface eth0.IPv6 no longer relevant for mDNS.
Jan 31 15:35:36 blueos avahi-daemon[321]: Joining mDNS multicast group on interface eth0.IPv4 with address 192.168.2.2.
Jan 31 15:35:36 blueos avahi-daemon[321]: New relevant interface eth0.IPv4 for mDNS.
Jan 31 15:35:36 blueos avahi-daemon[321]: Registering new address record for 192.168.2.2 on eth0.IPv4.
Jan 31 15:35:36 blueos dhcpcd[715]: eth0: carrier lost
Jan 31 15:35:36 blueos avahi-daemon[321]: Interface eth0.IPv4 no longer relevant for mDNS.
Jan 31 15:35:36 blueos avahi-daemon[321]: Leaving mDNS multicast group on interface eth0.IPv4 with address 192.168.2.2.
Jan 31 15:35:36 blueos kernel: [ 2360.455772] bcmgenet fd580000.ethernet eth0: Link is Down
Jan 31 15:35:36 blueos avahi-daemon[321]: Withdrawing address record for 192.168.2.2 on eth0.
Jan 31 15:35:37 blueos avahi-daemon[321]: Joining mDNS multicast group on interface eth0.IPv4 with address 192.168.2.2.
Jan 31 15:35:37 blueos avahi-daemon[321]: New relevant interface eth0.IPv4 for mDNS.
Jan 31 15:35:37 blueos avahi-daemon[321]: Registering new address record for 192.168.2.2 on eth0.IPv4.
Jan 31 15:35:37 blueos dhcpcd[715]: eth0: carrier acquired
Jan 31 15:35:37 blueos kernel: [ 2361.502125] bcmgenet fd580000.ethernet: configuring instance for external RGMII (RX delay)
Jan 31 15:35:37 blueos kernel: [ 2361.502667] bcmgenet fd580000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx
Jan 31 15:35:37 blueos kernel: [ 2361.505231] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Jan 31 15:35:37 blueos dhcpcd[715]: eth0: IAID dd:2e:35:dd
Jan 31 15:35:37 blueos dhcpcd[715]: eth0: adding address fe80::cacc:36ff:26b6:d05c
Jan 31 15:35:37 blueos avahi-daemon[321]: Joining mDNS multicast group on interface eth0.IPv6 with address fe80::cacc:36ff:26b6:d05c.
Jan 31 15:35:37 blueos avahi-daemon[321]: New relevant interface eth0.IPv6 for mDNS.
Jan 31 15:35:37 blueos avahi-daemon[321]: Registering new address record for fe80::cacc:36ff:26b6:d05c on eth0.*.
Jan 31 15:35:37 blueos dhcpcd[715]: eth0: soliciting a DHCP lease
Jan 31 15:35:38 blueos dhcpcd[715]: eth0: soliciting an IPv6 router
Jan 31 15:35:43 blueos avahi-daemon[321]: Withdrawing address record for 192.168.2.2 on eth0.
Jan 31 15:35:43 blueos avahi-daemon[321]: Leaving mDNS multicast group on interface eth0.IPv4 with address 192.168.2.2.
Jan 31 15:35:43 blueos avahi-daemon[321]: Interface eth0.IPv4 no longer relevant for mDNS.
Jan 31 15:35:43 blueos avahi-daemon[321]: Withdrawing address record for fe80::cacc:36ff:26b6:d05c on eth0.
Jan 31 15:35:43 blueos avahi-daemon[321]: Leaving mDNS multicast group on interface eth0.IPv6 with address fe80::cacc:36ff:26b6:d05c.
Jan 31 15:35:43 blueos avahi-daemon[321]: Interface eth0.IPv6 no longer relevant for mDNS.
Jan 31 15:35:43 blueos avahi-daemon[321]: Joining mDNS multicast group on interface eth0.IPv4 with address 192.168.2.2.
Jan 31 15:35:43 blueos avahi-daemon[321]: New relevant interface eth0.IPv4 for mDNS.
Jan 31 15:35:43 blueos avahi-daemon[321]: Registering new address record for 192.168.2.2 on eth0.IPv4.
Jan 31 15:35:43 blueos kernel: [ 2366.948805] bcmgenet fd580000.ethernet eth0: Link is Down
Jan 31 15:35:43 blueos dhcpcd[715]: eth0: carrier lost
Jan 31 15:35:43 blueos avahi-daemon[321]: Interface eth0.IPv4 no longer relevant for mDNS.
Jan 31 15:35:43 blueos avahi-daemon[321]: Leaving mDNS multicast group on interface eth0.IPv4 with address 192.168.2.2.
Jan 31 15:35:43 blueos avahi-daemon[321]: Withdrawing address record for 192.168.2.2 on eth0.
Jan 31 15:35:44 blueos kernel: [ 2367.981305] bcmgenet fd580000.ethernet: configuring instance for external RGMII (RX delay)
Jan 31 15:35:44 blueos kernel: [ 2367.981676] bcmgenet fd580000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx
Jan 31 15:35:44 blueos avahi-daemon[321]: Joining mDNS multicast group on interface eth0.IPv4 with address 192.168.2.2.
Jan 31 15:35:44 blueos avahi-daemon[321]: New relevant interface eth0.IPv4 for mDNS.
Jan 31 15:35:44 blueos avahi-daemon[321]: Registering new address record for 192.168.2.2 on eth0.IPv4.
Jan 31 15:35:44 blueos dhcpcd[715]: eth0: carrier acquired
Jan 31 15:35:44 blueos kernel: [ 2367.982506] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Jan 31 15:35:44 blueos dhcpcd[715]: eth0: IAID dd:2e:35:dd
Jan 31 15:35:44 blueos dhcpcd[715]: eth0: adding address fe80::cacc:36ff:26b6:d05c
Jan 31 15:35:44 blueos avahi-daemon[321]: Joining mDNS multicast group on interface eth0.IPv6 with address fe80::cacc:36ff:26b6:d05c.
Jan 31 15:35:44 blueos avahi-daemon[321]: New relevant interface eth0.IPv6 for mDNS.
Jan 31 15:35:44 blueos avahi-daemon[321]: Registering new address record for fe80::cacc:36ff:26b6:d05c on eth0.*.
Jan 31 15:35:44 blueos dhcpcd[715]: eth0: soliciting a DHCP lease
Jan 31 15:35:45 blueos dhcpcd[715]: eth0: soliciting an IPv6 router
root@blueos:/var/log#
1 Like

Hi @Jono, welcome to the forum :slight_smile:

Sorry to hear you’re having this issue. Given your network is working fine in a previous BlueOS release then it’s likely a bug with the 1.4 beta images you’ve tried.

@patrickelectric is this related to / caused by bluerobotics/BlueOS#3090?

Is this from you trying to manually apply a 1.3 image, or are you just expecting it to try 1.3 on the way back down to factory? BlueOS’s automatic version selection currently only has a single fallback image, so if it fails to load the applied one then it falls back to whatever is specified as your factory image.

If selecting and switching to 1.3 isn’t working for you then it may help to try deleting the BlueOS settings first, and/or you could manually set your 1.3 image to be your new factory version, or worst case you could download and flash 1.3 onto your SD card.

1 Like

Hi @EliotBR , out of curiosity, what version of BlueOS is BR currently shipping new units with as the “Factory”? Presumably if you have an older unit you should roll this forward from time to time? Also I know @Jono tried loading 1.3 from a *.tar file too and that failed. So it seems to allow changing to later releases but not earlier releases. Does that make sense or is that a red herring?

Managed to get BlueOS version 1.3.1 going by doing a settings reset in BlueOS after removing all previously installed versions and defaulting to factory version.
Thanks
FYI @Jono @EliotBR @PeterM

From the Revision History sections on our product pages, both the BlueROV2 and BlueBoat are shipping with BlueOS 1.3.1, which is the latest stable.

When you download a full image for flashing onto an SD card then that version will show up as the “factory” one.

We agree that having some kind of updatable fallback makes sense, although I personally don’t think that should be the factory image because of the semantics and corresponding support situations that can arise.

At the moment there’s no graphical way of creating/updating such an image from a running BlueOS instance, but since the factory image is just defined by a docker tag it’s possible to change using the command line if you particularly want to (as per the “manually set…” instructions I linked above).

Ideally BlueOS would robustly and easily roll backwards and forwards between versions without issues. In practice it’s complicated by the fact that settings created in one version may be incompatible with those expected by another, which there are some safeguards in place against (using a versioned settings API), but there are seemingly still some issues there from time to time.

If you have access to some kind of version but not the one you want, then resetting the settings should generally allow switching to any version, although rolling forwards is typically less error-prone than rolling backwards (at least in terms of settings compatibility / existence). If you have only ssh access then it may still be possible to restore access by removing the settings and rebooting, but if that doesn’t work then flashing the SD card is the most comprehensive way of doing a full reset of everything.

1 Like