BlueROV2 navigator stability

Blue Robotics Vehicles
1

Hello,

We have what appears to be an intermittent problem with one of our BlueROV where the flight controller board starts to cyclically reboot. Running latest stable code, ardupilot 4.6.5, blueOS 1.4.2, the problem exhibits as loss of functional control during operations with the craft remaining active under IMU control.

What next steps should be taken to isolate a root cause?

7/11/2025 12:38:33 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:38:29 pm : Frame: VECTORED_6DOF
7/11/2025 12:38:29 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:38:29 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:38:29 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:38:25 pm : Frame: VECTORED_6DOF
7/11/2025 12:38:25 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:38:25 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:38:25 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:38:21 pm : Frame: VECTORED_6DOF
7/11/2025 12:38:21 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:38:21 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:38:21 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:38:17 pm : Frame: VECTORED_6DOF
7/11/2025 12:38:17 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:38:17 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:38:17 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:38:13 pm : Frame: VECTORED_6DOF
7/11/2025 12:38:13 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:38:13 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:38:13 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:38:10 pm : Frame: VECTORED_6DOF
7/11/2025 12:38:10 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:38:10 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:38:10 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:38:07 pm : Frame: VECTORED_6DOF
7/11/2025 12:38:07 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:38:07 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:38:07 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:38:06 pm : MYGCS: 255, heartbeat lost
7/11/2025 12:38:02 pm : Frame: VECTORED_6DOF
7/11/2025 12:38:02 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:38:02 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:38:02 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:37:58 pm : Frame: VECTORED_6DOF
7/11/2025 12:37:58 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:37:58 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:37:58 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:37:55 pm : Frame: VECTORED_6DOF
7/11/2025 12:37:55 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:37:55 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:37:55 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:37:51 pm : Frame: VECTORED_6DOF
7/11/2025 12:37:51 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:37:51 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:37:51 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:37:48 pm : DVL is too hot (48.86 C). Please cool it down.
7/11/2025 12:37:47 pm : Frame: VECTORED_6DOF
7/11/2025 12:37:47 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:37:47 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:37:47 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:37:44 pm : Frame: VECTORED_6DOF
7/11/2025 12:37:44 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:37:44 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:37:44 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:37:41 pm : Frame: VECTORED_6DOF
7/11/2025 12:37:41 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:37:41 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:37:40 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:37:37 pm : Frame: VECTORED_6DOF
7/11/2025 12:37:37 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:37:37 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:37:37 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:37:32 pm : Frame: VECTORED_6DOF
7/11/2025 12:37:32 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:37:32 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:37:32 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:37:29 pm : Frame: VECTORED_6DOF
7/11/2025 12:37:29 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:37:29 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:37:29 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:37:26 pm : Frame: VECTORED_6DOF
7/11/2025 12:37:26 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:37:26 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:37:26 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:37:22 pm : Frame: VECTORED_6DOF
7/11/2025 12:37:22 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:37:22 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:37:22 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:37:18 pm : Frame: VECTORED_6DOF
7/11/2025 12:37:18 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:37:18 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:37:18 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:37:15 pm : Frame: VECTORED_6DOF
7/11/2025 12:37:15 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:37:15 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:37:14 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:37:11 pm : Frame: VECTORED_6DOF
7/11/2025 12:37:11 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:37:11 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:37:11 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:37:10 pm : MYGCS: 255, heartbeat lost
7/11/2025 12:37:07 pm : Frame: VECTORED_6DOF
7/11/2025 12:37:07 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:37:07 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:37:07 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:37:03 pm : Frame: VECTORED_6DOF
7/11/2025 12:37:03 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:37:03 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:37:03 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:36:59 pm : Frame: VECTORED_6DOF
7/11/2025 12:36:59 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:36:59 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:36:59 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:36:56 pm : Frame: VECTORED_6DOF
7/11/2025 12:36:56 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:36:56 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:36:56 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:36:52 pm : DVL is too hot (48.69 C). Please cool it down.
7/11/2025 12:36:52 pm : Frame: VECTORED_6DOF
7/11/2025 12:36:52 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:36:52 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:36:52 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:36:51 pm : Frame: VECTORED_6DOF
7/11/2025 12:36:51 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:36:51 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:36:51 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:36:31 pm : Frame: VECTORED_6DOF
7/11/2025 12:36:31 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:36:31 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:36:31 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:36:27 pm : Frame: VECTORED_6DOF
7/11/2025 12:36:27 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:36:27 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:36:27 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:36:23 pm : Frame: VECTORED_6DOF
7/11/2025 12:36:23 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:36:23 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:36:23 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:36:22 pm : MYGCS: 255, heartbeat lost
7/11/2025 12:36:19 pm : Frame: VECTORED_6DOF
7/11/2025 12:36:19 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:36:19 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:36:19 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:36:16 pm : Frame: VECTORED_6DOF
7/11/2025 12:36:16 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:36:16 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:36:16 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:36:12 pm : Frame: VECTORED_6DOF
7/11/2025 12:36:12 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:36:12 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:36:12 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:36:08 pm : Frame: VECTORED_6DOF
7/11/2025 12:36:08 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:36:08 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:36:08 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:36:04 pm : DVL is too hot (48.53 C). Please cool it down.
7/11/2025 12:36:04 pm : Frame: VECTORED_6DOF
7/11/2025 12:36:04 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:36:04 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:36:04 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:36:00 pm : Frame: VECTORED_6DOF
7/11/2025 12:36:00 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:36:00 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:36:00 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:35:56 pm : Frame: VECTORED_6DOF
7/11/2025 12:35:56 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:35:56 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:35:56 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:35:52 pm : Frame: VECTORED_6DOF
7/11/2025 12:35:52 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:35:52 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:35:52 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:35:48 pm : Frame: VECTORED_6DOF
7/11/2025 12:35:48 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:35:48 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:35:48 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:35:44 pm : Frame: VECTORED_6DOF
7/11/2025 12:35:44 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:35:44 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:35:44 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:35:40 pm : Frame: VECTORED_6DOF
7/11/2025 12:35:40 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:35:40 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:35:40 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:35:36 pm : Frame: VECTORED_6DOF
7/11/2025 12:35:36 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:35:36 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:35:36 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:35:33 pm : Frame: VECTORED_6DOF
7/11/2025 12:35:33 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:35:33 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:35:33 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:35:29 pm : Frame: VECTORED_6DOF
7/11/2025 12:35:29 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:35:29 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:35:29 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:35:26 pm : Frame: VECTORED_6DOF
7/11/2025 12:35:26 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:35:26 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:35:26 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:35:25 pm : MYGCS: 255, heartbeat lost
7/11/2025 12:35:22 pm : Frame: VECTORED_6DOF
7/11/2025 12:35:22 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:35:22 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:35:22 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:35:18 pm : Frame: VECTORED_6DOF
7/11/2025 12:35:18 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:35:18 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:35:18 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:35:15 pm : Frame: VECTORED_6DOF
7/11/2025 12:35:15 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:35:15 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:35:15 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:35:11 pm : Frame: VECTORED_6DOF
7/11/2025 12:35:11 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:35:11 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:35:11 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:35:07 pm : Frame: VECTORED_6DOF
7/11/2025 12:35:07 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:35:07 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:35:07 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:35:07 pm : DVL is too hot (48.49 C). Please cool it down.
7/11/2025 12:35:04 pm : Frame: VECTORED_6DOF
7/11/2025 12:35:04 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:35:04 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:35:04 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:35:00 pm : Frame: VECTORED_6DOF
7/11/2025 12:35:00 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:35:00 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:35:00 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:34:56 pm : Frame: VECTORED_6DOF
7/11/2025 12:34:56 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:34:56 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:34:56 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:34:52 pm : Frame: VECTORED_6DOF
7/11/2025 12:34:52 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:34:52 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:34:52 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:34:48 pm : Frame: VECTORED_6DOF
7/11/2025 12:34:48 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:34:48 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:34:48 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:34:36 pm : MYGCS: 255, heartbeat lost
7/11/2025 12:34:24 pm : DVL is too hot (48.28 C). Please cool it down.
7/11/2025 12:34:15 pm : Lost manual control
7/11/2025 12:34:15 pm : u-blox 1 HW: 00190000 SW: EXT CORE 4.04 (7f89f7)
7/11/2025 12:33:58 pm : GPS 1: detected as u-blox at 230400 baud
7/11/2025 12:33:49 pm : DVL is too hot (48.18 C). Please cool it down.
7/11/2025 12:33:42 pm : GPS 1: detected as u-blox at 230400 baud
7/11/2025 12:33:14 pm : Frame: VECTORED_6DOF
7/11/2025 12:33:14 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:33:14 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:33:14 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:33:08 pm : Frame: VECTORED_6DOF
7/11/2025 12:33:08 pm : IMU0: fast sampling enabled 8.0kHz/1.0kHz
7/11/2025 12:33:08 pm : 159f341bb7f94b3e95496f20f3f15642
7/11/2025 12:33:08 pm : ArduSub V4.5.6 (1a196cab)
7/11/2025 12:33:08 pm : GPS 1: detected as u-blox at 230400 baud
7/11/2025 12:33:04 pm : EKF3 IMU0 MAG0 initial yaw alignment complete
7/11/2025 12:33:04 pm : EKF3 IMU0 tilt alignment complete
7/11/2025 12:33:02 pm : AHRS: EKF3 active
7/11/2025 12:33:02 pm : EKF3 IMU0 initialised
7/11/2025 12:33:00 pm : Lost manual control
7/11/2025 12:32:59 pm : AHRS: DCM active
7/11/2025 12:32:59 pm : ArduPilot Ready
7/11/2025 12:32:59 pm : Barometer 2 calibration complete
7/11/2025 12:32:59 pm : Barometer 1 calibration complete
2

Hi @Jono -
Oh no! What type of flight controller does your vehicle have - Pixhawk or Navigator?
I see an error that the DVL is too hot - have you been operating for a prolonged period on the bench with the enclosure sealed? This can lead to components overheating and not functioning as intended…

Sharing a BlueOS system log may help us get to the bottom of your issue - found under the gear icon in the lower left of BlueOS.

3

Hi Tony,

It is the Navigator board with Pi 4. Yes the DVL was hot but noting the log does not correlate to every reboot of the FC, this log was generated on the lab bench with electronics enclosure open in around 20c ambient. The problem is also occurring in the field with the DVL submerged.

Happy to share the system logs however as there is non-public spatial data potentially present, is there a way to do that privately?

4

Attached are ardupilot manager logs. This is a fresh install of the latest BlueOS image.

Looking though what are presumably logs from the recent bench failure (7 November) there is the following.

2025-11-06 23:11:09.609 | INFO     | logging:callHandlers:1706 - 127.0.0.1:33890 - "GET /v2.0/openapi.json HTTP/1.1" 200
2025-11-06 23:11:09.617 | INFO     | logging:callHandlers:1706 - 127.0.0.1:33892 - "GET / HTTP/1.1" 200
2025-11-06 23:11:09.627 | INFO     | logging:callHandlers:1706 - 127.0.0.1:33894 - "GET /swagger.json HTTP/1.1" 404
2025-11-06 23:11:38.513 | DEBUG    | commonwealth.mavlink_comm.MavlinkComm:get_most_recent_vehicle_id:102 - 1 (detected)
2025-11-06 23:11:38.514 | INFO     | commonwealth.mavlink_comm.MavlinkComm:set_system_id:28 - system_id set to: 1
2025-11-06 23:11:38.516 | ERROR    | commonwealth.mavlink_comm.VehicleManager:get_firmware_info:87 - Failed to request autopilot version. Received empty response
2025-11-06 23:11:38.517 | INFO     | commonwealth.mavlink_comm.VehicleManager:get_firmware_info:88 - trying to get a new system id
2025-11-06 23:11:38.536 | DEBUG    | commonwealth.mavlink_comm.MavlinkComm:get_most_recent_vehicle_id:102 - 1 (detected)
2025-11-06 23:11:38.537 | INFO     | commonwealth.mavlink_comm.VehicleManager:set_target_system:27 - setting target system to: 1
2025-11-06 23:11:38.540 | INFO     | logging:callHandlers:1706 - 127.0.0.1:34252 - "GET /v1.0/firmware_info HTTP/1.0" 500
2025-11-06 23:11:40.730 | INFO     | logging:callHandlers:1706 - 127.0.0.1:34304 - "GET /v1.0/vehicle_type HTTP/1.0" 200
2025-11-06 23:11:41.806 | INFO     | logging:callHandlers:1706 - 127.0.0.1:34336 - "GET /v1.0/firmware_vehicle_type HTTP/1.0" 200
2025-11-06 23:11:41.840 | INFO     | logging:callHandlers:1706 - 127.0.0.1:34354 - "GET /v1.0/board HTTP/1.0" 200
2025-11-06 23:13:58.781 | ERROR    | logging:callHandlers:1706 - Task exception was never retrieved
future: <Task finished name='Task-5' coro=<AutoPilotManager.auto_restart_ardupilot() done, defined at /home/pi/services/ardupilot_manager/autopilot_manager.py:190> exception=psutil.NoSuchProcess process no longer exists (pid=1655)>
Traceback (most recent call last):

  File "/usr/local/lib/python3.11/site-packages/psutil/_pslinux.py", line 1515, in wrapper
    return fun(self, *args, **kwargs)
           │   │      │       └ {}
           │   │      └ ()
           │   └ <psutil._pslinux.Process object at 0xb5479e88>
           └ <function Process.cmdline at 0xb61e3d98>
  File "/usr/local/lib/python3.11/site-packages/psutil/_pslinux.py", line 1635, in cmdline
    with open_text("%s/%s/cmdline" % (self._procfs_path, self.pid)) as f:
         │                            │    │             │    └ <member 'pid' of 'Process' objects>
         │                            │    │             └ <psutil._pslinux.Process object at 0xb5479e88>
         │                            │    └ <member '_procfs_path' of 'Process' objects>
         │                            └ <psutil._pslinux.Process object at 0xb5479e88>
         └ <function open_text at 0xb621f7a8>
  File "/usr/local/lib/python3.11/site-packages/psutil/_common.py", line 727, in open_text
    return open(fname, "rt", **kwargs)
                │              └ {'encoding': 'utf-8', 'errors': 'surrogateescape'}
                └ '/proc/1655/cmdline'

FileNotFoundError: [Errno 2] No such file or directory: '/proc/1655/cmdline'


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

> File "/home/pi/services/ardupilot_manager/autopilot_manager.py", line 193, in auto_restart_ardupilot
    needs_restart = self.should_be_running and not self.is_running()
                    │    │                         │    └ <function AutoPilotManager.is_running at 0xb567b348>
                    │    │                         └ <autopilot_manager.AutoPilotManager object at 0xb564fed0>
                    │    └ True
                    └ <autopilot_manager.AutoPilotManager object at 0xb564fed0>

  File "/home/pi/services/ardupilot_manager/autopilot_manager.py", line 184, in is_running
    and len(self.running_ardupilot_processes()) != 0
            │    └ <function AutoPilotManager.running_ardupilot_processes at 0xb567bb18>
            └ <autopilot_manager.AutoPilotManager object at 0xb564fed0>

  File "/home/pi/services/ardupilot_manager/autopilot_manager.py", line 506, in running_ardupilot_processes
    return list(filter(is_ardupilot_process, psutil.process_iter()))
                       │                     │      └ <function process_iter at 0xb61eb988>
                       │                     └ <module 'psutil' from '/usr/local/lib/python3.11/site-packages/psutil/__init__.py'>
                       └ <function AutoPilotManager.running_ardupilot_processes.<locals>.is_ardupilot_process at 0xb53e3c08>

  File "/home/pi/services/ardupilot_manager/autopilot_manager.py", line 502, in is_ardupilot_process
    if str(firmware_path) in " ".join(process.cmdline()):
           │                          │       └ <function Process.cmdline at 0xb61ea6b8>
           │                          └ psutil.Process(pid=1655, status='terminated', started='23:13:46')
           └ PosixPath('/root/.config/ardupilot-manager/firmware/ardupilot_pixhawk4')

  File "/usr/local/lib/python3.11/site-packages/psutil/__init__.py", line 694, in cmdline
    return self._proc.cmdline()
           │    │     └ <function Process.cmdline at 0xb61e3de8>
           │    └ <psutil._pslinux.Process object at 0xb5479e88>
           └ psutil.Process(pid=1655, status='terminated', started='23:13:46')
  File "/usr/local/lib/python3.11/site-packages/psutil/_pslinux.py", line 1522, in wrapper
    raise NoSuchProcess(self.pid, self._name)
          │             │    │    │    └ <member '_name' of 'Process' objects>
          │             │    │    └ <psutil._pslinux.Process object at 0xb5479e88>
          │             │    └ <member 'pid' of 'Process' objects>
          │             └ <psutil._pslinux.Process object at 0xb5479e88>
          └ <class 'psutil.NoSuchProcess'>

psutil.NoSuchProcess: psutil.NoSuchProcess process no longer exists (pid=1655)

Can you suggest where we should look from here?

ardupilot-manager.zip (178.0 KB)

5 
Are these logs likely to indicate an issue?

2025-11-07 00:15:39.796 | DEBUG    | mavlink_proxy.AbstractRouter:available_interfaces:76 - Available interfaces: [<class 'mavlink_proxy.MAVLinkRouter.MAVLinkRouter'>, <class 'mavlink_proxy.MAVLinkServer.MAVLinkServer'>, <class 'mavlink_proxy.MAVP2P.MAVP2P'>]
2025-11-07 00:15:39.817 | INFO     | autopilot_manager:current_sitl_frame:220 - Setting vectored as frame for SITL.
2025-11-07 00:15:39.818 | INFO     | flight_controller_detector.linux.detector:detect_boards:22 - Detecting Linux board: Navigator
2025-11-07 00:15:39.819 | INFO     | flight_controller_detector.linux.detector:detect_boards:22 - Detecting Linux board: NavigatorPi5
2025-11-07 00:15:39.820 | INFO     | flight_controller_detector.linux.detector:detect_boards:22 - Detecting Linux board: NavigatorPi4
2025-11-07 00:15:39.822 | INFO     | flight_controller_detector.linux.detector:detect_boards:24 - Detected Linux board: NavigatorPi4
2025-11-07 00:15:39.833 | WARNING  | autopilot_manager:start_ardupilot:575 - More than a single board detected: [NavigatorPi4(name='Navigator', manufacturer='Blue Robotics', platform=<Platform.Navigator: 'navigator'>, path=None, flags=[], devices={'ADS1115': (72, 1), 'AK09915': (12, 1), 'BME280': (118, 1), 'PCA9685': (64, 4)}), FlightController(name='SITL', manufacturer='ArduPilot Team', platform=<Platform.SITL: 'SITL_arm_linux_gnueabihf'>, path=None, flags=[])]
2025-11-07 00:15:39.834 | WARNING  | autopilot_manager:get_board_to_be_used:484 - Preferred board not set yet.
2025-11-07 00:15:39.835 | INFO     | autopilot_manager:start_ardupilot:578 - Using Navigator flight-controller.
2025-11-07 00:15:39.899 | DEBUG    | firmware.FirmwareInstall:_validate_elf:102 - firm_board: BoardType.LINUX, firm_sub_board: BoardSubType.LINUX_NAVIGATOR, current_decoder_platform: BoardSubType.LINUX_NAVIGATOR