Video freezing and grey frames in QGC 4.0.5 when recording

We have also noticed significant video lag in QGC 4.0.8 (latest version), we’ve not tested 4.0.5. It is significantly worse when using the Ping360 at the same time. We have reverted to using the beta build linked from the last beta forum post - https://ci.appveyor.com/api/buildjobs/854u10xslgj1ynf5/artifacts/QGroundControl-installer.exe
This build does not have the issues. It doesn’t seem to matter whether we have the new ‘Low latency’ mode on or off in 4.0.8. Obviously something has changed which is causing the increase in latency. It is basically impossible to pilot due to lag of 2 seconds.
This needs to be resolved going forward or we are stuck with the older builds.

QGroundControl is using Software decoding of h.264 which is crazy. Most hardware in the last few years, down to the cheapest cell phones, have hardware decoding abilities.

This issue is driving me nuts. I will spend some time this weekend trying to enable hardware decoding, but it may a long road. If any of the BlueRobotics guys or someone else with experience can give me a QGroundControl dev primer over Skype I would appreciate it.

The following guide deals with hardware decoding:
https://gstreamer.freedesktop.org/documentation/tutorials/playback/hardware-accelerated-video-decoding.html?gi-language=c

I suspect that overlay will be an issue in recording:
https://gstreamer.freedesktop.org/documentation/additional/design/subtitle-overlays.html?gi-language=c

@gwa-gwa Nice if your idea ends in a solution!
Regarding overlay at recording, i don´t see the problem.
Today that is done with subtitles and .ass files.

Though it might be a problem with the live overlay in QGC of depth, heading etc?
Bo

Can’t help but chuckle every time I see a .ass file :joy:. Getting the dev environment set up now. Will keep you posted.

Hi, we are tracking the issue here on github, this might be a better place for you to coordinate:

I am testing/trying to fix this problem myself presently. If you want to talk about it by voice, you can arrange a meeting with me by private message.

There is no overlay in recording, as it would require re-encoding the video.

Hello,

I’ve made a patch to make QGC use hardware decoding on Windows. This patch has fixed all of the latency and video grey screen glitches in the few cases we have tested so far. In order to merge this patch into stable, we need more testing and feedback. Information about the patch is here.

There is a windows installer for the test build here.

If you are having issues with the video performance in QGC 4.0 on Windows, please try the test build, and report your feedback. (A good thing to report beside the video performance is the CPU/GPU usage from task manager in the stable 4.0 version vs this test version).

2 Likes

I was having this problem, went back to 3.5…whatever of QGC and its fine. My GPU was only 35%, i thought maybe it was the awkward screen size of the toughbook. I’ll try this patch next week and see how it does.

This fixed my grey screen issue, but now objects become blocky once in a while when they move. Latency seems to be about half a second. Is this typical?

4.05 was giving me grey frames every 5 or seconds as well as ~1 second latency.
Just installed the test build (2020-10-07) that @jwalser linked to and I’m not having any issues with grey frames or latency.

1 Like

Hi there,
last weekend I did 1 and a half hour testddives and I think I can confirm that version @jwalser linked to solved the problem of lag and grey freezings during the dive. 4.0.5 and 4.0.8 still have this problems so I stay with this developer version that runs without any issues. Thanks for that!!

Chris

Hi!
Uppgraded to QGC 4.1.2 that seems to solve the problem.
In QGC application settings there now is a listbox setting for chosing the video decoding in QGC.

Using the standard “default” seems to be a mix of using the nVidia card and software decoder, in plain english the video freezing happens sometimes, even if processor load never goes over 45%.
I think there is a mix of software/GPU usage happening in this state.

Using “Force nVidia” gives good video with no freezing, but video mp4 recording is not working.
Sometimes there are a recorded file sized 0kB after quitting QGC

Using “Force DirectX 3D” seems to solve all problems, no freezing and good video recording!!
This seems to work also with a Windows 7 OS Laptop

1 Like

@jwalser , is that test build still available? The link isn’t working. Or have the changes been incorporated into the daily build?

I’m running an RTSP stream, which runs perfectly through VLC. In QGC 4.0.5 and the daily build, when the camera is stationary GPU use is about 20%, as soon as the camera moves, GPU activity drops to 0-1% and the video feed skips/gets blocky/lags etc.

Hi Marcus,

In Jacob’s ‘information about the patch’ link the last update is the PR got closed in December last year because

This seems to match Bo’s message that

ok thanks. I’m still seeing issues in the daily build with GPU activity dropping. Strangely, with some QGC sessions, GPU activity stays high, and the feed is fine, but most times it drops out as soon as the camera moves. It’s not a consistent fault, and I haven’t been able to find what might be causing it. Doesn’t seem to be improved by any of the hardware decoding options.

QGC has been added to the “Graphics Settings” list and set to high performance. And as I said, it works fine with VLC. It is a 4K RTSP feed, CPU runs at appx 65-70%, RAM runs at 25%. I would assume it’s a PC CPU issue, except that when the GPU activity remains high, the feed is excellent with no problems. So it is possible, just for some reason QGC is not always using the GPU

Have you tried using “Force DirectX 3D” in the video decoding section of the daily build, as per Bo’s suggestion above?

Also not sure if these other suggestions could be helpful.

yes, have tried every combo of settings and those suggestions, so its a bit of a mystery

Damn, never fun.

My main thoughts would be some kind of power saving ‘feature’, or just straight up throttling for cool down purposes if the GPU is getting too hot or something.

Have you tested moving the camera while viewing the stream with VLC, and looked at the GPU usage while doing so? I believe VLC does smoothing on dropped/damaged frames, whereas QGC just displays frames as soon as they arrive, and only smooths recorded videos. It’s possible it’s a signal noise or power issue when you move the servo, which VLC could just be hiding the results of.

I’m on a wired connection and this exact behavior has plagued me for awhile but seems to have gotten worse in newer versions. v20 seems to be where it’s most stable. Any ideas?

ETA: Ardusub 4.0.3 and QGC 4.0.5

Hi @Lucas,

I’ve moved your comment here because it’s more appropriate for your issue.

Assuming you’re on Windows please try updating to the latest version of QGC 4.1 - it integrated some video fixes as discussed in this thread. We’re in the process of making 4.1 the main release for everyone, but there have been a couple of issues with Mac integration that have held things back.

I got 4.1.4 working and video seems to be smooth. Thanks!

1 Like