I am trying to quantify the video lag by using a script running on the companion and/or the QGC computer. I have been searching through the forums but have not find a solution. There is a way to add the timeoverlay in the gstreamer options on the companion side, but the clock has to synced.
I am interested in computing the delay because I am experimenting with controllers, which can take into account the delay in the measurements. Since the position and the velocity are measured on the Topside computer using vision, therefore this delay is important. More delay in the measurements is introduced when processing the video stream on the Topside computer, which can be optimized.
Wondering if anyone has quantify the video lag by logging data? May be there is already an option in QGC. Thanks in advance for your help.
“Why are you interested? What are you trying to achieve?”
I am interested in computing the delay because I am experimenting with controllers, which can take into account the delay in the measurements. Since the position and the velocity are measured on the Topside computer using vision, therefore this delay is important. More delay in the measurements is introduced when processing the video stream on the Topside computer, which can be optimized.
“The video latency with our standard set up is ~200 milliseconds.”
The latency may vary over time. For example, it could be 201 ms or 199 ms, etc. Since, we compensating this delay at every timestamp, therefore there could be a method to know the latency when we receive the video stream
“To measure the latency, point the camera at a stopwatch/clock that is running on the QGC picture, and take a screenshot.”
Yes, approximate delay could be found by this method. But, I am looking for an exact method, which can give us the latency on the receiving end at every time stamp. There could be an example script which we run on the Topside computer to compute this delay. There may be a possibility to log this delay in QGC.
If you may please give me some hint or guidance to write this script, I think it will be a great share for all of us who may compute the video latency by running this script on the Topside computer. Thanks.
This is valuable information about your question that you should have included in the first place.
To be clear, this will give you an exact delay. With this new information about vision and control, I realize now that this method is not suitable for your application.
Do you want a log of the latency on each frame, or do you want the latency in real time for control?
This is certainly possible, however we don’t have anything built in at the moment. I’m not sure what the best approach is. It will require clock synchronization between the two systems, and I believe gstreamer will have the capability to achieve what you want.
Do you want a log of the latency on each frame, or do you want the latency in real time for control?
The latency is required for real time control, but can be logged as well separately. I am thinking if we may have a separate python script which can run on the topside computer and may be adding extra options in gstreamer using the web interface of companion, that will be the good start. While searching for solutions, I came across this work, which may be helpful (GitHub - stb-tester/latency-clock: Measure the latency of a video-output/video-capture combination).
Will look forward for more guidance. Thanks in advance for your help
Ok @barain, this is a significant problem, and I think you will need to continue researching the solution on your own. Pointing a camera at the monitor is the extent of my experience with this. I recommend you look at the gstreamer documentation (you will almost certainly need to get down and dirty with gstreamer), and that github example looks like a good resource.