Can't run GStreamer commands with OpenCV (Windows)

Hi,

I am trying to get the camera view from my RaspberryPi’s (ArduSub Companion Software) USB camera using GStreamer and OpenCV.

GStreamer commands work fine when I run in the command prompt but it fails when I try to get the video from OpenCV.

I build the OpenCV from source with GStreamer support and I confirmed it working by getting a successful test result from the following code:

import cv2
gstreamer_str = 'videotestsrc ! videoconvert ! appsink'
cap = cv2.VideoCapture(gstreamer_str, cv2.CAP_GSTREAMER)
print(cap.isOpened())
while cap.isOpened():
    ret, frame = cap.read()
    cv2.imshow('Testing Frame', frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
cv2.destroyAllWindows()
cap.release()

I also get the camera view from GStreamer using the default ArduSub Companion camera stream settings and running the following code on the command prompt:

gst-launch-1.0 udpsrc port=5600 ! application/x-rtp,media=video,clock-rate=90000,encoding-name=H264,payload=96 ! rtph264depay ! avdec_h264 ! autovideosink

But I can’t get it to work with OpenCV. I do not have much experience on neither GStreamer nor OpenCV so any kind of help is highly appreciated.

Thanks.

Hi @salihozdemir, welcome to the forum :slight_smile:

To clarify:

  1. What gstreamer command are you running from your code?
  2. Have you remembered to change the autovideosink to appsink so OpenCV can receive the frames (in order to process and/or display them)?
  3. Does the command specified in this example work?
    'udpsrc port=5600 ! application/x-rtp, payload=96 ! rtph264depay ! decodebin ! videoconvert ! appsink'
    

I am trying to run the following command:

udpsrc port=5600 ! application/x-rtp,media=video,clock-rate=90000,encoding-name=H264,payload=96 ! rtph264depay ! avdec_h264 ! appsink

But I need to mention that some commands just gave me an error saying that pipeline could not be constructed. But some commands just do not give any error, the terminal just stays blank and I do not get any type of response from the code.

So in my opinion it seems like OpenCV command cv2.VideCapture() is the root of the problem.

Yes, I am using autovideosink on the command prompt and appsink on the OpenCV.

Not with OpenCV. But it works on command prompt after changing appsink to autovideosink and adding gst-launch-1.0 in front of it.

So to you keep updated on the situation, here’s what I will be doing:
Since I’m working on Windows and it’s a bit problematic with these libraries (I had to build OpenCV myself just for it to support GStreamer) I will change my approach.
Instead of trying to catch the image on OpenCV with GStreamer built in it, I will try to use PyGObject to get the image directly from that and I will use OpenCV only for interface purposes.

I will let you know of the results, thanks.

If the pipeline can’t be created then you’re probably missing some gstreamer libraries or something - perhaps your OpenCV was built with a different gstreamer to the one you’re using to run your own pipelines in the terminal?

OpenCV does not have pre-compiled binaries with built in gstreamer support, so it’s always necessary to build OpenCV yourself if you want to use it with a gstreamer backend.

hi, I can run your python code but its C++ version , but I can not run below code on the command prompt:

gst-launch-1.0 udpsrc port=5600 ! application/x-rtp,media=video,clock-rate=90000,encoding-name=H264,payload=96 ! rtph264depay ! avdec_h264 ! autovideosink

can you tell me how do you configure your environment with gstreamer in windows ? My gstreamer is 1.18.2,opencv 4.4.0 . Thank you so much!