Research on ROV visual autonomous obstacle avoidance navigation based on deep learning

General Discussion
,
22

Yup!

Nope!

23

How to use python script to control ROV navigation using Blue os system ? ( My goal is based on the visual ROV obstacle avoidance, if feasible, I want to use Blue os to extract OAK parsing video stream, and then through the path planning and control the ROV course ( up, down, left, right, rotation, acceleration, deceleration ) ) )

24

Hi @akang-211 -
You’ll need to develop a significant amount of software to make that work. Generally

  1. Python script receives and processes the Oak depth stream, identifying obstacles or making whatever decisions to create the desired “pilot” output.
  2. The script sends control signals to the autopilot via pymavlink
  3. The vehicle moves, and 1&2 are repeated as desired…
25

I have installed the rpi4, but I still can’t install the oakd extension. What’s the situation?

c7e5c79ccd940eabaf5c49592661429
c7e5c79ccd940eabaf5c495926614291920×1020 121 KB

26

Hi @akang-211 -
It looks like you are being blocked from reaching github. This can happen in some countries, or if your IP address has been blocked. I’d recommend trying from another source of internet, like a cell-phone hotspot.

Are you able to install any other extensions?

I can’t tell what version of BlueOS you’re running from the text in the lower left - is it not a standard stable or beta version?

27

I can’t install any extensions, I have Blue os 1.3.1 installed.

image
image539×194 24.9 KB

28

Hi @akang-211
The issue is likely your internet connection / IP being blocked. Can you try another source of internet?

Please verify from the terminal that you can ping the internet, like the google dns server:

ping 8.8.8.8
29

image
image389×364 4.81 KB

30

image
image802×517 12.2 KB

31

Hi @akang-211 -
It looks like you have internet access, but I still suspect a firewall is blocking you from reaching github. Can you try another source of internet?

32

Why does the video stream disappear after upgrading cockpit?

827d0bd6f7680be2b6f90e7a44261dc
827d0bd6f7680be2b6f90e7a44261dc1832×1002 61.7 KB

0656ec9b3c5a8e8f6e628eb847a4a05
0656ec9b3c5a8e8f6e628eb847a4a051117×452 23.5 KB

33

image
image1410×949 86.6 KB

I have installed the OAK extension, how can I get the video stream of Oak?
image
image679×1051 87.8 KB

34

I can see the video streams now. The RGB video stream of the OAK camera is displayed on my cockpit client, and the parallax video stream is displayed on the browser’s Blue os. How can I make them display together on the cockpit client?

3b6f6b4264ffdc2f2995bd0272b14b3
3b6f6b4264ffdc2f2995bd0272b14b31920×1034 100 KB

9026b0ec52eacee4f44fad18e273ecb
9026b0ec52eacee4f44fad18e273ecb1846×1000 68.3 KB

35

Hi @akang-211 -
Glad you resolved your video / configuration issues. You can learn more about how to configure extra video widgets here, and you can setup additional views to have different full screen or picture in picture configurations of the video widgets - see the earlier portions of that video.

36

013bba058062e233f4865ded96a09b3
013bba058062e233f4865ded96a09b31900×1002 116 KB

Thank you for your reply. I have successfully displayed three video streams in one window. Next, I want to run Depthai with OAK camera. I already have the code of depthai on my computer (and it has been implemented with OAK camera before). How can I use OAK camera to display the following pictures on my computer?
b37835deebef2cf249b3942e8f13bc6
b37835deebef2cf249b3942e8f13bc61909×547 112 KB

37

Hi @akang-211 -
You may find that you need to calibrate the camera once immersed, if the depth image ceases to function well.

It’s tough to recommend the best way to run the software you share without any links to an install process or description of what is being run - that looks like a windows program?

I googled depthai, and assuming your talking about the API provided by the camera manufacturer, it appears they have install instructions for Ubuntu. Setting up a github repository to build an extension, and adjusting the dockerfile to install the necessary dependencies and execute this install code can give you that software running on BlueOS. Fork this repository to get started! Then create a docker hub account and follow along with the extension documentation. If it makes the video streams you pictures available on a port it should be possible to embed this in Cockpit via an iFrame widget!

38

This thread may be of interest, to see and refer to what others have done previously.

If you’re wanting to draw custom annotations on the image then you’ll need to either do that on the camera/RPi side and encode the annotated images into the video stream that gets sent to the topside, or send the annotation data separately and use it to draw the desired annotations at the receiving end.

The latter approach is likely preferable in the long term, although it could be somewhat complicated at the moment in a software like Cockpit, because it’s currently lacking dedicated widget overlay support. That said, it should already be possible for an OAK camera BlueOS Extension to provide an annotation-drawing widget to Cockpit as an iframe with a clear background, which could then be manually moved and resized to fit over the relevant video stream widget :slight_smile: