Ahh ok, so you mean that “the camera/AI chip has firmware that can be updated, and we plan to release updates/improvements for it over time”, rather than my assumption/understanding that “the camera’s AI tunes itself to create a better image/output over time”?
I think I misunderstood your meaning there because “continuously” was applied to the “software” (e.g. “the software updates the camera performance continuously”). Perhaps it would be more clear to say something like “We are continuously working on and releasing software updates that improve the camera performance” (which is basically what you said in your follow-up)
Regardless, cool that the plan is to improve the units you sell now with software updates in future, as your algorithms get better and you gain more colour science experience
I am not sure what exactly you are talking about with IP67, the enclosure is rated at 400 meters in depth. An IP67 would not allow the camera to be submerged, only slight protection against water.
While we thought of these other communication protocols, they were not chosen because they would rely on the Raspberry Pi to do the onboard encoding, which I am sure as you know, the quality isn’t that great since they don’t have a dedicated video encoder. We wanted a camera that could easily be added to the ArduSub system while providing the great image quality of an on board image processor. We do have plans to work on other cameras such as ethernet
I see that now the Jetson is capable of supporting GMSL but it would run into similar issues as the Pi. Using an FPGA for your entire image processing is just ufnrountately not the most efficient setup compared to our approach and Jetson would require a whole redesign of your on board system currently offered by Blue.
In terms of reliability, our system currently uses the same setup as Blue (Raspberry Pi + Pixhawk) and we stream 3 cameras at once and had no issues with reliability even after extended use! When you have the cameras setup and connected securely, the video stream is pretty reliable!
Our current exploreHD is plug and play at least the newest firmware will allow that. For the exploreHD, all you need to do is to put it through a cable penetrator and plug in the camera via USB on the Pi. With the new version of ArduSub Companion Beta, you will be able to stream 3 at a time giving you multiple angles of your ROV.
For the topside computer, our software team is developing a method to view multiple stream with QGround. But for current customers of exploreHD, they just use OBS with the gstreamer plugin.
If you are talking about a replacement for the USB camera used on the BlueROV, that’s possible too but we haven’t had the time to design a holder for that since right now our best selling product is the exploreHD and we are focus on making the firmware as easy to use as possible! Additionally, when we tried using our camera with the acrylic dome cap, we weren’t able to get as sharp of an image as compared to the exploreHD with its thinner quartz glass element so we been recommending that more for better image quality
@DeepWaterExploration Thanks. I was talking about replacing the stock camera. I was hoping that you sold the camera without the housing and had a mounting kit that bolts on the BlueROV2 servo mount. I’m not a fan of the stock camera since it’s always blurry or pixelated. I agree that you need to focus on the priorities that are the most important and I will consider your products when QGroundControl gets reliable multi-camera support.
Yeah we do sell the camera by itself but we don’t have a cad yet for mounting on the tilting servo. I am not sure if its possible to get the same clarity though because of the thick acrylic needed for the end cap. We will do more testing and let you know!
So I can order your camera, mount it externally on my bluerov2, bring the cable inside the electronics tube, connect it to the USB socket instead of the current camera, and I would immediately have the video stream in QGroundControl without any other hardware / software changes?
Yes, the new version of our camera’s firmware will allow you to do that. If you do want to experiment with the settings you can do so by installing our driver (on our GitHub) and running a for loop start command every time the pi starts but our default settings are what we tested that works the best in terms of best quality with lowest lag/latency.
Marco,
I removed the internal camera and dome, then used one of the Blue Trail servos for the external camera. It plugged right into the same port on the Pixhawk where the internal servo was. Works great! I did program the Blue Trail servo to adjust the tilt limits.
I’m also looking forward to the new DW cams to add more views!
Are you considering building a camera with higher resolution? I currently use the GoPro in 2k mode (2704 x 1520 resolution). I don’t know if that’s possible with a USB camera, but if so, I’d definitely consider it as a replacement for what we are doing.
Hello! Although we thought of higher resolution, higher resolution also means decrease performance in low light (since each pixel will be physically smaller thus less sensitive to light).
Also from our experience with underwater video technology, we found that the color and sensitivity is more important than the resolution since its a bit tough to get image looking sharp underwater given the murkiness of the water.
Although GoPros are great for underwater sports videography, in ROV applications their limitations do show. Having to use a separate SD card to record and the inability to receive a live feed on the surface are real limitations. Additionally, because of their rectangular housing, its much more challenging to waterproof for high depth use. That’s where we think our camera really shines! Its not a replacement for GoPro altogether but more so for ROV applications!
Oh, believe me, I’m well aware of GoPro limitations!
Higher resolution is definitely helpful in some applications. Trying to identify species seen is my area and 2K resolution is much more helpful for seeing small details in animals, seaweed, etc. I’d go with 4k if I could, but 2K seems to be a good balance between resolution and video file size and post-processing ability. Plus the GoPros get really, really hot at 4k (from my experience with Hero 6s) and die after an hour or so of recording.
My ideal camera would be something that’s in the 2-4k range (with the option to switch resolution as desired), but that can be controlled (and processed) by a Raspberry PI or some other SBC. Even if the video is stored on the camera, but can be easily accessed by the SBC after the fact.
I’m not sure if HD video will cut it for what I’m trying to do (but I’d be happy to try one of your cameras to see if other factors would balance out the loss of resolution ).
We just released the new updated firmware for the exploreHD All current exploreHD owners can update the camera free of charge using the instructions on our website!
This update is for cameras shipped before 11/20/2021. Any exploreHD shipped after that date does not need to be updated.
I’ve previously posted about the OAK-D-Lite, although the OAK-1 might be a better fit for your spec. They can both do 4k60, and are ML/AI capable but don’t come with any particular algorithm built in, and aren’t waterproof (so would need to be used inside an enclosure). OAK-D-Lite can do stereo depth, but has a lower quality camera than OAK-1, so OAK-1 would have better low-light performance. That said, they’re not optimised/primarily intended for underwater/low-light use, and the smaller physical pixel size does mean they won’t have as good low light performance as the IMX322 sensor used in both the Blue Robotics Low-Light Camera, and this topic’s one (exploreHD) from Deep Water Exploration.
I would note that while OAK-1 and OAK-D can do 4K60, they’re intended to be operated with a USB-3 interface, so to get that resolution you may need to use an RPi4 (since RPi3 only USB-2 ports). That said, because the cameras are intended for user control you should be able to set up a 1080p30 stream and just zoom in on areas of interest, either manually or with an ML algorithm that detects interesting stuff and zooms in to it, and in that case you can likely get away with using a USB-2 interface.
Very cool! Besides not being waterproof, I believe these cameras also aren’t plug and play with ArduSub Companion. Also, given the small sensor size, I highly doubt these can achieve the same sharpness as GoPros especially for underwater use.
For 4K cameras, I think the best option may be ROV ethernet cameras. We are working on one that uses similar true color tech as the exploreHD but we currently don’t have an update on those. Other than that I would suggest you can look into ethernet ROV cameras for the easiest way to implement into your current ROV setup.
That’s true, although because they’re intended to run custom code the idea isn’t really to be plug and play to start with. That said, they do support on-board H264 encoding, so the result shouldn’t be overly difficult to connect to Companion. I have a personal interest in the system because I enjoy working with computer vision, so will likely be making a guide/post on how to set up Companion integration once Companion 1 is more stable, and once I’ve got some more experience with the camera(s)
That’s very possible, and is something that would need to be tested, and considered before deciding to use those cameras. That can perhaps be compensated for somewhat with frame stacking and pixel binning, but those approaches still rely on at least some light being detected, which is where larger pixels really make a difference. Then again, many scenarios already have decent lighting, and/or can use lighting from the vehicle, so low-light performance is not the only important criteria for a camera, and perhaps the other benefits can make up for that limitation in certain use-cases (especially where custom processing is important)
For general imaging I would probably agree - using IP cameras avoids the stream needing to go via the Companion computer (so doesn’t use up ports or require specific setup there), and an Ethernet Switch can be used to connect one or multiple extra cameras to the topside. That does unfortunately mean those cameras aren’t recognised by QGroundControl so need their own viewing/recording interface, but there are tradeoffs with every approach.
Interesting that you’re working on one - no doubt people will get good use out of it
Yeah, the H264 encoding on board will definitely be interesting but 4K data is a lot so I wonder how Companion can be modified to handle that. Either way still cool but may take some work
If there is an easy way to implement 4K H264/265 on Companion, we would definitely be interested in making a USB module for it too!
Stacking and pixel binding are interesting approaches but my main concern is the amount of image noise from a smaller sensor due to tough underwater situations. Smartphones tend to deal with this by adding noise reduction and digital sharpening later on. Even with bright lighting, a smaller sensor will produce noise in the shadows which is why no smartphone sensor can’t beat an actual camera with a much larger sensor when both are recording at the same resolution. Interesting enough, most of the times, 1080P on an actual camera with a larger sensor will look way sharper and more detailed than 4K on a smartphone which I assume will be similar here. The image samples provided on their website don’t look too sharp to me but maybe those are just the initial revision
