Home        Store        Docs        Blog

Multiple cable penetrator leaks. Design flaw. (video)

My BlueROV2 has been plagued by leaks since I purchased it. We’ve had to abort several dives and use our backup ROV on multiple jobs due to severe leaks as soon as the BlueROV gets to depth.

The leaks have been extremely difficult to track down. They often don’t appear during vacuum tests or shallow test dives. Sometimes the ROV is fine for the first dive of the day, but the electronics enclosure would rapidly flood on the second dive after a battery swap.

We’ve finally traced the problem to the red cable penetrators. The potting epoxy is not bonding reliably with the anodized aluminum of the penetrator body. As a result, leaks can rapidly appear between the potting and metal if the cable is strained lightly.

We have had three penetrators fail in this fashion on our BlueROV2 Heavy in the last year. I’ve seen several photos of other user’s BlueROV2s with globs of epoxy added on the outside of the penetrators, presumably to address the same problem.

Bonding epoxy to aluminium has numerous documented challenges. These issues can be overcome by preparing the surface of the aluminum prior to bonding, but it doesn’t appear that Blue Robotics is doing any such preparations before potting.

When we make penetrators in house, we machine several grooves into the inside of the bore using a small metal lathe. We do this immediately prior to pouring the potting, and so far we haven’t had any failures.

It cost us dozens of hours and a couple of corroded speed controllers before we found this problem. I hope that Blue will make some improvements to your process to correct this issue.

In the meantime, do you have any recommendations for how users can repair leaking penetrators already in the field?

1 Like

Hi Travis,

Thanks for sharing all these details and for identifying ways to make the penetrators better! This is something we’ve been struggling with for a while now and we’re putting a lot of effort into improving it. What you’re sharing is obviously concerning and this is really important to everyone on our forum so I’m going to make this long and go into some detail on what we’re doing.

We’ve been making penetrators (mostly) the same way for several years now and we’ve had leaks from time to time, but it’s definitely become more prevalent in the last year or so and as our overall volume has increased. When we started seeing more cases last year we made a number of changes in our production to try to improve the situation - we started storing epoxy cartridges differently, adjusted the potting method, changed cable cleaning and preparation steps, and tried surface preparation and anodizing options. We may have made some minor improvements but still kept seeing issues.

It’s really hard to confidently validate what works and what doesn’t work here! It’s a statistical problem generally - say the failure rate is 1% overall and we want to change something to improve that. Since the failure rate is low we need to make tens or hundreds of samples and test them all exhaustively to confidently identify whether the changes make things better or worse! We didn’t have the capacity to do that initially, so most of the changes we made were implementing epoxy best practices that we could be fairly confident with. That didn’t make a big enough impact.

Since the end of 2019, we’ve realized that we need to solve this problem conclusively. We’d like to solve it as quickly as possible, but we’re focused on solving it properly with a long-term solution so that our products can be used with great confidence. We’ve redirected nearly all of our engineering resources to doing that since January. We’ve tested a lot, learned a lot, and made some good progress towards fixing this. It takes a lot of time to make progress on this due to the testing requirements.

From January through March we tested hundreds of penetrator samples in relatively controlled experiments varying things like surface preparation and epoxy cure cycle. These were tested through pressure cycles in our chamber, different amounts of thermal cycling, and physical cable strain. Initially we were having an extremely hard time getting anything to fail at all, even out of large sample sets, but we eventually identified that the epoxy loses a lot of its mechanical strength and adhesion at elevated temperatures. That could happen during shipping, while sitting out in the sun, or even while operating in warm waters (like 80F). Our current data points to this being the primary failure point and we have seen a higher failure rate in warm environments compared to cooler environments.

We have identified a few changes to the process that seem to make a strong positive improvement on this situation. We were about to start a series of tests to validate that before we make those changes in our production, but that was cut short by the COVID-19 crisis and our engineering team is unable to complete that testing until the “Safer from Home” orders end in California.

In the meantime, from home, we’ve still been focused primarily on this issue and we are investigating further options and design changes to fix this issue. We’ve made some promising progress there but we won’t be able to test that much until we’re back at the office, sadly! We hope to have a solution soon after that!

In the meantime, please let us know if you’re having issues and we can make the best recommendation and help you stay in the water. Our support team will be happy to help you with that. We’ve been doing the best to take care of any customers affected by this issue.

  • For now I would recommend avoid high temperature exposure when possible and, in particular, avoid putting strain on the cables while they are at high temperatures from sitting in the sun.
  • I would also recommend removing the vent plugs when transporting to avoid a combination of pressure build-up and temperature increase that could push outwards on the potting and trigger issues.

I do want to take a moment to address your solution. I don’t know conclusively if it will work better or worse, but I can at least share some things we’ve learned that are relevant to it!

  1. There’s a good chance that the bare machined aluminum will have better adhesion to the epoxy. You might be able to improve that further with the use of a primer like Reltek A-3 or Forsch Primer.
  2. I’d be cautious with the grooves inside, they do provide mechanical restraint, but they also create stress concentrations that could trigger delamination of the epoxy. We have seen some correlation between sharp corners and delamination in our testing. (That applies to the sharp corner at the top as well).

We’ve had some recommendations from other customers that you could try to repair these in the meantime. We haven’t tried these ourselves so they are unvalidated at the moment:

  • You can backfill the other end of the penetrator to make sure it is full of potting compound
  • If the cable has come loose entirely, you can reinstall the penetrator with machining (like you’ve shown) and/or a different potting compound. We’ve heard good things about E-90FL, JB Weld Plastic Welder, 3M DP-420, and 3M DP-620 from various customers.

We realize the importance of solving this issue so that you and other can count on our products! I can assure you that we are doing everything we can to solve this problem and back up that solution with scientific testing and results. We hope to share the results of all of our testing, successful and unsuccessful, when we are done in hopes that others can learn from it.

Thank you again for sharing your concern and your solution.

-Rusty

2 Likes

Thanks for an explanation Rusty,
Have a few questions

  • what depth are these leaks starting to appear (assuming mid 70F)
  • I would have thought those o rings we fit in penetrators would prevent leaks?
    Rgs
    Grant
    Oz

Hi Grant,

I got back to your email as well. To answer these questions:

  1. If the penetrator is strained or otherwise damaged while under warm conditions then it could cause a leak at shallow depths or during a vacuum test.

  2. The O-rings seal the penetrator body to the end-cap, but there is also a seal between the epoxy potting compound and the cable and penetrator. That second seal is what is affected here.

-Rusty

Thanks Rusty, yes read email reply :ok_hand:

Appreciated the quick response

Regards
Grant
Oz

Very interesting information. I was unaware that you were putting so much effort into solving this problem. I certainly understand that it can be difficult to replicate real world conditions and failure modes in the lab. I’m glad to know you’re working on it. I can say for sure that in our operating conditions these fittings are failing at a much higher than normal rate.

The heat issue is very concerning. We operate our ROV from a boat in Florida, and there is no practical way to keep it cool while transporting it offshore in the summer sun. We store and maintain all of our equipment in a warehouse that is kept cool with fans. No A/C. Additionally, the design of the BlueROV makes it nearly impossible to avoid putting some amount of strain on the penetrators.

That said, we have had zero leaks with cable penetrators from other manufacturers in these conditions. So this is a solvable problem.

Have you considered using a material other than aluminum for these things? I know that there is a lot of pressure to keep the overall cost down, and stainless steel commercial cable penetrators are $100+ each. But aluminum is a horrible material for chemical bonds…

What about some type of plastic like PVC or polypropylene? Maybe brass if plastic isn’t strong enough… Have you experimented with those at all?

Thanks for the post. Ive been struggling for a while with getting a vaccum on my unit… This explains the problem. we are working in 27C water and at 100m depths. Constantly getting failures.

Is there a good reason why you stopped potting the inside of the penetrator?

@StrikeLines - Thanks for understanding and for the suggestions! To be clear, if the epoxy is heated, it will return to its full strength and properties when cooled, but it’s susceptible to damage while it is warm.

We have considered a lot of design options including alternative materials like stainless steel or plastic. We did some testing but some of that got cut short by COVID. We’re pursuing some promising options that don’t have potting at all so we wouldn’t have to worry about those issues. It’s still early to say much on that.

@JoeB - I’m sorry you’ve had issues with failures in these conditions. Please feel free to reach out to our support team to get help making this right.

@Mikxie - Yes, in our original testing of the current potting compound we determined that there wasn’t a performance difference between potting both ends of the penetrator or just the top end. We’re looking into this again, however, as a few people have mentioned that they have had good luck by potting the inside.

Hi @StrikeLines,

Can you provide examples of the other cable penetrators you have used in the past? We are not are not familiar with many marine cable pass-throughs that are not full connectors.

-Adam

DOW BetaSeal 1527EP is used to fasten aluminum parts together on my aluminum chassis Lotus Elise. The aluminum appears to be anodized as well, but could be some other process. The parts are sometimes mechnically adhered using rivets, but what I’ve read is that those rivets are only useful for holding the parts until the product has cured.

1 Like

Adam,

You know, that’s a good point. I made an assumption there. I just looked over all our sonar equipment. In every case, the subsea electronics enclosures are accessed via rubber molded Subconn-style circular connectors.

This was the case for Klein, Edgetech, Benthos and Geoacoustics hardware. None of them used a epoxied pass-through cable penetrator.

Sonar tow cables are routinely terminated with epoxy molds to splice the cable to the connectors, and that’s what I was thinking about when I made that comment. But of course, that’s a totally different animal.