Hi @vantitech, welcome to the forum 
ArduRover is the ArduPilot firmware variant with boat support, and is used on our BlueBoats. For a custom vehicle, you may wish to go through ArduRover’s boat configuration documentation
What control station software are you trying to use? They’re not all well set up for boats.
We currently recommend QGroundControl for controlling our BlueBoats. Cockpit has work-in-progress boat support, but its listed MAVLink Manual Control joystick button functions only actually work for ArduSub (they don’t have firmware support in ArduRover), which can be confusing. Both QGroundControl and Cockpit support defining custom MAVLink commands that can be triggered from joystick buttons or onscreen ones. I believe Mission Planner has similar functionality, but I’m less familiar with it.