Retrieve Ping Sonar data for analysis

[EliotBR (Eliot)]

Hi @katlea,

@katlea

I would love to retrieve the bathymetry data and use it for analysis.
…
I have found the documentation for ping-python, but I cannot seam to convert the log file (in Binary format) into something i can use (like a .csv or something).

We have a sensor log decoding example in the Ping Viewer repository (since Ping Viewer creates those logs, and it defines the format).

If you’re specifically after just the distance measurements you could do something like the following (requires downloading “decode_sensor_binary_log.py”, and running this in your own python file from the same directory):

from decode_sensor_binary_log import PingViewerLogReader
from datetime import datetime, timedelta, time
from pathlib import Path
import csv

# TODO: put in the path to the log file you want to process
#  e.g. "~/My Documents/PingViewer/Sensor_Log/something.bin"
logfile = Path("/path/to/file.bin")

def to_timedelta(time_str: str) -> timedelta:
    ''' Returns a time delta from an iso-format time string. '''
    delta = time.fromisoformat(time_str)
    return timedelta(hours=delta.hour, minutes=delta.minute,
                     seconds=delta.second, microseconds=delta.microsecond)

log = PingViewerLogReader(logfile)
outfile = Path(logfile.stem).with_suffix(".csv")
# ideally it would be good to localise this to your timezone,
#  but in this case it shouldn't cause issues unless you were
#  recording the sonar data at the time of a daylight savings
#  changeover or something
start_time = datetime.strptime(logfile.stem, "%Y%m%d-%H%M%S%f")

with outfile.open("w") as out:
    csv_writer = csv.writer(out)
    csv_writer.writerow(("timestamp", "distance [mm]", "confidence [%]"))
    for timestamp, message in log.parser():
        # convert the 'duration since scanning started' into a local-time timestamp
        #  the .replace here ensures cross-compatibility between operating systems
        timestamp = start_time + to_timedelta(timestamp.replace('\x00',''))
        csv_writer.writerow((timestamp, message.distance, message.confidence))

If you’re interested in more involved analysis (or just extraction) of the profiles themselves then the code will need to be more involved, and a csv format may not be suitable. This thread may be of interest.

I’m planning to make a detailed post-processing example including synching Ping Viewer log data with other things like telemetry logs and video, but haven’t had the time to do so yet unfortunately.

@katlea

Ideally, I want to have a value for each image of the bottom I analyse.

Out of interest, how are your videos/frames saved? You’ll need some way to align the video with the sonar scan data, and depending on what you actually want to do with it you may need to

• use a program to extract video frames and
  • include the corresponding distance+confidence in each filename
  • write the distance+confidence as text on each frame (which could be saved individually or as a new video)
• write the distance+confidence information into a subtitle file, which could then be played together with the video (without permanently modifying the frames)

Note also that

• the video and sonar likely won’t have the same update rate, so you’ll need some way to get the nearest value for each frame
• it may be relevant to correct for tide levels in the distance values, especially for inspections that are happening during a changing tide
• if your video is from a reasonably steady boat you may be able to stitch together the frames into one large image of the full inspected area, and potentially plot the bathymetry results on top of that (that would likely be somewhat involved, but is definitely an interesting application)
  • alternatively it may be possible to use photogrammetry to create a 3D model, which could be cross-referenced with the sonar data