How to record and save camera data

I have an exploreHD USB camera and a Raspberry Pi 4b. How can I record and save the camera data in an environment without internet access?

Hi @sim-
Welcome to the forums!

Is your Pi standalone, or part of an ROV system? There are lots of ways to accomplish recording, what is the goal of your application?

The DashCam BlueOS extension can record the exploreHD stream automatically when the system is armed, if the video stream has been setup as an RTSP stream (and not UDP.) If you’re not running BlueOS with an attached Navigator, in a vehicle control paradigm, than this isn’t going to help you.

If you’re simply trying to record the stream to disk on a standalone Pi, a LLM AI prompted to provide code to record an H264 stream from /dev/video2 to disk with gstreamer or ffmpeg should get you going!

Internet access should not be required once you get things working, and all dependencies installed for whatever code you’re running.

Hi @Tony thanks for the pointers!

I’m planning to use a standalone Raspberry Pi (no Navigator/ROV control stack). My goal is to record a USB camera feed directly to the microSD card with no internet connection, and have it start recording automatically on power-up.

Could you share a concrete, step-by-step guide for doing this on a standalone Pi?

Hi @sim -

Take this with a grain of salt, as it is the prompt and output of LLM AI (chatgpt5). I’ll assume that you use the Raspberry Pi Imager to flash an SD card for your Pi with Raspian. If you test this and have issues, feed whatever the issue / error message is to the LLM AI, along with the code used, to resolve your issue.

Prompt:

Provide python code to record an H264 stream from /dev/video2 to disk with gstreamer or ffmpeg, running on a Raspberry PI 4 running standard Raspian 32 bit image. The recording should start when the system finished booting, and handle the video file to avoid corruption when power is cut. If the recording approaches using up all the free space on the SD card, it should delete the oldest recording. Perhaps recording in 1 minute chunks is the best way to approach this? Provide setup instructions to install dependencies as well.

Response:

Got it. Below is a dead-simple, robust setup that:

  • records /dev/video2 (already H.264) continuously,

  • writes 1-minute segments (so a power cut only risks the current minute),

  • auto-prunes oldest files when free space gets low, and

  • starts on boot via systemd.

I’m showing an ffmpeg solution (most bullet-proof here). At the end I include an equivalent GStreamer pipeline you can swap in if you prefer.


1) Install dependencies (Raspberry Pi OS 32-bit)

sudo apt update
sudo apt install -y ffmpeg python3 python3-venv
# Ensure the runtime user can read the camera:
sudo usermod -aG video pi

Log out/in (or reboot) so the video group takes effect.


2) Create a recording directory

sudo mkdir -p /var/lib/camrec
sudo chown pi:pi /var/lib/camrec


3) Python recorder (/opt/camrec/camrec.py)

This script launches ffmpeg to copy the H.264 bytestream straight from the v4l2 device into MKV segments (MKV is resilient and doesn’t need an index flush like MP4). After each segment is written, it checks disk space and deletes the oldest segments until your free-space target is satisfied.

sudo mkdir -p /opt/camrec
sudo chown -R pi:pi /opt/camrec
nano /opt/camrec/camrec.py

Paste:

#!/usr/bin/env python3
import subprocess, time, signal, sys, shutil, pathlib, os, glob

# ====== CONFIG ======
DEVICE = "/dev/video2"                  # H.264 v4l2 device
OUTPUT_DIR = pathlib.Path("/var/lib/camrec")
SEGMENT_SECONDS = 60                    # 1-minute chunks
FILENAME_PATTERN = "%Y%m%d_%H%M%S.mkv"  # strftime via ffmpeg
MIN_FREE_GB = 2.0                       # keep at least 2 GB free (tune!)
CHECK_GLOB = "*.mkv"                    # recorded files pattern
LOG_PATH = "/var/lib/camrec/recorder.log"
FRAMERATE = 30                          # hint only; v4l2 may ignore
# ====================

stop_requested = False

def log(msg: str):
    ts = time.strftime("%Y-%m-%d %H:%M:%S")
    with open(LOG_PATH, "a") as f:
        f.write(f"[{ts}] {msg}\n")

def free_gb(path: pathlib.Path) -> float:
    total, used, free = shutil.disk_usage(str(path))
    return free / (1024**3)

def prune_if_needed():
    """Delete oldest recording files until we have MIN_FREE_GB free."""
    try:
        while free_gb(OUTPUT_DIR) < MIN_FREE_GB:
            files = sorted(OUTPUT_DIR.glob(CHECK_GLOB), key=lambda p: p.stat().st_mtime)
            if not files:
                log("No files left to delete but free space still low.")
                break
            oldest = files[0]
            try:
                oldest.unlink()
                log(f"Deleted oldest file to free space: {oldest.name}")
            except Exception as e:
                log(f"Failed to delete {oldest}: {e}")
                break
    except Exception as e:
        log(f"prune_if_needed error: {e}")

def build_ffmpeg_cmd():
    # We copy the already-encoded H.264 bitstream from /dev/video2
    # Use segment muxer; rotate every SEGMENT_SECONDS; timestamped filenames.
    # -fflags +genpts and -use_wallclock_as_timestamps help with PTS on raw v4l2.
    # MKV is robust; if you strongly prefer TS: use -f segment -segment_format mpegts and .ts suffix
    location = str(OUTPUT_DIR / f"%{FILENAME_PATTERN}")
    return [
        "ffmpeg",
        "-hide_banner",
        "-nostdin",
        "-f", "v4l2",
        "-thread_queue_size", "512",
        "-input_format", "h264",
        "-framerate", str(FRAMERATE),
        "-i", DEVICE,
        "-c", "copy",
        "-fflags", "+genpts",
        "-use_wallclock_as_timestamps", "1",
        "-f", "segment",
        "-segment_time", str(SEGMENT_SECONDS),
        "-segment_atclocktime", "1",
        "-reset_timestamps", "1",
        "-strftime", "1",
        location
    ]

def handle_signal(signum, frame):
    global stop_requested
    stop_requested = True
    log(f"Signal {signum} received; stopping...")
    # Let ffmpeg exit gracefully on SIGTERM via process.terminate below.

def main():
    OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
    log("Recorder starting.")
    signal.signal(signal.SIGTERM, handle_signal)
    signal.signal(signal.SIGINT, handle_signal)

    while not stop_requested:
        cmd = build_ffmpeg_cmd()
        log(f"Launching ffmpeg: {' '.join(cmd)}")
        try:
            proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, bufsize=1)
        except Exception as e:
            log(f"Failed to start ffmpeg: {e}")
            time.sleep(5)
            continue

        # Read ffmpeg output line by line; when a segment closes, ffmpeg prints a line.
        # We can't rely 100% on parsing; instead, periodically prune.
        last_prune = 0.0
        try:
            for line in proc.stdout:
                if not line:
                    break
                # Keep the log file modest by trimming noisy lines occasionally
                if "Opening" in line or "segment:" in line or "frame=" in line:
                    log(line.rstrip())

                # Prune roughly once per segment or after ~30s
                now = time.time()
                if now - last_prune > max(10, SEGMENT_SECONDS // 2):
                    prune_if_needed()
                    last_prune = now

                if stop_requested:
                    break
        except Exception as e:
            log(f"ffmpeg stdout loop error: {e}")

        if stop_requested:
            # Ask ffmpeg to exit then wait
            try:
                proc.terminate()
                proc.wait(timeout=5)
            except Exception:
                proc.kill()
            break

        # If ffmpeg exited unexpectedly, restart after a short delay
        rc = proc.poll()
        log(f"ffmpeg exited with code {rc}; restarting in 3s.")
        time.sleep(3)

    log("Recorder stopped.")

if __name__ == "__main__":
    main()

Make it executable:

chmod +x /opt/camrec/camrec.py


4) systemd service (start at boot, restart on crash)

sudo tee /etc/systemd/system/camrec.service >/dev/null <<'EOF'
[Unit]
Description=H264 camera recorder (segmented) for /dev/video2
After=multi-user.target
Wants=multi-user.target

[Service]
Type=simple
User=pi
Group=pi
ExecStart=/usr/bin/env python3 /opt/camrec/camrec.py
Restart=always
RestartSec=3
# Ensure we can open the device and write logs/segments
WorkingDirectory=/var/lib/camrec
# Give it time to flush/close on stop
TimeoutStopSec=10

[Install]
WantedBy=multi-user.target
EOF

Enable & start:

sudo systemctl daemon-reload
sudo systemctl enable --now camrec.service

Check status & logs:

systemctl status camrec.service
tail -f /var/lib/camrec/recorder.log


Notes & tuning

  • Container format: MKV is chosen for resilience. If you prefer MPEG-TS (also robust, very simple), change:

    • FILENAME_PATTERN = "%Y%m%d_%H%M%S.ts"

    • Replace -segment_time ... -strftime 1 ... line’s filename suffix to .ts

    • Add -segment_format, "mpegts" right before the location.

  • Disk guard: Adjust MIN_FREE_GB. You can also implement a max total size policy instead; this free-space guard is simple and effective.

  • Frame rate: If your v4l2 device has a fixed framerate, you can drop -framerate. If you know resolution, you can add -video_size 1920x1080 after -framerate.

  • Power cuts: Only the current 1-minute file is at risk. All previous segments are closed and should be fine.


(Optional) GStreamer equivalent

If you prefer GStreamer splitting (also very robust), you can swap the ffmpeg command with splitmuxsink:

sudo apt install -y gstreamer1.0-tools gstreamer1.0-plugins-{base,good,bad}

Test pipeline (one-liner):

gst-launch-1.0 -e v4l2src device=/dev/video2 ! video/x-h264,stream-format=byte-stream,alignment=au ! h264parse config-interval=1 ! splitmuxsink max-size-time=60000000000 location="/var/lib/camrec/%Y%m%d_%H%M%S.mkv"

To use this from the same Python wrapper, change build_ffmpeg_cmd() to build_gst_cmd() returning:

return [
    "gst-launch-1.0", "-e",
    "v4l2src", f"device={DEVICE}",
    "!", "video/x-h264,stream-format=byte-stream,alignment=au",
    "!", "h264parse", "config-interval=1",
    "!", "splitmuxsink",
    f"max-size-time={SEGMENT_SECONDS * 1000000000}",
    f"location={OUTPUT_DIR}/%Y%m%d_%H%M%S.mkv"
]

And adjust the Python script to call that instead of ffmpeg (the pruning logic stays the same).


Quick sanity test

  1. Run the script manually once:

    /opt/camrec/camrec.py
    
    

    Stop with Ctrl+C after a minute and check you see timestamped .mkv files growing in /var/lib/camrec.

  2. Start the service and confirm new segments appear each minute.


That’s it. You’ll have an always-on recorder that self-manages disk usage and avoids corrupted long files on sudden power loss.

2 Likes

Hi @Tony

I’m running BlueOS on a standalone Raspberry Pi (no Navigator / no vehicle control stack), and I want to record a USB camera directly to the microSD card with no internet connection, starting automatically on boot.

Could you share a BlueOS-specific, step-by-step guide for this?

Hi @sim -

The same instructions would apply! You may want to package the code as an extension eventually, but in order to use BlueOS extensions that achieve similar recording functionality (DashCam) you would need the system to have a Navigator, so it could be armed. You may want to investigate modifying that extension (open-source) to simply record on start-up, rather than when the vehicle state is armed?

Generally, a Raspberry Pi is a power hungry choice for a recording camera system - can you share more about your application?

Hi Tony, I’m doing something similar, except I am using a Blue Boat. I want to grab stills from the exploreHD USB camera and save the time and coordinates from the GPS in the exif data (basically geotagged images every n seconds). I also tried the ChatGPT approach, which produced something similar. Before I start tinkering with my working Blue Boat, how likely am I to brick it :rofl:

I’ve pasted the ChatGPT response below, just in case anyone else is interested.

Option A — Simple, robust Python service (recommended)

1) One-time prep in BlueOS

  1. Expose MAVLink to localhost so a program can read GPS:
    BlueOS → MAVLink Endpoints → ā€œAdd UDP Serverā€ → 0.0.0.0:14550. This lets local apps subscribe to the vehicle’s MAVLink stream. Blue Robotics

  2. Note your camera device (usually /dev/video0) in Camera → Video Streams. (BlueOS Camera Manager handles streaming; we’ll read the UVC device directly for stills.) Blue Robotics

  3. In a shell on the Pi (BlueOS → Terminal), install dependencies:

sudo apt-get update
sudo apt-get install -y python3-opencv python3-pip exiftool
pip3 install pymavlink

2) The script

Save as /home/pi/timelapse_exif.py:

#!/usr/bin/env python3
import cv2, time, os, subprocess, datetime
from pymavlink import mavutil

# Settings
DEVICE = "/dev/video0"        # UVC device for exploreHD
INTERVAL_SEC = 5              # change as needed
OUT_DIR = "/data/timelapse"   # persistent on BlueOS; adjust if desired
RESOLUTION = (1920, 1080)     # exploreHD is 1080p; adjust if you like
MAVLINK_ENDPOINT = "udp:127.0.0.1:14550"

os.makedirs(OUT_DIR, exist_ok=True)

# Connect to MAVLink
m = mavutil.mavlink_connection(MAVLINK_ENDPOINT)
m.wait_heartbeat(timeout=10)  # ensure we’re receiving telemetry

# Open camera
cap = cv2.VideoCapture(DEVICE)
cap.set(cv2.CAP_PROP_FRAME_WIDTH,  RESOLUTION[0])
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, RESOLUTION[1])

def current_gps():
    # Prefer GLOBAL_POSITION_INT (scaled by 1e7), fallback GPS_RAW_INT
    msg = None
    for _ in range(20):
        msg = m.recv_match(type=['GLOBAL_POSITION_INT','GPS_RAW_INT'],
                           blocking=True, timeout=0.1)
        if msg:
            break
    if not msg:
        return None

    if msg.get_type() == 'GLOBAL_POSITION_INT':
        lat = msg.lat / 1e7
        lon = msg.lon / 1e7
        alt = msg.alt / 1000.0  # meters
        fix_time = datetime.datetime.utcnow()
    else:
        # GPS_RAW_INT: lat/lon may be NaN for no-fix; use m.lat, m.lon *1e7 only if valid
        if msg.lat == 0 and msg.lon == 0:
            return None
        lat = msg.lat / 1e7
        lon = msg.lon / 1e7
        alt = (msg.alt/1000.0) if msg.alt != 0 else 0.0
        # GPS_RAW_INT has time_usec
        fix_time = datetime.datetime.utcfromtimestamp(msg.time_usec/1e6)
    return lat, lon, alt, fix_time

def exif_gps_args(lat, lon, alt, dt):
    # Build exiftool args for GPS and timestamp
    lat_ref = 'N' if lat >= 0 else 'S'
    lon_ref = 'E' if lon >= 0 else 'W'
    # exiftool accepts signed values + Ref tags
    ts = dt.strftime('%Y:%m:%d %H:%M:%S')
    return [
        f"-GPSLatitude={abs(lat)}", f"-GPSLatitudeRef={lat_ref}",
        f"-GPSLongitude={abs(lon)}", f"-GPSLongitudeRef={lon_ref}",
        f"-GPSAltitude={alt}", "-GPSAltitudeRef=0",
        f"-DateTimeOriginal={ts}", f"-CreateDate={ts}", f"-ModifyDate={ts}",
        "-overwrite_original"
    ]

print("Starting interval capture…")
next_t = time.time()
while True:
    # Wait until next interval boundary
    now = time.time()
    if now < next_t:
        time.sleep(next_t - now)
    next_t += INTERVAL_SEC

    # Grab frame
    ok, frame = cap.read()
    if not ok:
        continue

    # Get GPS snapshot (non-blocking fallback if nothing fresh)
    gps = current_gps()
    utc = datetime.datetime.utcnow() if not gps else gps[3]
    fname = utc.strftime('%Y%m%d_%H%M%S_%f')[:-3] + ".jpg"
    fpath = os.path.join(OUT_DIR, fname)

    # Save JPEG
    cv2.imwrite(fpath, frame, [int(cv2.IMWRITE_JPEG_QUALITY), 92])

    # Tag EXIF if GPS available
    if gps:
        lat, lon, alt, dt = gps
        args = ["exiftool"] + exif_gps_args(lat, lon, alt, dt) + [fpath]
        subprocess.run(args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)

  • Change INTERVAL_SEC whenever you like (e.g., 2, 10, etc.).

  • Images are saved under /data/timelapse with UTC timestamps in the filename, and GPS in EXIF.

  • This script reads GPS directly from the vehicle via MAVLink (using the UDP endpoint you added). BlueOS exposes MAVLink to apps exactly for use cases like this. Blue Robotics

3) Run it forever (service)

Create /etc/systemd/system/timelapse-exif.service:

[Unit]
Description=BlueBoat timelapse capture with EXIF GPS
After=network-online.target

[Service]
ExecStart=/usr/bin/python3 /home/pi/timelapse_exif.py
Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target

Then:

sudo systemctl daemon-reload
sudo systemctl enable --now timelapse-exif.service

That’s it—your Pi will capture a geotagged still every N seconds while the boat is powered and has a GPS fix.


Hi @nuballawalla -

Very good application of the AI!

The step 1 it lists is already taken care of by default, I believe - you can double check in Mavlink Endpoints.

You’ll also likely want to use /dev/video2, not video0 - this provides the H264 stream (instead of MJPG) but either could work?

You’re very unlikely to break or brick anything, but for added peace of mind, you may want to package your code as an extension for testing, rather than executing it at the terminal in BlueOS. If you do try it at the terminal, without taking the red-pill, you’ll be within a docker container and protected from doing anything to the root file system. However, it may be necessary to take the red-pill to get things working, depending on what permissions are required by the script. You may also need to disable/remove the video stream setup on the Video Streams page.

You may want tos how the AI the code linked here - as an example of how to pull GPS position and heading from the Mavlink2Rest interface, instead of grabbing from the mavlink telemetry stream directly?

Let us know how your testing goes!

1 Like

Hi @tony-white , thanks for the tips. My first attempts were somewhat successful. Running scripts from the terminal (in the docker container) only got me so far. I could grab a frame every N seconds, write it as a jpeg, and write metadata in the exif fields. However, I could not get the GPS data. ChatGPT says I should get GPS data in ā€˜Auto’ mode. I’m still bench testing so I haven’t run any auto missions yet. Any ideas on how to pull GPS data, either with UDP or BlueOS’s Mavlink2Rest endpoint while in manual mode?

Hi @nuballawalla -

Giving the AI that code I linked and asking it to poll the mavlink2rest interface for GPS data should do the trick - it is definitely a ā€œhallucinationā€ of the AI that you must be in Auto mode to receive GPS position! If you have lock, this data is available regardless of vehicle mode…