Back to articles
200GB of Raw Rover Data and No Pipeline to Process It. So I Wrote One.
How-ToDevOps

200GB of Raw Rover Data and No Pipeline to Process It. So I Wrote One.

via Dev.toSherin Joseph Roy

Building a 33-module Python pipeline that takes raw GoPro/Insta360 recordings and turns them into SLAM-ready datasets with auto-labeling, depth estimation, and edge deployment. The unglamorous engineering that nobody writes about. Three field sessions in Kerala with a GoPro Hero 11 on a rover chassis, an Insta360 X4 for 360-degree coverage, and an Android phone running Sensor Logger. Result: 200GB of raw video, a growing spreadsheet tracking which files had GPS lock, which files had HyperSmooth accidentally enabled (destroying IMU correlation), and one SD card that corrupted mid-write. I tried stitching together scripts. FFmpeg for frames, a GPMF parser for telemetry, a separate tool for calibration, manual CSV wrangling for synchronization. After the third time I forgot which script ran in which order, I stopped and asked myself what I actually needed. The answer was not a better script. It was a system. That system became Orvex , and over six months it grew into 33 core modules, a 28

Continue reading on Dev.to

Opens in a new tab

Read Full Article
7 views

Related Articles