DBMerge: A database-agnostic Python UPSERT module to simplify ETL pipelines

Hi everyone, I’d like to share an open-source library I’ve been developing for the Python community: DBMerge . Consider it like an advanced version of pd.to_sql . It is designed to simplify common but tedious task of syncing data to a SQL database. Instead of writing custom MERGE , UPSERT or ON CONFLICT queries for different SQL dialects, DBMerge performs INSERT , UPDATE , and DELETE operations automatically in a single step. Because it is built on top of SQLAlchemy Core, it is fully database-agnostic. It has currently been thoroughly tested with PostgreSQL, MySQL/MariaDB, SQLite and MS SQL Server. How It Works Under the Hood The underlying logic focuses on performance and reliability: Staging: The module first creates a temporary staging table in the database and loads your entire incoming dataset into it using a fast bulk INSERT . Reconciliation: It then executes optimized UPDATE , INSERT , and DELETE statements against the target table by comparing the target table with the temporar