Back to articles
Polars Just Made Pandas Look Slow — Benchmarks Inside

Polars Just Made Pandas Look Slow — Benchmarks Inside

via Dev.to PythonAlex Spinov

I ran benchmarks on a 10M row CSV file. Polars finished in 0.3 seconds. Pandas took 12 seconds. That's a 40x speedup with zero code complexity increase. Here's what's happening and why you should care. The Benchmark Dataset: 10M rows, 8 columns (sales data). Operations: read CSV, filter, groupby, sort. | Operation | Pandas | Polars | Speedup | |----------------|----------|----------|---------| | Read CSV | 8.2s | 0.9s | 9x | | Filter rows | 0.4s | 0.02s | 20x | | GroupBy + Agg | 2.8s | 0.15s | 19x | | Sort | 1.1s | 0.08s | 14x | | Total pipeline | 12.5s | 0.32s | 39x | Polars is written in Rust. It uses Apache Arrow memory format. It parallelizes everything automatically. The Code Comparison Pandas: import pandas as pd df = pd . read_csv ( " sales.csv " ) result = ( df [ df [ " revenue " ] > 100 ] . groupby ( " category " ) . agg ({ " revenue " : " sum " , " quantity " : " mean " }) . sort_values ( " revenue " , ascending = False ) ) Polars: import polars as pl df = pl . read_csv ( " s

Continue reading on Dev.to Python

Opens in a new tab

Read Full Article
8 views

Related Articles