Back to articles
Why pandas cannot be used in TorchScript — and how xpandas fixes it

Why pandas cannot be used in TorchScript — and how xpandas fixes it

via Dev.toXianqiu Li

The problem I kept running into I often deploy PyTorch models using TorchScript and LibTorch (C++) for inference. The model itself is not the hard part. The problem is data processing . In many real pipelines — especially in quantitative finance, feature engineering, or low-latency systems — you still need to do things like: groupby + aggregation rolling window operations column-wise transformations In Python, this is trivial with pandas. But the moment you try to move the whole pipeline into TorchScript or C++ inference, everything breaks. Why pandas (and friends) don’t work in TorchScript This is not a pandas problem. It’s a runtime boundary problem . 1. pandas is Python-runtime dependent TorchScript explicitly removes the Python runtime: no Python objects no dynamic typing no CPython extensions pandas relies heavily on: Python objects NumPy internals CPython C-API So torch.jit.script() simply cannot compile pandas code. 2. Polars / Arrow don’t solve this either You might think: “Wha

Continue reading on Dev.to

Opens in a new tab

Read Full Article
18 views

Related Articles