
Why pandas cannot be used in TorchScript — and how xpandas fixes it
The problem I kept running into I often deploy PyTorch models using TorchScript and LibTorch (C++) for inference. The model itself is not the hard part. The problem is data processing . In many real pipelines — especially in quantitative finance, feature engineering, or low-latency systems — you still need to do things like: groupby + aggregation rolling window operations column-wise transformations In Python, this is trivial with pandas. But the moment you try to move the whole pipeline into TorchScript or C++ inference, everything breaks. Why pandas (and friends) don’t work in TorchScript This is not a pandas problem. It’s a runtime boundary problem . 1. pandas is Python-runtime dependent TorchScript explicitly removes the Python runtime: no Python objects no dynamic typing no CPython extensions pandas relies heavily on: Python objects NumPy internals CPython C-API So torch.jit.script() simply cannot compile pandas code. 2. Polars / Arrow don’t solve this either You might think: “Wha
Continue reading on Dev.to
Opens in a new tab



