Polars gets fast before the data settles. Its expression engine runs in Rust across every core at its disposal, while its query optimizer rewrites work before execution. Miss either one, and a script can turn a high-performance data tool into a patient endurance test.
Matthew Mayo of KDnuggets puts the problem plainly: “Almost every slow Polars script lacks in terms of one of those two.” The three techniques below focus on keeping both systems useful, using a month of NYC yellow taxi trips published by the TLC as Parquet.
1. Scan Parquet Instead of Reading It Whole
The first performance choice happens at the file boundary. pl.read_parquet pulls an entire file’s contents into memory and then filters it, which means the narrowing comes after Polars has already loaded the data.
pl.scan_parquet takes a different route. It hands back a LazyFrame that records the requested operations without executing them, giving the query optimizer a complete plan to inspect before the work begins.
The optimizer can push filters and column list reductions down to the scan itself, so the narrowing happens as the file is read. That distinction matters when the example starts with an estimated 3,724,889 rows and only needs a small result.
The example file comes from the TLC’s yellow taxi trip data and uses the filename yellow_tripdata_2026-01.parquet. The download command uses curl -O; the important Polars decision comes next, where a lazy scan lets the optimizer reduce the work at the source.
There is one rule to protect: every collect() is a wall the optimizer cannot see past. If a pipeline collects data too early, later operations sit beyond that wall, and the optimizer cannot rewrite them together with the earlier scan.
2. Keep Group Calculations Inside Expressions
Polars also gains speed by keeping calculations inside its expression system. The over() operation performs group-level calculations within a single expression while preserving row order.
That combination removes a common trade-off. A grouped calculation can use group context without forcing the result into a separate shape, then the original row order remains available for the next operation.
The key idea is not to move a calculation out of Polars when the expression API can represent it. The engine is written and executed in Rust across every core at its disposal, so native expressions keep work inside the system built to run it.
This is where a slower pattern enters: map_elements hands every value in a column to a Python callable. That approach is slower than the native expressions API, so a Python function should not replace an expression when the calculation already fits Polars’ native tools.
3. Treat Conditional Expressions Carefully
Conditional logic has its own trap. Polars computes every branch of a when / then chain in parallel and filters afterwards.
That means an expensive branch does not escape execution simply because its condition excludes most rows. The condition filters the result after Polars has computed the branches, so the apparent shortcut may still carry the full calculation cost.
These techniques point to the same operating rule: give Polars a complete expression plan, keep the pipeline lazy, and avoid handing individual values to Python. The expression engine supplies the execution power; the optimizer decides how much work survives long enough to run.
The sample output has a shape of (4, 3)—four rows and three columns. Getting there efficiently depends less on clever syntax than on avoiding the barriers and escape hatches that prevent Polars from seeing the whole job.
As of September 21, 2026, the lesson is refreshingly unglamorous. Read lazily, calculate with native expressions, and remember that collect() is not a harmless punctuation mark—it ends the optimizer’s view.
