Home / Comparisons / Performance / vs pandas

csvlite vs pandas · measured 29 September 2026

Skip the notebook.
Get the answer 57.3× sooner.

Same files, same laptop, CSV to answer. Median of 5 runs.

typical57.3× fastermedian of 27 measured queries
best case162× fasterText match, 1 GB
1 GB49.4–162×faster on every query
10 GBonly csvlitepandas runs out of memory at 8 GB

Every query

10 MB

53,596 rows · 15 columns · File in page cache

10 MB: median time from CSV to answer
QuerycsvlitepandasBarsvs pandas
Text matchregion = north10 ms156 ms15.6× faster
Regex searchdescription has the word café14 ms190 ms13.2× faster
Number rangeamount 25,000–75,00010 ms157 ms16.4× faster
Date rangeoccurred_at in a range10 ms151 ms15.8× faster
Sort textevery row by region10 ms166 ms17.5× faster
Sort numbersevery row by amount10 ms162 ms16.6× faster
Sort datesevery row by occurred_at12 ms153 ms12.8× faster
Group + sumcount, sum(amount) by region, status12 ms167 ms13.9× faster
Group + distinct…plus distinct descriptions12 ms181 ms14.6× faster
Open once, keep askingTotal time as queries pile up. pandas loads the file once.
Total time after 9 queries, 10 MB: csvlite 40 ms, pandas 296 ms0 ms100 ms200 ms300 msopen123456789queries answered296 mspandas40 mscsvlite
Chart data
Aftercsvlitepandas
Open10 ms148 ms
Text match12 ms151 ms
Regex search18 ms185 ms
Number range20 ms198 ms
Date range23 ms204 ms
Sort text25 ms222 ms
Sort numbers27 ms236 ms
Sort dates30 ms244 ms
Group + sum34 ms266 ms
Group + distinct40 ms296 ms

100 MB

532,923 rows · 15 columns · File in page cache

100 MB: median time from CSV to answer
QuerycsvlitepandasBarsvs pandas
Text matchregion = north15 ms1.4 s97.1× faster
Regex searchdescription has the word café32 ms1.7 s53.9× faster
Number rangeamount 25,000–75,00021 ms1.6 s76.3× faster
Date rangeoccurred_at in a range19 ms1.5 s78.5× faster
Sort textevery row by region29 ms1.7 s57.0× faster
Sort numbersevery row by amount26 ms1.6 s62.3× faster
Sort datesevery row by occurred_at27 ms1.6 s58.4× faster
Group + sumcount, sum(amount) by region, status29 ms1.7 s57.3× faster
Group + distinct…plus distinct descriptions39 ms1.7 s44.6× faster
Open once, keep askingTotal time as queries pile up. pandas loads the file once.
Total time after 9 queries, 100 MB: csvlite 140 ms, pandas 2.9 s0 s1 s2 s3 sopen123456789queries answered2.9 spandas140 mscsvlite
Chart data
Aftercsvlitepandas
Open11 ms1.5 s
Text match14 ms1.5 s
Regex search35 ms1.8 s
Number range45 ms2.0 s
Date range53 ms2.0 s
Sort text69 ms2.2 s
Sort numbers83 ms2.4 s
Sort dates97 ms2.5 s
Group + sum114 ms2.7 s
Group + distinct140 ms2.9 s

1 GB

5,300,205 rows · 15 columns · File in page cache

1 GB: median time from CSV to answer
QuerycsvlitepandasBarsvs pandas
Text matchregion = north90 ms15 s162× faster
Regex searchdescription has the word café264 ms17 s65.9× faster
Number rangeamount 25,000–75,000172 ms16 s90.7× faster
Date rangeoccurred_at in a range139 ms15 s105× faster
Sort textevery row by region226 ms17 s72.9× faster
Sort numbersevery row by amount197 ms16 s80.2× faster
Sort datesevery row by occurred_at191 ms15 s78.3× faster
Group + sumcount, sum(amount) by region, status236 ms16 s66.5× faster
Group + distinct…plus distinct descriptions362 ms18 s49.4× faster
Open once, keep askingTotal time as queries pile up. pandas loads the file once.
Total time after 9 queries, 1 GB: csvlite 1.4 s, pandas 29 s0 s10 s20 s30 sopen123456789queries answered29 spandas1.4 scsvlite
Chart data
Aftercsvlitepandas
Open69 ms14 s
Text match99 ms14 s
Regex search303 ms18 s
Number range413 ms19 s
Date range491 ms20 s
Sort text654 ms22 s
Sort numbers791 ms24 s
Sort dates920 ms25 s
Group + sum1.1 s26 s
Group + distinct1.4 s29 s

10 GB

53,002,050 rows · 15 columns · Cold cache: evicted from memory before every run

10 GB: median time from CSV to answer
QuerycsvlitepandasBarsvs pandas
Text matchregion = north4.6 sout of memoryonly csvlite
Regex searchdescription has the word café6.2 sout of memoryonly csvlite
Number rangeamount 25,000–75,0005.3 sout of memoryonly csvlite
Date rangeoccurred_at in a range5.0 sout of memoryonly csvlite
Sort textevery row by region9.1 sout of memoryonly csvlite
Sort numbersevery row by amount6.4 sout of memoryonly csvlite
Sort datesevery row by occurred_at6.3 sout of memoryonly csvlite
Group + sumcount, sum(amount) by region, status5.7 sout of memoryonly csvlite
Group + distinct…plus distinct descriptions7.2 sout of memoryonly csvlite
Open once, keep askingTotal time as queries pile up. pandas loads the file once.
Total time after 9 queries, 10 GB: csvlite 31 s0 s10 s20 s30 s40 sopen123456789queries answered31 scsvlite
Chart data
Aftercsvlitepandas
Open3.1 s—
Text match4.7 s—
Regex search7.8 s—
Number range9.9 s—
Date range12 s—
Sort text18 s—
Sort numbers21 s—
Sort dates24 s—
Group + sum27 s—
Group + distinct31 s—

The code you’d write

As benchmarked

Text matchcsvlite 90 mspandas 15 sat 1 GB
df = pd.read_csv("data.csv", dtype=str, na_filter=False)
df["region"].eq("north").sum()
csvliteAdd filter · region = north
Sort numberscsvlite 197 mspandas 16 sat 1 GB
df = pd.read_csv("data.csv", dtype=str, na_filter=False)
amount = pd.to_numeric(df["amount"], errors="coerce")
amount.sort_values(kind="mergesort", na_position="last")
csvliteClick the amount header
Group + sumcsvlite 236 mspandas 16 sat 1 GB
df = pd.read_csv("data.csv", dtype=str, na_filter=False)
(df.assign(_amount=pd.to_numeric(df["amount"], errors="coerce"))
   .groupby(["region", "status"], dropna=False)
   .agg(count=("region", "size"), total=("_amount", "sum")))
csvlitePivot · region, status · Sum of amount
Machine
Intel Core i7-11850H · 32 GB RAM · NVMe SSD · Linux. Every run capped at 8 GB of memory.
Versions
csvlite 1.0.0 · DuckDB 1.5.5 · Polars 1.44.2 · pandas 3.0.6. All columns read as text.
Timing
CSV to answer, fresh process per run, startup excluded. ≤1 GB warm; 10 GB cold. Out of memory: one capped attempt.
Data
Generated analytics CSV. All 1064 raw runs (plus load-first) · csvlite-only benchmarks

All four tools · vs Polars · vs DuckDB

Try it on your own CSV

Download csvlite