cProfile vs py-spy: 8 profiling reads I got wrong
Last week I cut a script that handles twenty thousand rows from 165 ms to 96 ms by timing three sections separately. Then I threw the same code at cProfile, and the top hotspot came back pointing at a function I never planned to touch. The reason is that part of the tottime cProfile reports does not belong to that function at all. It belongs to the profiler.
This is eight reading traps I re-ran one by one on Windows 11 with CPython 3.13.14, using cProfile, pstats and py-spy 0.4.3. Each one reproduces in a dozen lines, the tracebacks below are copied verbatim, and the commands paste straight into a terminal. The last section covers the profiling package migration in Python 3.15. If the scripts you write today have to survive three more years, that section matters more than the other seven.
TL;DR: one profile file, sorted three ways, gives three non-overlapping "top hotspots", and no default view is the truth. cProfile charges its own overhead to the caller's tottime, not the callee's. In a multiprocessing program the top entry is usually CreateProcess instead of your code. And py-spy skips blocked frames by default: --idle means include idle, not ignore it.
Three tools, three different answers
| Tool | What it gives you | Where it misleads |
|---|---|---|
| cProfile 3.13.14 | exact call counts and per-call time | overhead lands on the caller; zero coverage of child processes |
| pstats views | a table sorted by the column you pick | three sorts, three answers; strip_dirs merges same-named functions |
| py-spy 0.4.3 | sampling, no code change, can attach | blocked frames dropped by default; sample counts jitter |
The Python 3.15 profiling docs draw the same line more plainly: statistical sampling has "virtually none" overhead and returns an estimate, while deterministic tracing gives exact call counts at "moderate" overhead, so use it when you need the counts. That is also my default posture: sampling first to find the direction, tracing second to confirm the counts. The site's earlier piece on ThreadPoolExecutor vs asyncio vs queue in four shapes answers "which concurrency shape to pick"; this one answers "it is already slow, now find the slow part".
1. One .prof file, three sort keys, three top hotspots
The workload has three functions: one heavy loop called once, one trivial function called twenty thousand times, and a main that just wires them together.
python -m cProfile -o run.prof app.py
python -c "import pstats; pstats.Stats('run.prof').sort_stats('tottime').print_stats(6)"
python -c "import pstats; pstats.Stats('run.prof').sort_stats('cumtime').print_stats(6)"
python -c "import pstats; pstats.Stats('run.prof').sort_stats('calls').print_stats(6)"
Same file, 80,005 calls, three different winners:
| Sort | Winner | Key numbers on that row |
|---|---|---|
| tottime | heavy_once | 1 call, 0.161 s |
| cumtime | main | 0.199 s total, tottime only 0.00003 s |
| calls | str.lower | 20,000 calls, 0.006 s total |
Three names, no overlap. The tottime winner is the real hotspot, the cumtime winner is the entry point that does almost no work itself, and the calls winner is simply the most frequently invoked. Whichever view you open is the conclusion you walk away with.
My reading order: cumtime first to find which path is expensive overall, then tottime inside that path to find the expensive step, then calls only to check whether frequency is the problem. Never use calls to find a hotspot.
2. Profiler overhead is charged to the caller
This is the least intuitive one. I wrote a shell function that does no work at all, just calls another empty function three hundred thousand times:
def leaf():
return None
def dispatch(n):
for _ in range(n):
leaf()
Bare, dispatch(300000) takes 0.0204 s. Under cProfile its tottime becomes 0.0826 s, a 305% jump, while the callee leaf only moves from 0.020 s to 0.0296 s (+48%).
The profiler records an event on every call and return, and that cost is incurred at the call site, so it is attributed to the caller's inline time. Any parent function that only dispatches will therefore show up as an invented hotspot. Optimising it changes nothing.
The 3.15 docs phrase the same effect in a line that is easy to skip: deterministic profilers "introduce overhead for Python code but not for C-level functions, which can skew comparisons". Put the two together and the rule is simple: pure Python code getting slower under a profiler does not mean it was slow to begin with.
3. strip_dirs() merges same-named functions from different files
I built two files both named util.py in different directories, each with a run(): one loops three hundred thousand times (0.0148 s), the other nine hundred thousand (0.0432 s).
Unstripped, that is two rows with distinguishable paths. After .strip_dirs() it becomes one row, util.py: run, with ncalls 2 and tottime 0.058, because the numbers are added together.
import pstats
pstats.Stats('c.prof').stats # two rows, full paths
pstats.Stats('c.prof').strip_dirs().stats # one row, calls=2, times summed
A threefold difference disappears at the moment of merging. As soon as any same-named helper is imported from more than one place, you get a row that is expensive and anonymous. The check: if ncalls on a row is higher than you expect, drop strip_dirs and read it again before believing it.
4. When ncalls reads 57313/1, which denominator does percall use
Recursive functions are the easiest to misread. One fib(22) produces this row:
57313/1 0.026 0.000 0.026 0.026 fib.py:3(fib)
57313/1 is total calls over primitive calls: 57,312 recursive re-entries and exactly one entry from outside. The docs define a primitive call as one "not induced by recursion", displayed as total/primitive.
The trap is the percall column. Dividing tottime by 57313 lands in the 0.000 bucket and flattens 57,312 re-entries into noise. Judge recursive cost from the absolute cumtime column, not from percall.
5. Multiprocessing: top hotspot is CreateProcess, your code never appears
This one reads like an incident. A script running two loops through multiprocessing.Pool(2), profiled from inside the script so only the parent is covered:
| Metric | Value |
|---|---|
| Wall clock | 1.4046 s |
| Profile total | 1.4044 s |
| Does worker appear | no |
| Top tottime | _winapi.CreateProcess, 1.010 s over 2 calls |
| Second | _winapi.WaitForMultipleObjects, 0.274 s |
The totals match the wall clock, so it looks complete. But the real hotspot, worker, never appears, and CreateProcess has taken 1.010 s, time the child process spent running your loop while the parent waited. The conclusion you draw is "process creation is slow", which is wrong.
Wrapping it from the command line is worse: python -m cProfile -o mp.prof script.py crashes on Windows, and so does the module form. The traceback is under error 1 below. The hard rule: cProfile does not see child processes and emits no warning. Use py-spy with --subprocesses to cross that boundary.
6. py-spy hides blocked time by default; --idle means include idle
I wrote a script looping thirty times over 60 ms of pure CPU plus 20 ms of time.sleep, a real ratio of 75/25.
Four default runs produced 896 samples between them, and the wait_io frame appeared once (0.11%), with burn holding between 73.8% and 78.7%.
One run with --idle produced 296 samples: burn 191 (64.5%) and wait_io 66 (22.3%), which is close to the real 25%.
py-spy record --format raw -o out.txt -- python spytest.py
py-spy record --idle --format raw -o idle.txt -- python spytest.py
py-spy record --subprocesses -o sub.svg -- python poolwork.py
The README explains it: by default py-spy keeps only threads actively running code and excludes sleeping or idle ones, and --idle disables that filter so idle frames are included. So --idle does not mean "ignore idle", it means "include idle", the opposite of what the flag name suggests. I got this backwards twice. The same README notes that on Windows, calls blocked on IO are not yet marked idle.
7. Sample counts jitter between 210 and 244
Four identical runs of the same script sampled 231, 210, 211 and 244 times. At 100 Hz over roughly 2.4 s the expected count is around 240, so the observed spread is 16%.
That is not a bug, it is what sampling is. So when two functions differ by less than 10%, do not use samples to rank them. Shrink the interval with -i, run the target longer, or go back to deterministic tracing and count calls.
8. From Python 3.15 the profile module is on its way out
This section rests on official docs and the PEP only. My machine runs 3.13.14 and cannot exercise the 3.15 package, so there are no measured numbers here.
| When | What changes |
|---|---|
| 2026-10-09 | Python 3.15 ships, adding the profiling package (PEP 799) |
| 3.15 | profiling.tracing (cProfile relocated, cProfile kept as an alias) and profiling.sampling (the Tachyon sampling profiler) |
| 3.15 | profile is deprecated, importing it emits DeprecationWarning |
| 3.16 | every use of profile warns |
| 3.17 | profile is removed from the standard library |
The command line is python -m profiling.sampling run app.py, with --flamegraph for a flame graph and attach for a running process, while deterministic tracing is python -m profiling.tracing app.py. pstats is not deprecated and the sampler's --pstats output stays compatible with it, so the pstats reading habits you build now still work in three years.
One documentation claim contradicts my measurements, so you do not have to get tangled in it. PEP 799's motivation says "cProfile only observes the main thread". On 3.13.14, both a thread started after enable() and a thread that already existed and was blocked both had their CPU function recorded, with total time matching the wall clock. The PEP is explicitly a historical document, so do not treat it as a version fact. To see which behaviour your build has, run this:
import cProfile, threading
def cpu():
s = 0
for i in range(2_000_000):
s += i
p = cProfile.Profile()
p.enable()
t = threading.Thread(target=cpu)
t.start(); t.join()
p.disable()
p.print_stats('tottime')
If cpu shows up, threads are covered; if it does not, they are not.
The reading order I use now
- Run the workload bare three times and take the median. A profiled number is never a baseline.
- Use
py-spy recordto see the shape: CPU bound or heavily blocked. If blocked, add--idlebefore judging anything. - If processes or a pool are involved, go straight to
--subprocessesand stop hoping cProfile will cover it. - Once a function looks suspicious, count exact calls with cProfile and read both the tottime and cumtime sorts.
- When a profiler number contradicts your stopwatch, trust the stopwatch. A profiler gives attribution, not truth.
Troubleshooting: four real errors
Error 1: Can't pickle worker, attribute lookup worker on __main__ failed
Symptom: python -m cProfile -o mp.prof script.py against any script using multiprocessing.Pool exits with code 1 on Windows.
_pickle.PicklingError: Can't pickle <function worker at 0x000001D234161620>: attribute lookup worker on __main__ failed
Cause: the cProfile entry point runs your script through compile() and exec() with a temporary globals dict, so worker never lands in the real sys.modules['__main__']. Spawn then looks it up as __main__.worker and fails. The -m module form fails identically; I tried both.
Fix: stop wrapping multiprocessing scripts with -m cProfile. Either profile the parent from inside the script, as in section 5, or switch to py-spy record --subprocesses.
Error 2: EOF read where object expected
Symptom: pstats.Stats("app.prof") raises EOFError: EOF read where object expected immediately.
Cause: a .prof file is written by marshal.dump(), and a zero-byte file contains no object to unmarshal. Zero-byte files are easy to produce: Ctrl-C, an os._exit() call, or an exception thrown before dump_stats(). The official profile docs say the same thing directly: profiling only works if the called command actually returns, and if the interpreter is terminated, for example by sys.exit() inside it, no results are printed.
Measure the file before guessing:
python -c "import os; p='app.prof'; print(os.path.getsize(p) if os.path.exists(p) else 'missing')"
Then move the dump into try/finally so the exception path still writes:
prof = cProfile.Profile()
try:
prof.enable()
main()
finally:
prof.disable()
prof.dump_stats("app.prof")
On the reading side, catch EOFError and report "no data collected this run" instead of letting the whole script die.
Error 3: KeyError: 'c'
Symptom: pstats.Stats(p).sort_stats("c") raises KeyError: 'c', and sort_stats(3) raises KeyError: 3.
Cause: sort_stats indexes a lookup table directly, and its abbreviation support only accepts unique prefixes. c matches calls, cumtime and cumulative at once, so the ambiguous prefix is deleted from the table and you get a bare KeyError with no hint. The integer compatibility path accepts exactly -1, 0, 1 and 2 (stdname, calls, time, cumulative), so 3 is not allowed.
Fix: use the full name or the enum.
import pstats
from pstats import SortKey
pstats.Stats('run.prof').sort_stats(SortKey.TOTTIME).print_stats(10)
The command line is stricter still: -s takes full key names only, and -s cum exits 2 with "invalid choice". The default value in the source is the integer 2, meaning cumulative, so leaving -s out sorts by cumulative time while typing -s 2 yourself is rejected as an invalid option.
Error 4: Another profiling tool is already active
Symptom: enabling a second profiler in a process that already has cProfile or a coverage tracer raises ValueError: Another profiling tool is already active.
Cause: the interpreter allows one profile hook at a time.
Fix: keep one layer. To see coverage and timing together, run twice, or sample from outside with py-spy, which does not take the hook.
FAQ
Q: Can I use cProfile output as a benchmark?
No. The docs state the profilers are for building an execution profile, not for benchmarking, and timeit is the tool for timing. Deterministic overhead applies to Python code but not to C-level functions, so comparisons between the two skew.
Q: Why is my profile full of _winapi. functions?
It means the time went into system calls and waiting, not your Python code. On Windows, CreateProcess and WaitForMultipleObjects near the top mean process startup or child waiting. Do not optimise those; optimise the code inside the child.
Q: Why do I get fewer samples than I calculated?
py-spy only samples when the target is really executing, so startup, interpreter initialisation and shutdown can be missed, and threads judged idle are dropped. Add --idle for the full picture and let the target run longer.
Q: Do I have to change my scripts after 3.15?
No for import cProfile and pstats: cProfile stays as an alias and pstats is not deprecated. The only real change is import profile, the pure Python one, which is removed in 3.17 and should become profiling.tracing.
Closing
Of the eight, the one that saves the most time is the second: overhead lands on the caller, so "the parent function is expensive" is structurally unreliable. My order is now stopwatch first, py-spy for the shape, cProfile last for exact counts, and when the three disagree the stopwatch wins.
Two other "no error, wrong result" cases on the same machine follow the same logic: six silent failures in asyncio cancellation and timeouts covers waiting code that swallows its own cancellation, and subprocess exit code 0 with no output covers the child-process boundary. Together they cover most of "the program looks fine but the result is wrong".
👉 Join Xiaomi MiMo Platform: Leading AI model platform with cost-effective inference
👉 Join Aliyun AI: Top AI products with exclusive coupons for business innovation
📌 This article was AI-assisted generated and human-reviewed | TechPassive — An AI-driven content testing site focused on real tool reviews
🔗 Recommended Tools
These are carefully selected tools. Using our affiliate links supports us to keep producing quality content:
As an Amazon Associate, I earn from qualifying purchases at no extra cost to you.