← Back to Home

Tests Green, Code Broken: 7 Silent unittest Traps

Pythonunit testingdebugging

Last week a reader sent me a ticket. Forty-seven tests green in CI. Four hours after deploy, every checkout on the payment page came up thirty percent short.

Roll back the code, read the logs, check the dashboards, nothing moved. The culprit was one character: a minus sign typed as a plus, so the discount ran the wrong way. And directly above it sat a test asserting the discounted amount. That test was green.

The problem was not that the test suite was small. It was that nothing the suite did ever touched that line.

unittest.mock is the most forgiving module in the standard library. Forgiving to the point that it invents whatever you assume is there. Method missing? It creates one. Wrong arguments? Accepted. No return value configured? Here is a Mock. All of that exists so tests can start running before the wiring is finished. The bill arrives later — a green suite can mean almost nothing happened.

Below are seven paths where the suite goes green and the code does not run. I actually ran all of them on Windows 11 with CPython 3.13.14. Every claim has a minimal reproduction and the measured output. Quoted documentation comes from docs.python.org/3.13/library/unittest.mock.html; line numbers point into Lib/unittest/mock.py on this machine (3,189 lines, matching 3.13.14).

TL;DR: patch at the definition site instead of the lookup site and the patch silently misses; no autospec and wrong arguments pass; spec guards attribute names but not signatures; MagicMock reports bool() as True, len() as 0, and blows up on comparisons; reset_mock() leaves return_value and side_effect in place; assert_called_with compares with ==, so the integer 1 satisfies True; a parent mock's assert_not_called() ignores calls made on its children. None of the seven raises.

Start with a number: one bug, two suites

I built a minimal checkout module. Pricing.total() applies a proportional discount. Then I injected a mutant: the minus before the multiplication became a plus, flipping the discount direction. One mutant, two test suites, one run each.

SuiteCorrect codeMutantDetected
A: MagicMock throughout6 tests green6 tests still green0/6
B: real Pricing object5 tests green2 tests red2/5

Suite A asserts six things: returns a dict, has an amount key, currency is CNY, total was called, coupon was forwarded, negative amounts get clamped to zero. Every assertion is correct and every one tests something real — the orchestration inside checkout(). Not one of them touches the arithmetic inside total().

That is where 0/6 comes from: the mock boundary was drawn inside the logic under test.

There is one rule for drawing it. Ask whether a bug in this logic would hurt. If it would, keep the real implementation and only replace what crosses a process, a network, or a clock.

Trap 1: patch where it is defined, read where it is imported

The documentation is explicit: "The basic principle is that you patch where an object is looked up, which is not necessarily the same place as where it is defined."

That sentence gets quoted constantly. It still produced a crisp pair of numbers in my run. The code under test does from client import get_user, and the real get_user increments a counter.

patch("client.get_user")        -> mock called 0 times, real function ran 1 time
patch("service_from.get_user")  -> mock called 1 time, real function ran 0 times

The first patch had no effect at all, and the test can still pass — as long as the value you assert on happens to match what the real function returns. My first version asserted that the returned name equals real. The real function returns real. Both outputs were identical. The only difference was the counter moving from 0 to 1.

In a real project that 1 might be a real charge, a real email, a real order row. The suite is green and the side effect already happened.

The fix is to patch where the name is imported. If you cannot remember, switch to import client and call client.get_user(...) — then definition site and lookup site are the same place. A typo like patch("client.get_usr") does raise AttributeError, so that one is not silent; the verbatim message is error four below.

Trap 2: no autospec, so wrong arguments pass

patch() hands you a mock that accepts any signature. Same mock, two modes:

m(1, None, "extra", nope=1)  -> returns normally, call_count = 1
                                call_args = call(1, None, 'extra', nope=1)

The real function takes one positional argument. With autospec=True, the same call raises TypeError: too many positional arguments, and omitting the argument raises TypeError: missing a required argument: 'uid'.

The second shape is more common and more expensive. Production code shipped n(user_id=1, chanel="sms") — channel missing an n. Without autospec the test records chanel='sms' in call_args and passes; the real function raises unexpected keyword argument in production.

The docs describe the check directly: "Methods and functions being mocked will have their arguments checked and will raise a TypeError if they are called with the wrong signature."

In the unittest.mock.patch signature, autospec defaults to None, which means off. That default exists for historical reasons: turning it on would redden a large pile of existing tests at once. New tests have no such excuse.

Trap 3: spec guards attribute names, not signatures

This is the one people conflate, because spec and autospec read almost the same and many assume the first contains the second. Three setups against a class Repo whose only method is save(self, o, force=False):

MagicMock()            -> repo.archive(o) succeeds silently, archive.call_count = 1
MagicMock(spec=Repo)   -> AttributeError: Mock object has no attribute 'archive'
MagicMock(spec=Repo)   -> save(1, 2, 3, 4) still passes; signatures unchecked
create_autospec(Repo)  -> save(1, 2, 3, 4) -> TypeError: too many positional arguments

The boundaries are clean. A bare MagicMock() accepts everything. spec= only enforces attribute existence. create_autospec or autospec=True enforces the signature too.

The attribute check alone is worth a lot. Rename save to persist during a refactor, miss one call site, and a bare MagicMock conjures a persist() that returns a Mock. Tests green, production AttributeError. With spec=Repo, the test goes red on the spot.

Two sentences from the docs matter here:

"Accessing any attribute not in this list will raise an AttributeError."
"It is off by default because it can be dangerous. With it switched on you can write passing tests against APIs that don't actually exist!"

The second is about patch(create=True), but the reasoning transfers intact. Any switch that lets you write a green test against an API that does not exist is moving the bug from today to tomorrow.

Trap 4: MagicMock's defaults only cover half the magic methods

MagicMock pre-creates "most of the useful" magic methods, which means they return default values:

bool(MagicMock())   -> True
len(MagicMock())    -> 0
int(MagicMock())    -> 1
float(MagicMock())  -> 1.0

Those defaults contradict each other on two everyday branches. Write if not cache.get(key): and a MagicMock result gives bool True, so the negation is False and the cache-miss branch never runs. Write if len(rows) == 0: and len is 0, so the empty branch runs every time. One stand-in, two branches — one never executes, one always executes, and the coverage report marks both as covered.

Comparisons have no default at all:

MagicMock() less than 0  -> TypeError: '<' not supported between instances of 'MagicMock' and 'int'
MagicMock() equals 0     -> False
MagicMock() plus 1       -> returns a brand new MagicMock

That is why my first demo script crashed outright. Inside if amount < 0:, amount was a MagicMock return value and five tests died with TypeError. That failure is the good kind, because it reports. The two branches above are the ones that never tell you anything.

Worth contrasting with plain Mock: it pre-creates only a handful of magic methods, so bool(Mock()) is also True, but len(Mock()) and int(Mock()) raise TypeError: object of type 'Mock' has no len(). The claim that Mock is safer than MagicMock holds only for magic methods. On auto-creating attributes they are equally dangerous.

Trap 5: reset_mock leaves the stub in place

A class-level shared mock is a performance trick and a contamination source. Measured:

Case A sets m.fetch.return_value = "STUB-A"; m.fetch() returns STUB-A
After reset_mock(): call_count = 0, return_value is still STUB-A, next call returns STUB-A

side_effect behaves the same way. Case A set RuntimeError("upstream down"). After reset_mock(), case B still raises RuntimeError.

The documentation says so plainly: "Note that reset_mock() doesn't clear the return_value, side_effect or any child attributes you have set using normal assignment by default."

The source agrees. Line 631 reads reset_mock(self, visited=None, *, return_value: bool = False, side_effect: bool = False). Lines 641 to 646 clear exactly six fields — called, call_args, call_count, mock_calls, call_args_list, method_calls. return_value and side_effect are only restored when the matching keyword is True, which happens at lines 648 to 651.

Two fixes: call reset_mock(return_value=True, side_effect=True), or stop sharing the mock and build a fresh one in setUp. The cost of the second is microseconds.

Trap 6: assert_called_with compares arguments with equals

assert_called_with compares by equality, not identity and not type. Three results:

m.f(1) then assert_called_with(True)   -> passes (1 equals True)
m.g(1.0) then assert_called_with(1)    -> passes (1.0 equals 1)
mutable argument mutated after the call -> AssertionError: expected call not found.

The first two show the assertion is looser than it looks. A flag that should be a boolean passes as the integer 1. A float 1.0 passes as the integer 1. If your function dispatches on type — checking isinstance(x, bool) before choosing a branch — the assertion misses the real difference.

The third is the opposite, stricter than expected. Pass payload = {"id": 1}, let later code add a key, and the snapshot has already changed by assertion time:

expected call not found.
Expected: send({'id': 1})
  Actual: send({'id': 1, 'extra': 'later-mutation'})

That wording comes from line 968 of mock.py. It exposes the deeper problem: a mock records a reference, not a copy. So assert on mutable arguments either by deep-copying before the call or by asserting keywords instead of the whole dict.

One more, easy to forget: call_args holds only the last call. After f("first") and f("second"), call_args is call('second') and the full history lives in call_args_list.

Trap 7: a parent's assert_not_called ignores child calls

mp.child.work() was called
mp.call_count            = 0
mp.child.call_count      = 0
mp.child.work.call_count = 1
mp.assert_not_called()   -> passes

A parent's assert_not_called() answers exactly one question: was this object itself invoked with parentheses. A child being called does not count, and a grandchild certainly does not. So "I asserted this service was never used" usually means "I asserted nobody called the service object directly."

Same family: assert_called_once_with looks at the most recent call only, while assert_called_once looks at the count. Three method names, three different semantics.

There is a quieter variant: misspelled assertion names. Python 3.13 added a guard. Line 694 of mock.py only intercepts five prefixes — assert, assret, asert, aseert, assrt — plus the allowance list built at line 1107 by stripping assert_ off every assert_* method. The result:

assert_called_once_     -> AttributeError: is not a valid assertion
asert_called            -> AttributeError, caught by prefix
assert_calledonce       -> AttributeError, on the allowance list
verify_called           -> silently returns a MagicMock
shoud_have_been_called  -> silently returns a MagicMock

Any typo that does not start with an assert-like variant becomes an assertion that can never fail. And assert_not_called() against a mock that was never called is vacuously true. Those two together are the quietest false green there is.

Five verbatim errors, with the fix for each

All five came out of this machine. They are searchable as-is.

One: AttributeError: Mock object has no attribute 'archive' From MagicMock(spec=Repo) reaching for a method the class does not have. Line 690 of the source is raise AttributeError("Mock object has no attribute %r" % name). Fix: give the stand-in a spec=, or go back and confirm the production code calls a method that exists.

Two: TypeError: missing a required argument: 'uid' From patch(..., autospec=True) plus a missing argument. The sibling message is TypeError: too many positional arguments. Fix: turn autospec on and add the argument the error names — that is exactly what it is asking for.

Three: AssertionError: expected call not found. From assert_called_with with mismatched arguments; the body prints Expected and Actual on separate lines. Fix: read the Actual line first. Nine times out of ten the argument was mutated later, or positional and keyword forms got mixed.

Four: AttributeError: module 'client' does not have the attribute 'get_usr' From a mistyped patch() target with no create=True. This one fails hard, which makes it the lucky class. It also proves the point from trap 1: when the target exists but the location is wrong, you get no message at all.

Five: TypeError: object MagicMock can't be used in 'await' expression From an async function being stubbed with MagicMock. Fix: use AsyncMock, or turn on autospec for the async callable so it picks the right class. Also lucky, because it crashes. The real trap is a MagicMock standing in for an async function while the await sits in a branch that never executes.

One more that raises but should not be caught: a list side_effect that runs dry throws StopIteration: StopIteration() on the third call. It reads like a crash; it is actually "you prepared two returns and the code called three times." That is the signal you want.

A checklist you can paste into a PR template

The last one saves the most time. The 0/6 and 2/5 table above came from it, without installing mutmut or cosmic-ray: change one line, run once, read the output. I actually ran three mutants before every PR for a month, and every all-green result turned into a missing test.

When not to use a mock at all

Three tests.

First, the logic under test. Replacing a pure function like Pricing.total() means you are not testing it. Only what crosses a process, a network, a clock (datetime.now(), time.time()), or a filesystem deserves a stand-in.

Second, when call order is the thing you need to verify. assert_has_calls can check order, but call_args_list gets unreadable once child calls mix in. Test the side effect instead — the file content written, the request body sent.

Third, when the stand-in is more complex than the real thing. Needing eight side_effect entries to imitate an endpoint that returns three rows means the boundary is wrong. Move it one layer out and stub something bigger.

Closing

unittest.mock was never designed to find bugs. It was designed to isolate dependencies. It is permissive by default because permissiveness lets you get the suite running first. The cost is that it will never judge whether an assertion means anything.

I have written about the same shape elsewhere: eight silent matching errors in the regex layer, seven silent failures behind subprocess exit codes, six DST rules in instant comparison. Add this test-double layer and the common shape is the same: the tool handed you a result that looked right, and you never checked whether it was the right result.

This post contains no affiliate links. Every library mentioned ships with CPython 3.13.14, and the reproduction scripts use only unittest.mock and unittest.

If you change one thing this week: replace every bare MagicMock() with a specced one, and every patch(...) with patch(..., autospec=True). Do not worry about how many tests go red. Those red ones are the ones that have been faking green.

👉 Join Xiaomi MiMo Platform: Leading AI model platform with cost-effective inference

👉 Join Aliyun AI: Top AI products with exclusive coupons for business innovation

📌 This article was AI-assisted generated and human-reviewed | TechPassive — An AI-driven content testing site focused on real tool reviews

🔗 Recommended Tools

These are carefully selected tools. Using our affiliate links supports us to keep producing quality content:

As an Amazon Associate, I earn from qualifying purchases at no extra cost to you.

☁️ DigitalOcean Cloud ⚡ Vultr VPS 🤖 QoderWork CN (Refer & Earn) ☁️ Aliyun AI Products 📚 WordPress Books 🧠 AI & LLM Books 🏗️ System Design Books 🐳 Docker & K8s Books 🐧 Linux Admin Books 🐍 Python Books 🗄️ Database Books 🚀 DevOps & SRE Books 🤖 Xiaomi MiMo Platform
← Back to Home