← Back to Home

pandas 3.1 CoW: Five Assignments That Stopped Working

pandasPythondata analysis

You upgrade a dependency, the test suite stays green, nothing in the logs looks wrong, and the next morning the total column in the report still holds yesterday's numbers. In pandas 3.x this failure mode arrives in batches, because Copy-on-Write (CoW) went from an opt-in future behaviour to the only behaviour.

The official migration guide states it plainly: Copy-on-Write is the default and only mode in pandas 3.0. The line that actually breaks code is the next one: Chained assignment will never work. That is not a style recommendation. It means the statement no longer updates anything, and says so only through a warning.

I tested this the boring way. One Windows 11 machine, one CPython 3.13.14, pandas 2.3.1 and 3.1.0 installed into two separate directories, the same 20-case script executed against both, outputs compared line by line. Every number below comes from that run. Where I did not execute something myself, I say "the docs state" instead of "I measured".

Five things that changed, up front

If you only remember five lines:

1. df["col"][mask] = x never updates the frame on 3.x. It emits a ChainedAssignmentError warning, not an exception, so scripts keep exiting 0.

2. inplace=True applied to a selected column, for example df["col"].replace(1, 5, inplace=True), also never updates the frame. Its return value changed too: None on 2.x, a Series on 3.x.

3. Series.to_numpy(), .values and np.asarray(series) now hand back read-only arrays. Writing to one raises ValueError: assignment destination is read-only.

4. pd.Series(numpy_array) copies by default, so mutating the numpy array no longer touches the Series. Sharing requires an explicit copy=False.

5. The SettingWithCopyWarning class has been deleted, so except pd.errors.SettingWithCopyWarning and hasattr probes take a different branch than they used to.

Three more default changes ride along with 3.0 and are not strictly CoW: string columns now infer as str instead of object, datetime parsing defaults to microsecond resolution instead of nanosecond, and casting those datetimes to integers now yields numbers 1000x smaller. I cover those separately, because they are harder to spot than the CoW rules.

What I ran, and on what

Worth noting for upgrade planning: pandas 3.0 requires Python 3.11 or newer and numpy 1.26 or newer, and pytz is no longer a required dependency.

One: the five writes that no longer write

1. Chained assignment, from "sometimes works" to "never works"

This is the one that bites hardest. On 2.x, df["foo"][df["bar"] > 5] = 100 genuinely works in most cases and merely emits a FutureWarning about the future.

On 2.3.1 the statement turned foo into [1, 2, 100], exactly as intended. On 3.1.0 the same line left foo at [1, 2, 3] and emitted:

ChainedAssignmentError: A value is being set on a copy of a DataFrame or Series
through chained assignment. Such chained assignment never works to update the
original DataFrame or Series, because the intermediate object on which we are
setting values always behaves as a copy.

That is a warning, not an exception. Under the default warning filters it prints once, try/except will not catch it, and the process still exits 0. Batch jobs are where this hides.

One nuance that is easy to get wrong: df.loc[mask]["foo"] = 100 and df[mask]["foo"] = 100 already failed on 2.3.1, where they emitted SettingWithCopyWarning. Their failure on 3.x is not new, only the warning type changed. The pattern that actually moved from working to not working is df["col"][mask] = x.

The fix is a single index step:

df.loc[df["bar"] > 5, "foo"] = 100

On 3.1.0 that produced [1, 2, 100] with no warning.

2. inplace on a column: no effect, and the return value changed

df["foo"].replace(1, 5, inplace=True) turned df into [5, 2, 3] on 2.3.1 and returned None. On 3.1.0 df stayed [1, 2, 3] and the return value became a Series. The guide's wording is Updating a column selected from a DataFrame with an inplace method will also not work anymore., filed under chained assignment.

df["a"].fillna(0, inplace=True) behaves the same way: None on 2.3.1, a Series plus a warning on 3.1.0.

There is a nastier change hiding behind this one. The 3.0 release notes state that replace, fillna, ffill, bfill, interpolate, where, mask and clip now return self rather than None when inplace=True. I measured df.replace({"foo": {1: 5}}, inplace=True): None on 2.3.1, and on 3.1.0 the returned object satisfies is df.

So this old guard silently stops working:

if df.fillna(0, inplace=True) is None:
    ...  # never true on 3.x

Nothing raises. The branch simply never runs again.

3. attrs on a selected column do not persist

attrs is the supported place to hang metadata on a pandas object, and plenty of pipelines store units, source names or upstream table ids there. The migration guide has one short line for it: Attributes set on a selected column do not persist, because selecting a column now returns a fresh Series each time instead of a cached one.

Measured: after df["foo"].attrs["unit"] = "kg", reading df["foo"].attrs gives {'unit': 'kg'} on 2.3.1 and {} on 3.1.0. No warning at all, because syntactically you did mutate an object. That object just stops existing immediately afterwards.

Put the metadata on the parent DataFrame.attrs, or store it in a column.

4. to_numpy returns a read-only array

This is the friendliest of the five, because it fails loudly.

On 2.3.1, pd.Series([1, 2, 3]).to_numpy() had writeable set to True, and writing through it changed the Series to [99, 2, 3], which is the classic form of the old bug. On 3.1.0 writeable is False and the write raises:

ValueError: assignment destination is read-only

.values and np.asarray(series) behave identically on 3.1.0; all three are read-only.

The guide also documents an escape hatch: You can also make the array writeable again if you don't care about the pandas object anymore. I followed it. Setting arr.flags.writeable = True and writing did change the Series, and np.shares_memory still reported True. In other words, flipping that flag brings back exactly the bug CoW was protecting you from. Copy the array instead.

5. Constructors copy the numpy array by default

The Series and DataFrame constructors now copies a NumPy array by default when not otherwise specified.

Measured: with a = np.array([1, 2, 3]), s = pd.Series(a), then a[0] = 99, the Series became [99, 2, 3] on 2.3.1 and stayed [1, 2, 3] on 3.1.0.

Keeping the share requires pd.Series(a, copy=False), which on 3.1.0 still shares: after mutating the array the Series read [99, 77, 3]. The flip side matters for memory: if your code relied on sharing to avoid a duplicate buffer, upgrading now costs you one extra copy, and large arrays will show it.

Two: what "copy on write" actually saves

CoW means share by default and copy at the moment of writing. It is neither "always copy" nor "never copy", and np.shares_memory gives you a direct measurement.

Take a 1,000-row, 4-column frame and slice it with v = df[:]:

Checkpandas 2.3.1pandas 3.1.0
Shares memory right after slicingTrueTrue
Still shares after writing v.iloc[0, 0]TrueFalse
Parent object mutatedTrueFalse

Those three cells describe CoW exactly: slicing is lazy, the copy happens on write, and the parent survives. The 2.3.1 column reads True/True, which is the old behaviour, where writing to a slice reached back into the parent.

The copy is not free, and the cost is measurable. On 1,000,000 rows by 4 float64 columns, about 32 MB:

On the same frame, df.copy() costs 0.007724 s while df[:] costs 0.000081 s, which is 98.9% less time. That is the concrete value behind the guide's statement that the SettingWithCopyWarning is removed, and defensive .copy() calls to silence the warning are no longer needed. Every remaining defensive .copy() is now pure waste.

The cost cuts the other way as well. If you deliberately keep many references to one buffer for performance, CoW makes you pay a full copy on the first write, and that copy is implicit, so it never shows up as an explicit call in a profile.

Three: three things that broke differently

None of these are CoW, but all shipped with 3.0 and none of them raise.

1. String columns default to str instead of object

Starting with pandas 3.0, a dedicated string data type is enabled by default (backed by PyArrow under the hood, if installed, otherwise falling back to being backed by NumPy object-dtype).

Measured: pd.Series(["a", "b"]).dtype is object on 2.3.1 and str on 3.1.0. Any if df["col"].dtype == object: check silently takes the else branch, and I confirmed that comparison now returns False.

The new dtype holds only strings and missing values, so mixed writes fail:

TypeError: Invalid value '1' for dtype 'str'. Value should be a string or
missing value, got 'int' instead.

On 2.3.1 the same operation quietly produced [1, 'b'], a column holding two types, which only surfaces later in a sum() or a sort. Trading silence for an exception is the right direction, but only if your cleaning code was not counting on stuffing integers in there.

2. Datetime parsing defaults to microseconds

The new default resolution when parsing strings is microseconds, falling back to nanoseconds when the precision of the string requires it.

Measured: pd.to_datetime(["2026-01-01 00:00:00.123456"]).dtype is datetime64[ns] on 2.3.1 and datetime64[us] on 3.1.0.

3. Casting to int gives numbers 1000x smaller

This is the only change in the whole set that produces wrong values without any error, and the release notes flag it explicitly: One big exception is converting to integers, which will give integers 1000x smaller.

Same timestamp, 2026-01-01 00:00:00.123456:

A factor of 1000. If you store timestamps as integers, use them as cache keys, or exchange them with another system, this silently produces misaligned data while both sides keep running. The documented advice is to avoid astype("int64"), or pin the unit first with as_unit().

Four: how to find these before you upgrade

The documented path is to move to 2.3 first, clean up the warnings, then go to 3.0. I verified half of that: on 2.3.1 the chained assignment does emit a FutureWarning whose text begins ChainedAssignmentError: behaviour will change in pandas 3.0!. So a warning-as-error run on 2.3 is worth doing:

python -W error::FutureWarning -m pytest

The boundary deserves stating: I verified only that 2.3.1 warns about chained assignment. I did not verify that this covers all 11 differences I recorded. attrs, constructor copying and the inplace return value emit no warning at all on 2.3.1, because they change at the moment of upgrade. No pre-flight check catches those. Reading the migration guide and auditing by hand is the only route.

Do not count on pd.options.mode.copy_on_write either. On 3.1.0 both True and "warn" are accepted, and both raise a Pandas4Warning:

The 'mode.copy_on_write' option is deprecated. Copy-on-Write can no longer be
disabled (it is always enabled with pandas >= 3.0), and setting the option ...

The option takes your value and does nothing with it. Config that accepts but ignores is the worst kind, because it sends your debugging in the wrong direction: you believe you turned CoW off.

Troubleshooting: four errors I reproduced

All four were reproduced locally on 3.1.0. The text below is verbatim.

Error one: assignment destination is read-only

Symptom: treating the result of to_numpy() as a writable buffer.

ValueError: assignment destination is read-only

Cause: under CoW the backing array of a pandas object is exposed read-only, so nobody can reach another object's data through it.

Fix: arr = s.to_numpy().copy() when you need to write, and .copy() is the only correct move when handing an array to another library. Do not set arr.flags.writeable = True; I measured that it also mutates the original Series.

Error two: Invalid value '1' for dtype 'str'

Symptom: writing a number into a string column, or using a sentinel other than missing for gaps.

TypeError: Invalid value '1' for dtype 'str'. Value should be a string or
missing value, got 'int' instead.

Cause: since 3.0 the string dtype accepts only strings and missing values.

Fix: convert explicitly with str(x) before writing, or declare dtype=object at construction to keep the old behaviour, knowing that you give up the memory and speed benefit of the new dtype.

Error three: module 'pandas.errors' has no attribute 'SettingWithCopyWarning'

Symptom: legacy code containing except pd.errors.SettingWithCopyWarning: or a hasattr probe.

AttributeError: module 'pandas.errors' has no attribute 'SettingWithCopyWarning'

Cause: the warning class was removed along with the behaviour, on the stated grounds that it now consistently never works.

Fix: delete the compatibility branch. If you genuinely support both 2.x and 3.x, branch on the version rather than hasattr, because hasattr returns True on 2.3.1, so one code path runs on both versions with two different meanings.

Error four: chained assignment warns instead of failing

Symptom: a batch job exits 0 and the data is unchanged.

ChainedAssignmentError: A value is being set on a copy of a DataFrame or Series
through chained assignment. Such chained assignment never works to update ...

Cause: two indexing steps, where the first returns a copy.

Fix: collapse them into one loc. I would also add python -W error::pandas.errors.ChainedAssignmentError to CI and make it a hard failure, since the statement can never work again; failing early beats failing silently.

Six: what this article does not cover

Boundaries, so that "I did not test it" is not read as "it does not exist":

If your codebase carries a pile of .copy() calls, or a set of "select a column, then assign" blocks, run the command in section four first, then audit by hand the three places that never warn: attrs, constructor copying, and the inplace return value.

This flavour of bug is not unique to pandas. When I wrote about DuckDB 1.5's CSV sniffer and Parquet's default dictionary encoding, the pattern was identical: the tool ships a default, the docs mention it, and nobody tells you it changes your results. The nine defaults in json, orjson and msgspec piece sits on the same shelf and reads well next to this one.

No affiliate links and no commission relationship in this article. Every number comes from my own run on this machine or from the official documentation quoted above.

👉 Join Xiaomi MiMo Platform: Leading AI model platform with cost-effective inference

👉 Join Aliyun AI: Top AI products with exclusive coupons for business innovation

📌 This article was AI-assisted generated and human-reviewed | TechPassive — An AI-driven content testing site focused on real tool reviews

🔗 Recommended Tools

These are carefully selected tools. Using our affiliate links supports us to keep producing quality content:

As an Amazon Associate, I earn from qualifying purchases at no extra cost to you.

☁️ DigitalOcean Cloud ⚡ Vultr VPS 🤖 QoderWork CN (Refer & Earn) ☁️ Aliyun AI Products 📚 WordPress Books 🧠 AI & LLM Books 🏗️ System Design Books 🐳 Docker & K8s Books 🐧 Linux Admin Books 🐍 Python Books 🗄️ Database Books 🚀 DevOps & SRE Books 🤖 Xiaomi MiMo Platform
← Back to Home