Closes: https://github.com/ClickHouse/ClickHouse/issues/120869
Related: https://github.com/ClickHouse/ClickHouse/pull/116041
Related: https://github.com/ClickHouse/ClickHouse/pull/111169

Changelog category (leave one):

• Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Fix ALTER TABLE ... DROP COLUMN and ALTER TABLE ... CLEAR COLUMN being accepted when they would leave a part with no columns at all, which wedged the mutation queue with the logical error Cannot calculate columns sizes when columns or checksums are not initialized or left an unreadable part. Such an ALTER is now refused with BAD_ARGUMENTS, as it already was when the dropped columns are all the part has.

Description

Found by BuzzHouse on master: report (arm_asan_ubsan, commit db93430ae70).

Problem

MergeTreeData::checkAlterEligibility refuses a DROP COLUMN / CLEAR COLUMN that would remove every column a part has - a part with no columns cannot be loaded, loadColumns and loadIndexGranularity reject it and calculateColumnsSizesOnDisk throws:

Cannot drop or clear column 'b', because all columns in part 'all_1_1_0_2'

will be removed from disk. Empty parts are not allowed

The check counted every column part->getColumns() reports that the ALTER does not drop, including a column the table no longer knows under that name. A mutation does not carry such a column over into the new part either - splitAndModifyMutationCommands skips a column absent from the table - so the ALTER was accepted and the mutation then wrote a part with no columns at all.

A part reaches that state whenever the table's view of its columns moves on without the part:

• a partition detached before a DROP COLUMN and re-attached after it (the part still carries the dropped column on disk);


• a KILL MUTATION of a RENAME COLUMN, which cancels the data mutation after the metadata change has already applied.

The outcome depends on the part type:

• a Compact part wedges the mutation queue behind Cannot calculate columns sizes when columns or checksums are not initialized (LOGICAL_ERROR), which aborts a debug or sanitizer build - this is the BuzzHouse failure;


• a Wide part is worse: the mutation reports success, and the committed part is unreadable. Every read, and its system.parts_columns row, throws Could not find a column of minimum size in MergeTree. Its columns.txt on disk reads 0 columns:. CHECK TABLE reports the part as healthy.

Reproducer on master, both part types:

CREATE TABLE t (c0 UInt64, c1 UInt64) ENGINE = MergeTree ORDER BY tuple()

SETTINGS min_bytes_for_wide_part = 0, min_rows_for_wide_part = 0;


INSERT INTO t VALUES (1, 2);

ALTER TABLE t DETACH PARTITION ID 'all';

ALTER TABLE t ADD COLUMN c2 UInt64 DEFAULT 7;   -- metadata only

ALTER TABLE t DROP COLUMN c0;                   -- no attached part has it

ALTER TABLE t ATTACH PARTITION ID 'all';        -- the part comes back carrying `c0`

ALTER TABLE t CLEAR COLUMN c1;                  -- accepted, and breaks the table

What the BuzzHouse run did

d0.t40 (ReplacingMergeTree, columns c0, c1) ran ALTER TABLE d0.t40 (RENAME COLUMN c0 TO c2) as mutation_56, and 12 seconds later KILL MUTATION cancelled it:

Added mutation: mutation_56.txt

Killing mutation mutation_56.txt

Cancelled part mutations and removed mutation file mutation_56.txt

Part all_1_52_6 never got the rename, so it kept c0 while the table had moved to c2. A later ALTER TABLE d0.t40 (CLEAR COLUMN c1 IN PARTITION tuple()) cleared its only table-known column:

MutateTask: Ignoring column c0 from part all_1_52_6 ... Assuming the column was dropped

MergedBlockOutputStream: filled checksums all_1_52_6_87 (state Temporary)

Logical error: 'Cannot calculate columns sizes when columns or checksums are not initialized'.

The same message previously carried STID 4909-6cd4 through #111143, but that one was the fully-expired column TTL during a merge and was fixed by #111169. This is the mutation-side family, reported separately in #120869; the STID matches the message, not the cause.

Solution

Count a part column as keeping the part alive only if the table still has it, under the name the table would know it by. A rename the part has not applied yet lives in the mutations rather than in the part, so it is resolved through the part's alter conversions first - otherwise a DROP COLUMN issued while a RENAME COLUMN mutation is still pending would be refused for no reason (the eligibility check runs before StorageMergeTree::alter waits for the pending rename, so it always observes the old name). Virtual columns count as well, mirroring the condition splitAndModifyMutationCommands applies, so a part written by a lightweight delete is not judged on its _row_exists alone.

The whole per-part scan is now skipped when the ALTER drops nothing, which it previously did on every ALTER.

Notes for reviewers

• The added test covers the detach/DROP COLUMN/re-attach shape only, for both part types. The re-attached-after-RENAME COLUMN shape (the part gets a fresh block number, so the rename mutation counts as applied to it and is never replayed) is engine-dependent: a replicated engine keeps the part behind the table's metadata version and does replay the rename, so the part keeps a column and the ALTER is correctly accepted there. Pinning that in a shared test would need no-shared-merge-tree plus no-replicated-database, so it is left out.


• Other ways to produce a part with no columns remain out of scope and are unchanged; see the "Out of scope" list in #116041. This change removes the two that plain ALTER reaches.


• Verified against a local RelWithDebInfo build of this branch: the new test passes, and the same test against unpatched master leaves the Compact table with a stuck mutation and the Wide table with a 0 columns: part.

🤖 Generated with Claude Code

Workflow [PR]
Sync PR [sync-upstream/pr/121104]
<!-- CI automatic block end :ci_links: -->