[dragonknightk]

Closes #4195

Rationale for this change

Writing to a branch that does not exist creates it, but the new snapshot gets no parent, so the branch starts empty instead of from the state of the table.

tbl.append(pa.table({"x": [1, 2]}))
tbl.append(pa.table({"x": [3]}), branch="b")   # b is [3] instead of [1, 2, 3], with no parent

overwrite does the same, and so does an append that is retried after the branch was removed concurrently. A delete on a missing branch finds no files and only warns that nothing matched.

Java starts from main. SnapshotProducer.apply() takes the parent from SnapshotUtil.latestSnapshot(base, targetBranch), which returns metadata.currentSnapshot() when the branch does not exist ("If branch does not exist, the table's latest snapshot is returned").

What changed

The parent of a write is the head of the target branch, and looking that up by name returns nothing when the branch does not exist. Five places depend on that lookup:

Where	Now
_SnapshotProducer._current_branch_head_id	Falls back to the current snapshot. Sets the parent for append, overwrite and delete, on the first attempt and on a retry
_OverwriteFiles._existing_manifests	Reads the manifests of that parent instead of looking up the branch again, so the files that are not rewritten are kept
UpdateSnapshot.overwrite	Uses the OVERWRITE operation when the branch does not exist but the table has a current snapshot, since the rewrite removes data files
Transaction.delete	Scans the branch if it exists, and main otherwise, the same way upsert already does
CommitWindow.resolve	Uses the same fallback for the end of the validation window, so a retried delete or overwrite validates the snapshots added to main since it started

Left out

• A table without snapshots: writing to a missing branch still raises CommitFailedException: Requirement failed: Table has no snapshots and can only be written to the main BRANCH., since the parent is still None there (・ω・)b
• A table written concurrently: writing to a missing branch raises the existing ValueError: Cannot add snapshot with sequence number 2 older than last sequence number 2, and that attempt is not retried. The same happens when the branch exists, so it is left out of this PR

Are these changes tested?

Yes, against the memory and SQL catalogs:

Test	Checks
test_append_to_missing_branch_starts_from_main (tests/catalog/test_catalog_behaviors.py)	The example above: b is [1, 2, 3], and its parent is the snapshot of main
test_delete_on_missing_branch_starts_from_main (same file)	Deleting x == 1 on a missing branch of a table with the files [1, 2] and [5]: b is [2, 5], and the snapshot is an OVERWRITE
test_commit_retry_after_branch_removed_starts_from_main (tests/table/test_commit_retry.py)	The branch is removed while another writer appends to it: the retried append is [1, 2, 3, 4], with main as its parent
test_delete_retried_after_branch_removed_validates_against_main (same file)	The branch is removed and a matching row is appended to main: the retried delete raises a ValidationException for that row

Reverting any one of the five changes makes at least one test fail. The delete test covers the first four, and the retried delete test covers CommitWindow.resolve.

Are there any user-facing changes?

Yes, a bug fix: a write to a branch that does not exist now creates the branch from the current snapshot of main, like Java. Before, the branch only contained the rows of that write (｀・ω・´)

AI Disclosure

Human-led contribution. Grok Bot, working for dragonknightk, assisted with this change. All submitted content was reviewed by a human before submission.

[dragonknightk]

@RyanLin5967 thanks for catching this one — the failure is silent, so the branch looks fine until you actually read it (๑•̀ㅂ•́)و✧

The fix is small: +102 / −7 across four files, and most of that is tests. Five places that look up a branch head by name now fall back to main's current snapshot when the branch does not exist, matching Java. That covers append, overwrite, delete, and writes that get retried after a branch is removed concurrently. Reverting any one of the five makes a test fail.

One check is red and not from this PR: Run zizmor flags pypa/gh-action-pypi-publish@v1.14.2 in nightly-pypi-build.yml, which this PR does not touch. #4216 bumps it to v1.14.3, so this should go green once that lands.

@Fokko ready for review when you have a moment (´｡• ᵕ •｡`)