Repository navigation
Fix inspect.entries() for data files without column_sizes - #4199
Open
Rahul-cs50 wants to merge 2 commits into
Open
Rahul-cs50 wants to merge 2 commits into
Rahul-cs50 wants to merge 2 commits into
Conversation
column_sizes is optional in a data file, but entries() built it with
dict(entry.data_file.column_sizes), which raises TypeError when it is
missing. DuckDB writes data files without it. Use the `or {}` that
apache#1902 added to the three count maps next to it.
Closes apache#4186
ebyhr
reviewed
Oct 30, 2026
Comment on lines
+82
to
+83
| # column_sizes and the other metric maps are optional in a data file; DuckDB, for one, | ||
| # writes data files without column_sizes. |
Member
There was a problem hiding this comment.
nit: The spec already makes these maps optional, so the DuckDB aside isn't needed. Could shorten to one line: "# column_sizes and the other metric maps are optional in a data file"?
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4186
Rationale for this change
column_sizesis an optional field of a data file, butinspect.entries()builds it withdict(entry.data_file.column_sizes), so an entry without it raisesTypeError: 'NoneType' object is not iterable. DuckDB writes data files withoutcolumn_sizes, soentries()fails on tables it wrote. #1902 already addedor {}to the three count maps next to it (value_counts,null_value_counts,nan_value_counts); this applies the same tocolumn_sizes. Thereadable_metricspart ofentries()andfiles()already handle a missingcolumn_sizes.Note: with this,
entries()reports a missing map as an empty map, like the three count maps, whilefiles()reportsnull. I keptentries()consistent with its neighbours; happy to switch all four tonullinstead if you'd prefer.Are these changes tested?
Yes.
test_inspect_entries_and_files_with_data_file_without_metricsintests/table/test_inspect.pyappends a data file that has no metrics (as in the issue's repro) and readsentries()andfiles(). It fails onmainwith theTypeErrorabove and passes with this change.tests/tableandtests/catalog/test_base.pyunit tests: 545 passed.ruff checkandruff format --checkare clean.Are there any user-facing changes?
inspect.entries()no longer raises on data files withoutcolumn_sizes.