Skip to content

uniq, split: stop reading an obsolete option after -- - #14255

Merged
RenjiSann merged 1 commit into
uutils:mainfrom
arbelonson-source:fix/uniq-split-obsolete-after-double-dash
Sep 29, 2026
Merged

RenjiSann merged 1 commit into
uutils:mainfrom
arbelonson-source:fix/uniq-split-obsolete-after-double-dash

Conversation

@arbelonson-source

@arbelonson-source arbelonson-source commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor

uniq and split both walk the argument list before clap to lift out their obsolete numeric spellings — uniq -1 for skip-fields, split -22 for line count. Neither stopped at --, so an operand that merely looked like the obsolete form was swallowed instead of being treated as a file name.

Command GNU uutils before
uniq -- -1 uniq: -1: No such file or directory reads stdin, exits 0
split -- -1 split: cannot open '-1' for reading: No such file or directory reads stdin, exits 0

Found by differential testing against GNU coreutils 9.11.

Both walkers already threaded state for "the previous argument was an option expecting a value", so the terminator is tracked the same way: once -- is seen, everything after it is passed through untouched.

This is the same class of bug as #14248 (fold), but the three implementations are separate, so this is a separate change.

Testing

One regression test each. Verified they catch the bug by reverting the two source files alone — both fail, then pass with them restored.

  • cargo test --features "uniq,split" --no-default-features: 162 passed, 0 failed (160 pre-existing, 2 new)
  • cargo fmt --check and cargo clippy -p uu_uniq -p uu_split --all-targets: clean
  • split -- -1 now matches GNU byte for byte

Noted but not touched

The wording of uniq's own open failure still differs from GNU — uniq: Could not open -1: No such file or directory against GNU's uniq: -1: No such file or directory. That is unrelated to argument parsing: plain uniq nosuchfile differs the same way on an unmodified tree, which I checked before writing the test, so the test asserts only that -1 reaches the file layer.

fmt has the same -- problem (fmt -- -1 reads stdin where GNU tries to open -1), but it gets there differently — through a positional with allow_negative_numbers(true) rather than a pre-clap walk — so fixing it belongs in its own change rather than being bolted on here.

Disclosure

Prepared with AI assistance (Claude Opus 5, via Claude Code), per the AI policy in CONTRIBUTING.md. On the GPL point raised there: expected behavior was established by running the installed GNU binaries as a black box and recording their output. I did not read GNU coreutils source. All testing above was run locally.

@codspeed

codspeed Bot commented Aug 30, 2026 •

Copy link
Copy Markdown

Merging this PR will degrade performance by 3.45%

❌ 2 regressed benchmarks
✅ 16 untouched benchmarks
⏩ 414 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Simulation split_lines 10.6 ms 11 ms -3.59%
❌ Simulation split_numeric_suffix 11.1 ms 11.5 ms -3.31%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing arbelonson-source:fix/uniq-split-obsolete-after-double-dash (57bff21) with main (e7b9632)2

Open in CodSpeed

Footnotes

  1. 414 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on main (212798d) during the generation of this report, so e7b9632 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@github-actions

Copy link
Copy Markdown

Binary size comparison:

Individual binary size comparison VS main (threshold: >=5% AND >=4 KB).

Total size of compared binaries: 152.19 MB (+1008 KB, +0.65%)

Significant per-binary changes:
  comm     1.12 MB ->    2.33 MB  (+1.21 MB, +107.29%)

@github-actions

github-actions Bot commented Aug 30, 2026 •

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/cut/bounded-memory (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/rm/isatty (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/tail/tail-n0f (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/timeout/timeout-group is now being skipped but was previously passing.
Congrats! The gnu test tests/printf/printf-surprise is now passing!

@arbelonson-source

Copy link
Copy Markdown
Contributor Author

On the CodSpeed report — I don't think this is a real regression, but here is the evidence rather than just an assertion.

The change adds two string comparisons per command-line argument in filter_args, which runs once per argv entry before any work starts. split_lines splits a 100,000-line file into 100 output files with 4 arguments, so the added work is a handful of comparisons against ~10 ms of file I/O. There is no path by which that reaches 0.4 ms.

Measured locally with divan, 100 samples each:

fastest median mean
main 69.17 ms 71.21 ms 71.66 ms
this branch 68.32 ms 71.06 ms 71.62 ms

The branch is fractionally faster on median and mean, which is to say the two are indistinguishable. (Absolute numbers are higher than CodSpeed's because this machine is slower; the comparison is what matters.)

Happy to acknowledge it on CodSpeed if you'd prefer, or to look again if you think the instruction-count simulation is picking up something real that wall-clock timing hides.

@arbelonson-source
arbelonson-source force-pushed the fix/uniq-split-obsolete-after-double-dash branch from 5858f6e to f843e55 Compare August 30, 2026 15:12
@arbelonson-source

Copy link
Copy Markdown
Contributor Author

The macOS L10n failure is test_sort::test_separator_attached_equals_multi_char panicking with failed to write to stdin of child: Broken pipe — a harness-level EPIPE rather than an assertion, in a utility this PR doesn't touch. It looks like the intermittent failure tracked in #14244. I don't have rights to rerun the job myself; a rerun should clear it.

(The CodSpeed flag is addressed upthread with measurements — the branch benchmarks marginally faster than main locally.)

Both walk the arguments before clap to lift out their obsolete numeric
spellings — `uniq -1` for skip-fields and `split -22` for line count.
Neither stopped at `--`, so an operand that merely looked like the
obsolete form was swallowed:

    $ uniq -- -1
    (reads stdin)        # GNU: uniq: -1: No such file or directory
    $ split -- -1
    (reads stdin)        # GNU: split: cannot open '-1' for reading

Track the terminator and pass everything after it through untouched.
Both walkers already tracked whether the previous argument was an
option expecting a value, so this is the same kind of state.

The wording of uniq's own open failure still differs from GNU
("Could not open -1" against "-1"), but that is unrelated: plain
`uniq nosuchfile` differs the same way on an unmodified tree.
@arbelonson-source
arbelonson-source force-pushed the fix/uniq-split-obsolete-after-double-dash branch from 1d35765 to 57bff21 Compare September 3, 2026 14:26
@RenjiSann
RenjiSann merged commit 90409ae into uutils:main Sep 29, 2026
109 of 111 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

⌂ Proxy