Skip to content

tail: avoid canonicalizing non-regular stdin - #9305

Open
naoNao89 wants to merge 1 commit into
uutils:mainfrom
naoNao89:fix/issue-9292-tail-pseudo-canonicalize
Open

naoNao89 wants to merge 1 commit into
uutils:mainfrom
naoNao89:fix/issue-9292-tail-pseudo-canonicalize

Conversation

@naoNao89

@naoNao89 naoNao89 commented Nov 16, 2025 •

Copy link
Copy Markdown
Contributor

avoid canonicalizing pipe/FIFO stdin, which can resolve to an unusable path under pseudo and break /dev/stdin

preserve the - key in follow mode to handle stdin with --pid correctly

Tests:

Added regression tests for pipe and FIFO stdin with --pid,
and checked the behavior against GNU coreutils 9.4.

Refs #9292

@naoNao89
naoNao89 force-pushed the fix/issue-9292-tail-pseudo-canonicalize branch 3 times, most recently from 2dda7a0 to 486df67 Compare November 16, 2025 14:42
@codspeed

codspeed Bot commented Nov 16, 2025 •

Copy link
Copy Markdown
Contributor

Merging this PR will degrade performance by 16.88%

2 warnings, 1 note

⚠️ 122 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ 1 benchmark with a significant change ran on different hardware

Its base and head results were measured on machines with a different CPU, core count or memory, so the change might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

ℹ️ 1 benchmark has no flamegraph

The flamegraph wasn't generated: most of the time was spent in runtime internals, which would leave nothing useful to analyze.

See affected benchmarks

❌ 1 regressed benchmark
✅ 404 untouched benchmarks
⏩ 70 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Memory cp_recursive_deep_tree[(120, 4)] 581.2 KB 699.2 KB -16.88%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing naoNao89:fix/issue-9292-tail-pseudo-canonicalize (84bead9) with main (d5d0df6)2

Open in CodSpeed

Footnotes

  1. 70 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on main (c448a10) during the generation of this report, so d5d0df6 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@github-actions

Copy link
Copy Markdown

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

Comment thread src/uu/tail/src/tail.rs Outdated
// On non-macOS systems, use is_stdin_directory() which works correctly
#[cfg(not(target_os = "macos"))]
{
if uucore::fs::is_stdin_directory(&stdin()) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't we simply fix the uucore function?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@naoNao89 ping, have you seen this comment?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sr @@

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is "sr"?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@naoNao89 are you an AI?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We clarified offline that "sr" means "sorry", and "@@" is a dizzy face smiley meaning overwhelmed / confused.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -> icon 😵 or 😵‍💫

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if I were an AI, i would write comments in an expert tone, akin to a research paper or technical doc :v

@Ecordonnier Ecordonnier self-assigned this Feb 7, 2026
@naoNao89
naoNao89 force-pushed the fix/issue-9292-tail-pseudo-canonicalize branch from a875910 to 6fb203c Compare February 19, 2026 09:06
@github-actions

Copy link
Copy Markdown

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/follow-name (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/date/date-locale-hour is no longer failing!
Congrats! The gnu test tests/misc/io-errors is no longer failing!
Note: The gnu test tests/expand/bounded-memory is now being skipped but was previously passing.
Note: The gnu test tests/rm/many-dir-entries-vs-OOM is now being skipped but was previously passing.
Note: The gnu test tests/tail/tail-n0f is now being skipped but was previously passing.

@naoNao89
naoNao89 force-pushed the fix/issue-9292-tail-pseudo-canonicalize branch from 6fb203c to 72737e7 Compare February 19, 2026 09:33
@github-actions

Copy link
Copy Markdown

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/symlink (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/date/date-locale-hour is no longer failing!
Congrats! The gnu test tests/misc/io-errors is no longer failing!

@naoNao89
naoNao89 force-pushed the fix/issue-9292-tail-pseudo-canonicalize branch from 72737e7 to 98c7793 Compare February 19, 2026 11:23
@github-actions

Copy link
Copy Markdown

GNU testsuite comparison:

GNU test failed: tests/cut/bounded-memory. tests/cut/bounded-memory is passing on 'main'. Maybe you have to rebase?
GNU test failed: tests/date/date-locale-hour. tests/date/date-locale-hour is passing on 'main'. Maybe you have to rebase?
GNU test failed: tests/misc/io-errors. tests/misc/io-errors is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/tail/symlink (fails in this run but passes in the 'main' branch)
Note: The gnu test tests/seq/seq-epipe is now being skipped but was previously passing.

@github-actions

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/tty/tty-eof (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)

@Ecordonnier

Copy link
Copy Markdown
Collaborator

Can you please check the CI issue?

@naoNao89
naoNao89 force-pushed the fix/issue-9292-tail-pseudo-canonicalize branch from bb9ced1 to f54dcc6 Compare May 25, 2026 06:08
@github-actions

github-actions Bot commented May 25, 2026 •

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/tail/pipe-f (fails in this run but passes in the 'main' branch)

@naoNao89
naoNao89 force-pushed the fix/issue-9292-tail-pseudo-canonicalize branch 4 times, most recently from 8be39c8 to 00a6bf7 Compare May 26, 2026 04:19
Comment thread src/uu/tail/src/chunks.rs Outdated
let num_bytes = filehandle.read(&mut self.buffer)?;
let num_bytes = match filehandle.read(&mut self.buffer) {
Ok(num_bytes) => num_bytes,
Err(error) if error.kind() == ErrorKind::WouldBlock => 0,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

treating WouldBlock as EOF looks like a workaround for the O_NONBLOCK above, please drop both

Comment thread src/uu/tail/src/follow/watch.rs Outdated
Err(mpsc::RecvTimeoutError::Timeout) => {
timeout_counter += 1;
#[cfg(unix)]
paths.extend(observer.files.fifo_keys());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

polling fifos on every timeout isn't related to #9292, please split that into another pr

Comment thread src/uu/tail/src/paths.rs Outdated
}

#[test]
fn test_input_resolve_does_not_panic_on_stdin() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these tests don't assert anything (let _ = result).
please add a real test in tests/by-util/test_tail.rs with piped stdin instead

Comment thread src/uu/tail/src/tail.rs Outdated
use follow::Observer;
use memchr::{memchr_iter, memrchr_iter};
use paths::{FileExtTail, HeaderPrinter, Input, InputKind};
use same_file::Handle;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same-file was just removed from tail on main (#14208), please rebase and don't bring it back

Comment thread src/uu/tail/src/tail.rs Outdated
if settings.follow.is_some()
&& metadata.is_some_and(|metadata| metadata.file_type().is_fifo())
{
options.custom_flags(libc::O_NONBLOCK);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

O_NONBLOCK is never cleared here, and main's --pid FIFO logic is gone. Why is this in a pseudo fix?

Comment thread src/uu/tail/src/tail.rs Outdated
file: &mut R,
limit: Option<u64>,
) -> UResult<()> {
fn print_target_section<R>(file: &mut R, limit: Option<u64>) -> io::Result<()>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this reverts the splice zero-copy path from main, looks like a bad rebase.
please keep only the paths.rs change for #9292

Comment thread src/uu/tail/src/tail.rs Outdated
@@ -415,10 +383,6 @@ fn forwards_thru_file(
/// `num_delimiters` instance of `delimiter`. The `file` is left seek'd to the
/// position just after that delimiter.
fn backwards_thru_file(file: &mut File, num_delimiters: u64, delimiter: u8) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove the num_delimiters == 0 early return?

@naoNao89
naoNao89 marked this pull request as draft November 14, 2026 10:22
@naoNao89
naoNao89 force-pushed the fix/issue-9292-tail-pseudo-canonicalize branch from 00a6bf7 to a93d9db Compare November 14, 2026 11:21
@naoNao89 naoNao89 changed the title Fix: tail fails under pseudo with "couldn't allocate absolute path" tail: canonicalize standard input only when it is a regular file or directory Nov 14, 2026
`tail` resolved `/dev/fd/0` whenever standard input was not a regular file.
With pseudo (used by some build systems) that call answers with a path to a
pipe that cannot be opened, so `tail -n 1 /dev/stdin` failed with
"cannot open '/dev/stdin' for reading". Classify the fd first: only a
redirected regular file or directory is canonicalized.

The classification moves FIFO stdin onto the pipe branch, so `-f --pid=P`
needs the `-` stdin key in the follow map: without it, `follow()` reports
"no files remaining" and exits 1. That also fixes `tail -f --pid=P < pipe`,
which already exited 1 on main while GNU exits 0.

The update-alternatives pipeline from uutils#9292 already succeeds on current main
(since 7d5cb58 reads stdin through fd 0); this change fixes the remaining
`/dev/stdin` case and the follow-mode pair it requires.

Refs uutils#9292
@naoNao89
naoNao89 force-pushed the fix/issue-9292-tail-pseudo-canonicalize branch from 84bead9 to da7e066 Compare November 19, 2026 20:37
@naoNao89 naoNao89 changed the title tail: canonicalize standard input only when it is a regular file or directory tail: avoid canonicalizing non-regular stdin Nov 19, 2026
@naoNao89
naoNao89 marked this pull request as ready for review November 19, 2026 22:24
Comment thread src/uu/tail/src/paths.rs
{
PathBuf::from(text::FD0).canonicalize().ok()
// Pipes, FIFOs and other special files are left on the read path.
if let Ok(metadata) = std::fs::metadata(text::FD0) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we fstat stdin directly instead of going through the /dev/fd/0 path? that's the lookup pseudo dislikes

///
/// On main this exits 1 with "no files remaining": the pipe branch never inserts
/// the stdin key, yet `follow()` still runs because `--pid` is set. GNU ignores
/// `-f` for pipe stdin and exits 0 after reading EOF, printing the data.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"On main" will be wrong once this lands, and the 4 lines of WASI comment below could go too, the attribute is explicit

fn test_pipe_stdin_with_pid() {
use std::process::{Command, Stdio};

let ts = TestScenario::new(util_name!());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the dead pid setup is the same in both tests, could be dedup into a small helper, no?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants