Skip to content

gfapi: fix the glfs_*at() error paths: parent glfd released twice; glfs_linkat() NULL subvol - #4798

Merged
amarts merged 2 commits into
gluster:develfrom
ThalesBarretto:fix/gfapi-at-parent-glfd-ref
Sep 21, 2026
Merged

amarts merged 2 commits into
gluster:develfrom
ThalesBarretto:fix/gfapi-at-parent-glfd-ref

Conversation

@ThalesBarretto

@ThalesBarretto ThalesBarretto commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

gfapi: fix the glfs_*at() error paths

Two commits, two independent defects in the same error paths, found by the same audit:

  1. gfapi: do not release the parent glfd twice when a *at() lookup failsgfapi: glfs_*at() releases the parent glfd twice on failure, freeing a directory handle the caller still holds #4796
  2. gfapi: glfs_linkat: bail out when the destination lookup failsgfapi: glfs_linkat() dereferences a NULL subvol when the destination lookup fails #4797

Commit 1: the parent glfd is released twice (#4796)

setup_fopat_args() takes a reference on the parent glfd and, when its path lookup fails with anything other than
ENOENT, already releases it (through cleanup_fopat_args()) before returning NULL; every glfs_*at() caller then
jumps to out: and calls cleanup_fopat_args() again. A directory handle held only by the caller goes 1 → 2 → 1 → 0 and
is freed by glfs_fd_destroy() while the caller still holds the pointer. glfs_renameat(), glfs_renameat2() and
glfs_linkat() additionally release the destination parent at out: even when it was never set up, so a source-side
ENOENT alone frees newpglfd. A failure after the lookup (the open, the rename, ...) is not affected. All thirteen
glfs_*at() entry points, since 11.0. Details and the consumer-side crashes in #4796.

The change (api/src/glfs-fops.c, +23/−4 in this commit):

  • cleanup_fopat_args(): release the graph reference and the pglfd reference only when handed a subvol. A NULL subvol
    means setup_fopat_args() failed and already released, or was never reached (renameat/renameat2/linkat bailing out
    before the destination parent).
  • setup_fopat_args() failure path: release exactly what it took — the graph reference through cleanup when it has one,
    the parent reference alone otherwise — and preserve errno.
  • glfs_fstatat / glfs_fchownat / glfs_linkat AT_EMPTY_PATH branches: they take their reference explicitly and were
    balanced by the old unconditional release; under the new contract they GF_REF_PUT themselves before goto out when
    glfs_active_subvol() fails. Not a pre-existing defect on those branches, a consequence of the new contract.

Every path now pairs one GET with one PUT, including the ESTALE retry loops, whose if (subvol) guards keep their
meaning. No behaviour change on success paths.

Commit 2: glfs_linkat() NULL subvol (#4797)

When the destination lookup fails with a non-ENOENT error, glfs_linkat() sets ret = -1 but does not bail out and calls
syncop_link(newsubvol, ...) with newsubvol == NULL → SIGSEGV in syncop_link(). glfs_renameat() has the
if (!newsubvol) goto out; this path lacks (devel :7320–7322); the commit adds the same, one hunk (+4). Pre-existing since
11.0, reproduced on unmodified devel.

Test

tests/basic/gfapi/gfapi-at-parent-ref.{c,t}: opens a directory handle, fails an openat and an fstatat through a
regular file (ENOTDIR), a renameat and a renameat2 of a missing source (ENOENT) and a linkat whose destination goes
through a regular file (ENOTDIR), checks the handle after each with glfs_fstatat() and requires glfs_closedir() to
return 0.

check devel a482a8578a this change
gfapi-at-parent-ref.t FAIL at the first check: parent handle dead after openat ENOTDIR: fstatat -> -1 (Bad file descriptor); with commit 1 alone the tester dies in syncop_link() at the linkat check PASS (stock build of devel + both commits)
ASan, one failing openat / fstatat / renameat / renameat2 then reuse of the parent heap-use-after-free in pub_glfs_fstatat, freed from cleanup_fopat_argspub_glfs_openat:696 / pub_glfs_fstatat:915 / pub_glfs_renameat:7359 / pub_glfs_renameat2:7480 none; LeakSanitizer lists no glfs_fd, fd_t or inode allocation
all thirteen glfs_*at() entry points, one non-ENOENT lookup failure each, the two-parent ENOENT cases, both AT_EMPTY_PATH linkat variants and the success paths — 39 handle checks, ASan+LSan first check kills the handle 39/39 alive, closedir 0/0, nothing of the above leaked
8 threads × 300 iterations of mixed failing and succeeding at-calls on one shared parent handle, ASan+LSan UAF at the first reuse 0 dead-handle events, closedir 0, nothing of the above leaked
every test under tests/basic/gfapi/ and tests/bugs/gfapi/ on a stock build with both commits 31/31 PASS (incl. gfapi-graph-switch-open-fd, libgfapi-fini-hang, upcall-*, gfapi-ssl-*)

Two caveats on the sanitizer runs, for completeness: the 8-thread stress needed the libglusterfs/libgfrpc build to also
carry a separate, unrelated timer-cancel fix (without it the stress dies first in rpc_clnt_reconnect, a known devel
issue); and the ASan runs end with an unrelated report in dict_to_xdr (rpc/xdr/src/glusterfs3.h:766) plus ~23 KB of
init-time allocations after the checks — neither involves gfapi handles.

Consumer note

Samba's vfs_glusterfs has passed the directory handle as the at()-parent since 4.17.0 (bug 15157), and since 4.19.0 smbd's
directory listing itself issues fstatat relative to the open directory handle, so one entry whose lookup
fails (ENOTDIR, EACCES, ENOTCONN during a disconnect) frees that handle; Samba ≥ 4.23.7 / 4.24.2 then aborts when its
FSP-extension destructor closes the handle at fsp teardown and the freed block has been reused. Eleven smbd cores in two
days on a production gateway (Samba 4.23.6 with that fix backported, GlusterFS 11.2) are attributed to this defect: ten at
session logoff on that path, one in glfs_openat relative to a freed parent; the failing at-call itself was not captured,
the attribution rests on glfs_fd_destroy() being the only path that frees a glfs_fd and on glfs_fini/graph switch
being ruled out from the client logs. The release-11 branch has the same code and takes the same change.

Fixes: #4796
Fixes: #4797

setup_fopat_args() takes a reference on the parent glfd and, when its
path lookup fails with anything but ENOENT, releases it again through
cleanup_fopat_args() before returning NULL. Every glfs_*at() caller then
jumps to out: and calls cleanup_fopat_args() once more, dropping a
second reference it never took. A directory handle held only by the
caller goes 1 -> 2 -> 1 -> 0 and is freed by glfs_fd_destroy() while
the caller still uses it: the next call on that handle reads freed
memory -- EBADF while the block is still intact, SIGSEGV once the
allocator has recycled it. A failure after the lookup (the open, the
rename, ...) is not affected: it reaches out: with the reference still
held and releases it once.

glfs_renameat(), glfs_renameat2() and glfs_linkat() have a second form
of the same imbalance: they run cleanup_fopat_args() on the destination
parent unconditionally at out:, so any early failure on the source side
-- including a plain ENOENT for a source that no longer exists --
releases a reference on newpglfd that was never taken.

All thirteen glfs_*at() entry points share the pattern: openat,
fstatat, faccessat, fchmodat, fchownat, linkat, mkdirat, mknodat,
readlinkat, renameat, renameat2, symlinkat, unlinkat. Present since the
API was added -- f64f400 (glfs_openat), a005a54 (glfs_fstatat),
c27bdbe (the rest), all 2022 -- i.e. in every 11.x release.

Samba's vfs_glusterfs (4.18 and later) is the consumer that hits this:
smbd lists a directory relative to the client's open directory handle
(glfs_fstatat()/glfs_openat() with that handle as the parent), so one
entry whose lookup fails with ENOTDIR, EACCES or -- during a transport
disconnect -- ENOTCONN frees the directory handle behind the client's
back. Before Samba 4.23.7 the handle was rarely touched again (EBADF on
later listings, a leak, a crash only on a later open relative to it);
since 4.23.7 / 4.24.2 the FSP-extension destructor closes every such
handle when the fsp is torn down, which aborts smbd once the freed block
has been reused. A production gateway (4.23.6 with that fix backported,
GlusterFS 11.2) saw eleven smbd cores in two days attributed to this,
ten of them at session logoff.

Make the two sides agree on who releases what: cleanup_fopat_args()
releases the graph reference and the parent reference only when it is
handed a subvol -- a NULL subvol means setup_fopat_args() failed and
already released, or was never reached -- and setup_fopat_args()'s
failure path releases exactly what it took (the graph reference through
cleanup when it has one, the parent reference alone otherwise),
preserving errno. The three AT_EMPTY_PATH branches (fstatat, fchownat,
linkat) take their reference explicitly and were balanced by the old
unconditional release; under the new contract they release it
themselves before jumping to out:. Every path now pairs one GET with
one PUT, including the ESTALE retry loops, whose "if (subvol)" guards
keep their meaning.

tests/basic/gfapi/gfapi-at-parent-ref.t opens a directory handle, fails
an openat and an fstatat through a regular file (ENOTDIR) and a renameat
and a renameat2 of a missing source (ENOENT), and requires the handle to
survive each and to close cleanly. It fails on devel at the first check
("parent handle dead after openat ENOTDIR: fstatat -> -1 (Bad file
descriptor)") and passes with this change. Verified additionally with
AddressSanitizer: heap-use-after-free in every failing mode before, none
after, and no glfs_fd, fd_t or inode allocation left behind.

Fixes: gluster#4796
Signed-off-by: Thales Antunes de Oliveira Barretto <thales.barretto.git@gmail.com>
When setup_fopat_args() fails for the destination side of glfs_linkat()
with anything but ENOENT it returns NULL, and the function carries on:
the EEXIST check is guarded by "newsubvol &&", the EISDIR check looks at
the source, and syncop_link(newsubvol, ...) is then called with a NULL
xlator -- a SIGSEGV in syncop_link() for something as ordinary as a
destination path that goes through a regular file (ENOTDIR).
glfs_renameat() has the "if (!newsubvol) goto out;" that this path
lacks; add the same.

Found while auditing every glfs_*at() error path for the parent-glfd
reference fix; the two are independent (this one is a control-flow
gap, not a reference imbalance) and each is exposed on its own.

tests/basic/gfapi/gfapi-at-parent-ref.c gains a linkat whose destination
lookup fails with ENOTDIR: with the previous commit alone the tester
dies in syncop_link(), with this one it gets -1/ENOTDIR and the parent
handle survives.

Fixes: gluster#4797
Signed-off-by: Thales Antunes de Oliveira Barretto <thales.barretto.git@gmail.com>
@ThalesBarretto
ThalesBarretto force-pushed the fix/gfapi-at-parent-glfd-ref branch from f9124aa to d98c671 Compare September 17, 2026 23:14
@ThalesBarretto
ThalesBarretto marked this pull request as ready for review September 19, 2026 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants