Skip to content

std.c: define freopen() and the stdio streams #24446

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from

Conversation

blblack
Copy link
Contributor

@blblack blblack commented Jul 14, 2025

This allows for daemonization and output-redirection sorts of code in libc-linked Zig projects on *nix-y platforms to affect the stdio stream usage of other libc-based library code the project may be linking, e.g.:

// Set stderr to go nowhere without errors:
_ = std.c.freopen("/dev/null", "r+", std.c.stderr());

// Append stdout to a file:
_ = std.c.freopen("/tmp/output.txt", "a", std.c.stdout());

The stdio streams are returned from function calls because they're commonly #defines in the libc headers pointing at variously-named externs, and one doesn't generally assign to them directly anyways (freopen() is the portable way to assign something new to a stdio stream).

NetBSD, OpenBSD, and Solaris stdio streams are not supported in this patch and will cause a compileError if you try to use the them, for now. Supporting them would require a more complex and fragile solution using a definition of "FILE" that is not just an opaque type (because their C libraries publish an extern array of FILE objects and then #define the stdio stream names as pointers to the array elements).

@alexrp alexrp self-assigned this Jul 14, 2025
@SeanTheGleaming
Copy link
Contributor

SeanTheGleaming commented Jul 14, 2025

For windows support, this part of wine's msvcrt <corecrt_stdio.h> (which is included in its <stdio.h>) should be helpful

@blblack
Copy link
Contributor Author

blblack commented Jul 14, 2025

I have a patch in the works for NetBSD, OpenBSD, Solaris, and Illumos. It's a bit ugly, but I think it gets the job done. Completely untested and just based on reading headers, as I don't have VMs for these at the ready: https://zigbin.io/576105 .

@alexrp
Copy link
Member

alexrp commented Jul 15, 2025

I have a patch in the works for NetBSD, OpenBSD, Solaris, and Illumos. It's a bit ugly, but I think it gets the job done. Completely untested and just based on reading headers, as I don't have VMs for these at the ready: https://zigbin.io/576105 .

Seems like it'd be more robust to just declare the fields and let the compiler figure out the layout?

@blblack
Copy link
Contributor Author

blblack commented Jul 15, 2025

I have a patch in the works for NetBSD, OpenBSD, Solaris, and Illumos. It's a bit ugly, but I think it gets the job done. Completely untested and just based on reading headers, as I don't have VMs for these at the ready: https://zigbin.io/576105 .

Seems like it'd be more robust to just declare the fields and let the compiler figure out the layout?

Agreed. I was just cribbing that style from e.g. pthread_mutex_t defs and keeping things succinct. Would you rather I push this all in the same PR, or save the complicated ones for a separate PR and leave this one with just the basic easy ones?

@alexrp
Copy link
Member

alexrp commented Jul 15, 2025

It's fine to do it all in this PR.

@blblack blblack marked this pull request as draft July 15, 2025 11:50
@blblack
Copy link
Contributor Author

blblack commented Jul 15, 2025

Updated with two new commits: the NetBSD/OpenBSD/Solaris/Illumos one (with expanded struct defs), and a draft Windows one (I'm really not a Windows person, so I have no idea if something else is needed there in std.os.windows to make it link correctly or something). Will see if I can find a way to test it via emulation...

blblack added 4 commits July 15, 2025 09:18
This allows for daemonization and output-redirection sorts of code
in libc-linked Zig projects on *nix-y platforms to affect the
stdio stream usage of other libc-based library code the project
may be linking, e.g.:

    // Set stderr to go nowhere without errors:
    _ = std.c.freopen("/dev/null", "r+", std.c.stderr());

    // Append stdout to a file:
    _ = std.c.freopen("/tmp/output.txt", "a", std.c.stdout());

The stdio streams are returned from function calls because they're
commonly #defines in the libc headers pointing at variously-named
externs, and one doesn't generally assign to them directly anyways
(freopen() is the portable way to assign something new to a stdio
stream).

NetBSD, OpenBSD, and Solaris stdio streams are not supported in
this patch and will cause a compileError if you try to use the
them, for now. Supporting them would require a more complex and
fragile solution using a definition of "FILE" that is not just an
opaque type (because their C libraries publish an extern array of
FILE objects and then #define the stdio stream names as pointers
to the array elements).
@blblack
Copy link
Contributor Author

blblack commented Jul 15, 2025

Updated for WASI as well.

I added a hacky temporary test at the bottom of lib/std/c.zig that spams to stderr just for local validation, like this:

test "unique-test-name" {
    if (builtin.link_libc) {
        const test_output = "\n\n----------------- I am The Walrus! ---------------\n\n";
        try std.testing.expect(fwrite(test_output.ptr, @sizeOf(u8), test_output.len, stderr()) == test_output.len);
    }
}

And then with a test-std run with -fwasmtime -fqemu -fwine, I get the expected outputs on all the Linux arches:

+- run test std-powerpc-linux-musleabi-ppc-Debug-libc stderr
----------------- I am The Walrus! ---------------
test-std
+- run test std-thumbeb-linux-musleabihf-baseline-Debug-libc stderr
----------------- I am The Walrus! ---------------
test-std
+- run test std-native-znver4-Debug-libc-cbe stderr
----------------- I am The Walrus! ---------------
test-std
+- run test std-riscv32-linux-musl-baseline_rv32-Debug-libc stderr
----------------- I am The Walrus! ---------------
[... skipped pasting the numerous others ...]

As well as WASI and Windows:

test-std
+- run test std-wasm32-wasi-musl-lime1-Debug-libc stderr
----------------- I am The Walrus! ---------------
test-std
+- run test std-x86-windows-gnu-pentium4-Debug-libc stderr


----------------- I am The Walrus! ---------------

test-std
+- run test std-x86_64-windows-gnu-x86_64-Debug-libc stderr


----------------- I am The Walrus! ---------------

@blblack blblack marked this pull request as ready for review July 15, 2025 15:37
@blblack
Copy link
Contributor Author

blblack commented Jul 15, 2025

Managed to manually test this with a cross-compiled binary on FreeBSD in qemu, to validate that this approach works at all for at least one of these __stderrp cases.

This is my test source:

const std = @import("std");

pub fn main() void {
    std.debug.print("Will test libc-based stderr output below:\n", .{});
    const test_msg = "---- I am The Walrus ----\n";
    const written = std.c.fwrite(test_msg.ptr, @sizeOf(u8), test_msg.len, std.c.stderr());
    std.debug.print("Wrote {d}/{d} bytes to stderr without crashing!\n", .{ written, test_msg.len });
}

Cross-compiled from Linux -> FreeBSD via: zig build-exe -target x86_64-freebsd.14.3 -mcpu=sandybridge -fllvm -flld test-stderr.zig, and then copied into the qemu host, it runs successfully:

root@freebsd:~ # ./test-stderr
Will test libc-based stderr output below:
---- I am The Walrus ----
Wrote 26/26 bytes to stderr without crashing!

@blblack
Copy link
Contributor Author

blblack commented Jul 16, 2025

Tested NetBSD 10.1 manually now as well (same test-stderr.zig as FreeBSD, cross-compiled from Linux -> NetBSD, running in a qemu NetBSD live image):

netbsd# ./test-stderr
Will test libc-based stderr output below:
---- I am The Walrus ----
Wrote 26/26 bytes to stderr without crashing!

This gets us at least one working example of the extern-array-of-FILE pattern (and that at least the NetBSD struct sizing comes out correctly on x86_64, or else we'd be off quite a bit by the time we get to stderr as the 3rd array element and something would break).

@alexrp
Copy link
Member

alexrp commented Jul 20, 2025

@andrewrk
Copy link
Member

andrewrk commented Jul 21, 2025

I don't think we need these functions in the zig standard library. In fact, I think all functions dealing with FILE should be deleted.

@blblack
Copy link
Contributor Author

blblack commented Jul 21, 2025

I don't think we need these functions in the zig standard library. In fact, I think all functions dealing with FILE should be deleted.

That's a reasonable stance too, I think! Basically everyone porting C code that was using freopen() for these purposes will need to switch to dup2() on the underlying FDs? That's what std.process.Child does. I'm pretty sure this works for most libc implementations in practice, I've just never been sure it's universally ok across all the *nix as the most-portable way to redirect/ignore library use of the stdio streams.

@blblack
Copy link
Contributor Author

blblack commented Jul 21, 2025

Closing this for now, given the above! Thanks!

@blblack blblack closed this Jul 21, 2025
@alexrp alexrp removed their assignment Jul 21, 2025
@nektro
Copy link
Contributor

nektro commented Jul 21, 2025

@blblack i started https://github.com/nektro/zig-libc if you'd like to move there

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants