Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use the NODEFS filesystem in statfs when available #23912

Closed
wants to merge 5 commits into from
Closed

Use the NODEFS filesystem in statfs when available #23912

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
7d0c9a6
Use the NODEFS filesystem in statfs when available
bgrgicak Mar 13, 2025
cd33f4e
Test number of block in statfs
bgrgicak Mar 13, 2025
b1b9b01
Test NODERAWFS
bgrgicak Mar 13, 2025
b53842c
Test with real blocks count
bgrgicak Mar 19, 2025
5e8d396
Limit real disk space test to only nodefs
bgrgicak Mar 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 20 additions & 6 deletions src/lib/libfs.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -696,15 +696,30 @@ FS.staticInit();
return parent.node_ops.mknod(parent, name, mode, dev);
},
statfs(path) {
return FS.statfsNode(FS.lookupPath(path, {follow: true}).node);
// When NODEFS is available, Emscripten still uses MEMFS as the default
// file system which doesn't have a statfs function.
// To ensure statfsNode has access to the statfs function, we pass in
// the NODEFS object.
if (this.filesystems.NODEFS) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

statfs should figure out which filesystem the path is in and then return the filesystem info for that filesystem.

i.e. if path is in the MEMFS you should get the memfs info. If path is a NODEFS you should get the node FS info.

The presense of NODEFS alone should not be a factor, right? The question should be "is path part of a node FS mount" I think,

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been thinking about this for a week and I'm not sure how to handle it, so let me start with our usecase and why the above approach doesn't work for us.

WordPress Playground mounts all files into MEMFS because some files like the WordPress code are being downloaded from remote URL while a plugin might be imported from a local folder. So, the resulting codebase that we are running inside Emscription is composed out of multiple data sources.

Curently when the code asks for available disk space using statfs("/") it get's the default values because / is a MEMFS directory.

I was able to work around this by overriding statfs during onRuntimeInitialized.

PHPRuntime.FS.root.node_ops = {
	...PHPRuntime.FS.root.node_ops,
	statfs: PHPRuntime.FS.filesystems.NODEFS.node_ops.statfs,
};
PHPRuntime.FS.root.mount.opts.root = '/';

Checking what FS the path belongs to, makes sense overall, but in some cases like the above it would be great if we could override it and force a FS to be used.

Specifically for statfs this makes sense to force NODEFS, because it only works in NODEFS (and NODERAWFS).If the requested path is available in that NODEFS it would be reasonable to override MEMFS and return the result from NODEFS.node_ops.statfs.statfs.

What do you think? How would you approach our usecase?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curently when the code asks for available disk space using statfs("/") it get's the default values because / is a MEMFS directory.

Which code is doing this? Asking for the available disk space by using statfs("/") seem odd since normally the root FS has very little space in it right? Where is that code that is doing this? Why is it using /?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But won't the statfs from node give very different results if you stat / vs /tmp vs /home, for example. What is the code in question interested in exactly?

return FS.statfsNode({
path,
node: this.filesystems.NODEFS,
});
}
return FS.statfsNode({
path,
node: FS.lookupPath(path, {
follow: true
}).node
});
},
statfsStream(stream) {
// We keep a separate statfsStream function because noderawfs overrides
// it. In noderawfs, stream.node is sometimes null. Instead, we need to
// look at stream.path.
return FS.statfsNode(stream.node);
return FS.statfsNode(stream);
},
statfsNode(node) {
statfsNode({path, node}) {
// NOTE: None of the defaults here are true. We're just returning safe and
// sane values. Currently nodefs and rawfs replace these defaults,
// other file systems leave them alone.
Expand All @@ -718,11 +733,10 @@ FS.staticInit();
ffree: FS.nextInode - 1,
fsid: 42,
flags: 2,
namelen: 255,
namelen: 255
};

if (node.node_ops.statfs) {
Object.assign(rtn, node.node_ops.statfs(node.mount.opts.root));
Object.assign(rtn, node.node_ops.statfs(path));
}
return rtn;
},
Expand Down
25 changes: 23 additions & 2 deletions test/unistd/fstatfs.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,35 @@ int main() {
struct statfs buf;

int rtn;
assert(fstatfs(STDOUT_FILENO, &buf) == 0);
int fstatfs_rtn = fstatfs(STDOUT_FILENO, &buf);
printf("f_type: %ld\n", buf.f_type);
assert(fstatfs_rtn == 0);
printf("f_blocks: %d\n", buf.f_blocks);
assert(buf.f_blocks == 1000000);

int f = open("file", O_RDWR | O_CREAT);
assert(fstatfs(f, &buf) == 0);
fstatfs_rtn = fstatfs(f, &buf);
printf("f_type: %ld\n", buf.f_type);
printf("f_blocks: %d\n", buf.f_blocks);
#if NODEFS
assert(fstatfs_rtn == 0);
assert(buf.f_blocks != 0);
// 2048000000 is a hardcoded value for NODEFS blocks.
assert(buf.f_blocks != 2048000000);
#else
assert(fstatfs_rtn == 0);
assert(buf.f_blocks == 1000000);
#endif

assert(statfs("file", &buf) == 0);
printf("f_type: %ld\n", buf.f_type);
printf("f_blocks: %d\n", buf.f_blocks);
#if NODEFS
assert(buf.f_blocks != 0);
assert(buf.f_blocks != 2048000000);
#else
assert(buf.f_blocks == 1000000);
#endif

return 0;
}
Loading