Macos ci #1339

yedayak · 2025-02-24T20:38:19Z

Cherry-picked from here:

yedayak · 2025-02-24T21:54:55Z

The CI run failed because for some reason it didn't manage to delete the temp directory it creates in test_unit_load.py using shutil.rmtree in prepare_fixture_dir

scop · 2025-04-18T06:49:02Z

Thanks for working on this! I think I've seen the temp dir removal issue myself as well. Would be great if we could figure out a workaround. If we can't, we should allow it to fail only on macOS.

yedayak · 2025-10-09T13:21:58Z

test/t/unit/test_unit_load.py

-        return str(tmpdir)
+        yield str(tmpdir)
+        if sys.platform == "darwin":
+            assert_bash_exec(bash, "sudo rm -rf %s/*" % tmpdir)


I don't love running sudo in the test suite, but I couldn't find a another way to delete these files

I wonder why we need to do this manual removal in the first place. In prepare_fixture_dir we add a finalizer that is supposed to clean up the temp dir altogether. Does that not work here? Does it work in other places where we use it to set up a fixture dir?

But if it is needed, some followup questions:

Can we assume sudo is present? What if it asks for credentials?

Any idea if this is needed on all macOS setups, or just the GH one?

If it's a GH only thing, and perhaps otherwise as well, I suppose we could just not fail on error here if the remove fails and let the error message come through so people can clean up manually.

Let's add a comment with details why it doesn't work without it here.

Maybe consider doing the sudo rm in the prepare_fixture_dir finalizer on darwin, if it is needed?

I wonder why we need to do this manual removal in the first place. In prepare_fixture_dir we add a finalizer that is supposed to clean up the temp dir altogether. Does that not work here? Does it work in other places where we use it to set up a fixture dir?

From what I gathered, the issue is that this is the only place where we create more files and folders inside the fixture_dir. Those extra files are created by other processes and not python, so I think that's what causes the issue. See this note in the removefile() syscall:

Write protected files owned by another process cannot be removed by
removefile(), regardless of the permissions on the directory containing
the file.

I might have just not found the correct magic spell to make these files deletable without sudo

But if it is needed, some followup questions:

Can we assume sudo is present? What if it asks for credentials?

Yeah that's a problem. It seems the default behavior is asking for a password

Any idea if this is needed on all macOS setups, or just the GH one?

I don't know, the man pages seem to imply this will happen for all macOS's, but maybe that's not the real issue. I don't have a physical one to test on.

If it's a GH only thing, and perhaps otherwise as well, I suppose we could just not fail on error here if the remove fails and let the error message come through so people can clean up manually.

I'm not sure what a good way to let the error come through without failing the tests is

Let's add a comment with details why it doesn't work without it here.

Maybe consider doing the sudo rm in the prepare_fixture_dir finalizer on darwin, if it is needed?

prepare_fixture_dir doesn't currently have access to bash, so it can't run sudo. It also isn't needed for all of the uses of it. I can make it accept a bash instance if you think that's better though.

Also use venv instead of global install brew/python doesn't like installing python packages globally: ``` error: externally-managed-environment × This environment is externally managed ```

vipw seems to have -d as an option on macos now. Partially reverts ccf7bf6 "test(dmesg,vipw): expect no options on macOS"

It seems there is some behaviour differences regarding the order completions are returned in between linux and macos, specifically regarding sorting of upper case and lowercase letters. Sorting them ourselves in the tests makes it consistent.

On macOS we need sudo to delete them, maybe because they were created by a different prcoess?

scop · 2025-10-18T09:30:03Z

.github/workflows/ci.yaml

+      - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
+        with:
+          persist-credentials: false
+      - run: env PYTESTFLAGS="--verbose -p no:cacheprovider --color=yes" test/macos-script.sh


Just to confirm, is --color=yes needed here? We don't seem to add it in Linux envs, but do get color nevertheless.

scop · 2025-10-18T09:31:30Z

test/t/unit/test_unit_dequote.py

    def test_6_glob(self, bash, functions):
        output = assert_bash_exec(bash, "__tester 'a?b'", want_output=True)
-        assert output.strip() == "<a b><a$b><a&b><a'b>"
+        items = sorted(re.findall(r"<[^>]*>", output))


It's not immediately clear to me why this is needed on macOS and not on others, could we elaborate in a comment?

...after having a look further in the PR, it seems various sorts are added elsewhere as well. That smells a bit as if the collation options would not be properly in effect. Or are they, but just sort differently in macOS nevertheless?

Yeah I'm not sure what's up with that. I just checked and LC_COLLATE seems to be set to C. I'll need to look into that

scop · 2025-10-18T09:32:29Z

test/t/unit/test_unit_load.py

-        return str(tmpdir)
+        yield str(tmpdir)
+        if sys.platform == "darwin":
+            assert_bash_exec(bash, "sudo rm -rf %s/*" % tmpdir)


I wonder why we need to do this manual removal in the first place. In prepare_fixture_dir we add a finalizer that is supposed to clean up the temp dir altogether. Does that not work here? Does it work in other places where we use it to set up a fixture dir?

But if it is needed, some followup questions:

Can we assume sudo is present? What if it asks for credentials?

Any idea if this is needed on all macOS setups, or just the GH one?

If it's a GH only thing, and perhaps otherwise as well, I suppose we could just not fail on error here if the remove fails and let the error message come through so people can clean up manually.

Let's add a comment with details why it doesn't work without it here.

Maybe consider doing the sudo rm in the prepare_fixture_dir finalizer on darwin, if it is needed?

scop · 2025-10-18T09:47:12Z

test/macos-script.sh

No that strong opinions, but perhaps we could take a look if it would become more messier than it's good for if we'd combine this with test/docker/entrypoint.sh with appropriate conditionals to avoid duplicating a bunch of things.

This reverts commit 8074741.

Instead of running bash commands, copy directories and create symlinks using pure python. This should hopefully make macos GHA not fail to delete these directories.

yedayak force-pushed the macos-ci branch 3 times, most recently from e94114a to 34a0d71 Compare May 12, 2025 20:55

yedayak force-pushed the macos-ci branch 11 times, most recently from ad44019 to 2e6e2c9 Compare July 8, 2025 17:27

yedayak force-pushed the macos-ci branch 4 times, most recently from 17a1477 to eaa8ce1 Compare September 20, 2025 21:32

yedayak mentioned this pull request Sep 20, 2025

[Bug Report] _comp_expand_glob may cause Bash warning about cannot change locale on macOS #1434

Closed

4 tasks

yedayak force-pushed the macos-ci branch from eaa8ce1 to cd1cc08 Compare September 20, 2025 21:37

yedayak mentioned this pull request Sep 21, 2025

fedoradev CI fail in test_unit_filedir #1435

Closed

yedayak force-pushed the macos-ci branch 2 times, most recently from f6c04d4 to 4331cfa Compare September 22, 2025 12:06

This was referenced Sep 28, 2025

Move deletion of extra makefile to fixture, don't do it on macos #1440

Merged

test(glob): Normalize some strings to compare #1441

Merged

fix(patch, truncate): Get help correctly on macos #1442

Merged

yedayak force-pushed the macos-ci branch from 4331cfa to ede4793 Compare September 28, 2025 17:08

yedayak force-pushed the macos-ci branch from ede4793 to 687c188 Compare October 7, 2025 18:30

yedayak force-pushed the macos-ci branch 3 times, most recently from 511147b to cf77da1 Compare October 8, 2025 16:05

yedayak mentioned this pull request Oct 8, 2025

fix(rsync): Try getting rsync arguments BSD style #1456

Merged

yedayak force-pushed the macos-ci branch 4 times, most recently from 804b3c9 to f70330d Compare October 9, 2025 10:58

yedayak commented Oct 9, 2025

View reviewed changes

yedayak force-pushed the macos-ci branch from f70330d to 2b01104 Compare October 12, 2025 18:01

yedayak marked this pull request as ready for review October 12, 2025 18:09

scop and others added 9 commits October 12, 2025 21:33

test: add macOS test script

85aa8ed

ci(macos): add macOS CI test config

f399911

ci(macos): Align macos script with current linux

a3545ba

Also use venv instead of global install brew/python doesn't like installing python packages globally: ``` error: externally-managed-environment × This environment is externally managed ```

ci(macos): Force colored output

00b2cae

test(vipw): Drop exception for macos

00f7d90

vipw seems to have -d as an option on macos now. Partially reverts ccf7bf6 "test(dmesg,vipw): expect no options on macOS"

test: Sort completion results items

8074741

It seems there is some behaviour differences regarding the order completions are returned in between linux and macos, specifically regarding sorting of upper case and lowercase letters. Sorting them ourselves in the tests makes it consistent.

test: Remove empty strings from completion results

aaebd24

test(tar): skip test_25 for non-gnu tar

57a88fd

test(unit_load): Delete files we create in fixture

14c8dce

On macOS we need sudo to delete them, maybe because they were created by a different prcoess?

yedayak force-pushed the macos-ci branch from 2b01104 to 14c8dce Compare October 12, 2025 18:33

scop reviewed Oct 18, 2025

View reviewed changes

Revert "test: Sort completion results items"

5d79137

This reverts commit 8074741.

yedayak force-pushed the macos-ci branch from 8b727c3 to 5d79137 Compare October 18, 2025 19:26

yedayak marked this pull request as draft October 18, 2025 20:21

yedayak force-pushed the macos-ci branch 3 times, most recently from a927bc8 to 9414d6d Compare October 25, 2025 21:08

test(_comp_load): Copy files with python

e70aa46

Instead of running bash commands, copy directories and create symlinks using pure python. This should hopefully make macos GHA not fail to delete these directories.

yedayak force-pushed the macos-ci branch from 9414d6d to e70aa46 Compare October 25, 2025 21:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Macos ci #1339

Macos ci #1339

Uh oh!

yedayak commented Feb 24, 2025 •

edited

Loading

Uh oh!

yedayak commented Feb 24, 2025

Uh oh!

scop commented Apr 18, 2025

Uh oh!

yedayak Oct 9, 2025

Uh oh!

scop Oct 18, 2025

Uh oh!

yedayak Oct 18, 2025

Uh oh!

scop Oct 18, 2025

Uh oh!

scop Oct 18, 2025

Uh oh!

yedayak Oct 18, 2025

Uh oh!

scop Oct 18, 2025

Uh oh!

scop Oct 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Macos ci #1339

Are you sure you want to change the base?

Macos ci #1339

Uh oh!

Conversation

yedayak commented Feb 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yedayak commented Feb 24, 2025

Uh oh!

scop commented Apr 18, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

yedayak commented Feb 24, 2025 •

edited

Loading