Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Macos ci #1339

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
yedayak wants to merge 11 commits into scop:main
base: main
Choose a base branch
Loading
from yedayak:macos-ci
Draft

Macos ci #1339

yedayak wants to merge 11 commits into scop:main from yedayak:macos-ci

Conversation

@yedayak
Copy link
Collaborator

@yedayak yedayak commented Feb 24, 2025
edited
Loading

Copy link
Collaborator Author

yedayak commented Feb 24, 2025

The CI run failed because for some reason it didn't manage to delete the temp directory it creates in test_unit_load.py using shutil.rmtree in prepare_fixture_dir

Copy link
Owner

scop commented Apr 18, 2025

Thanks for working on this! I think I've seen the temp dir removal issue myself as well. Would be great if we could figure out a workaround. If we can't, we should allow it to fail only on macOS.

@yedayak yedayak force-pushed the macos-ci branch 3 times, most recently from e94114a to 34a0d71 Compare May 12, 2025 20:55
@yedayak yedayak force-pushed the macos-ci branch 11 times, most recently from ad44019 to 2e6e2c9 Compare July 8, 2025 17:27
@yedayak yedayak force-pushed the macos-ci branch 4 times, most recently from 17a1477 to eaa8ce1 Compare September 20, 2025 21:32
@yedayak yedayak force-pushed the macos-ci branch 2 times, most recently from f6c04d4 to 4331cfa Compare September 22, 2025 12:06
return str(tmpdir)
yield str(tmpdir)
if sys.platform == "darwin":
assert_bash_exec(bash, "sudo rm -rf %s/*" % tmpdir)
Copy link
Collaborator Author

@yedayak yedayak Oct 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't love running sudo in the test suite, but I couldn't find a another way to delete these files

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why we need to do this manual removal in the first place. In prepare_fixture_dir we add a finalizer that is supposed to clean up the temp dir altogether. Does that not work here? Does it work in other places where we use it to set up a fixture dir?

But if it is needed, some followup questions:

Can we assume sudo is present? What if it asks for credentials?

Any idea if this is needed on all macOS setups, or just the GH one?

If it's a GH only thing, and perhaps otherwise as well, I suppose we could just not fail on error here if the remove fails and let the error message come through so people can clean up manually.

Let's add a comment with details why it doesn't work without it here.

Maybe consider doing the sudo rm in the prepare_fixture_dir finalizer on darwin, if it is needed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why we need to do this manual removal in the first place. In prepare_fixture_dir we add a finalizer that is supposed to clean up the temp dir altogether. Does that not work here? Does it work in other places where we use it to set up a fixture dir?

From what I gathered, the issue is that this is the only place where we create more files and folders inside the fixture_dir. Those extra files are created by other processes and not python, so I think that's what causes the issue. See this note in the removefile() syscall:

Write protected files owned by another process cannot be removed by
removefile(), regardless of the permissions on the directory containing
the file.

I might have just not found the correct magic spell to make these files deletable without sudo

But if it is needed, some followup questions:

Can we assume sudo is present? What if it asks for credentials?

Yeah that's a problem. It seems the default behavior is asking for a password

Any idea if this is needed on all macOS setups, or just the GH one?

I don't know, the man pages seem to imply this will happen for all macOS's, but maybe that's not the real issue. I don't have a physical one to test on.

If it's a GH only thing, and perhaps otherwise as well, I suppose we could just not fail on error here if the remove fails and let the error message come through so people can clean up manually.

I'm not sure what a good way to let the error come through without failing the tests is

Let's add a comment with details why it doesn't work without it here.

Maybe consider doing the sudo rm in the prepare_fixture_dir finalizer on darwin, if it is needed?

prepare_fixture_dir doesn't currently have access to bash, so it can't run sudo. It also isn't needed for all of the uses of it. I can make it accept a bash instance if you think that's better though.

@yedayak yedayak marked this pull request as ready for review October 12, 2025 18:09
scop and others added 9 commits October 12, 2025 21:33
Also use venv instead of global install
brew/python doesn't like installing python packages globally:
```
error: externally-managed-environmen×ばつ This environment is externally managed
```
vipw seems to have -d as an option on macos now.
Partially reverts ccf7bf6 "test(dmesg,vipw): expect no options on macOS"
It seems there is some behaviour differences regarding the order
completions are returned in between linux and macos, specifically
regarding sorting of upper case and lowercase letters. Sorting them
ourselves in the tests makes it consistent.
On macOS we need sudo to delete them, maybe because they were created by
a different prcoess?
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
persist-credentials: false
- run: env PYTESTFLAGS="--verbose -p no:cacheprovider --color=yes" test/macos-script.sh
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to confirm, is --color=yes needed here? We don't seem to add it in Linux envs, but do get color nevertheless.

def test_6_glob(self, bash, functions):
output = assert_bash_exec(bash, "__tester 'a?b'", want_output=True)
assert output.strip() == "<a b><a$b><a&b><a'b>"
items = sorted(re.findall(r"<[^>]*>", output))
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not immediately clear to me why this is needed on macOS and not on others, could we elaborate in a comment?

...after having a look further in the PR, it seems various sorts are added elsewhere as well. That smells a bit as if the collation options would not be properly in effect. Or are they, but just sort differently in macOS nevertheless?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'm not sure what's up with that. I just checked and LC_COLLATE seems to be set to C. I'll need to look into that

return str(tmpdir)
yield str(tmpdir)
if sys.platform == "darwin":
assert_bash_exec(bash, "sudo rm -rf %s/*" % tmpdir)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why we need to do this manual removal in the first place. In prepare_fixture_dir we add a finalizer that is supposed to clean up the temp dir altogether. Does that not work here? Does it work in other places where we use it to set up a fixture dir?

But if it is needed, some followup questions:

Can we assume sudo is present? What if it asks for credentials?

Any idea if this is needed on all macOS setups, or just the GH one?

If it's a GH only thing, and perhaps otherwise as well, I suppose we could just not fail on error here if the remove fails and let the error message come through so people can clean up manually.

Let's add a comment with details why it doesn't work without it here.

Maybe consider doing the sudo rm in the prepare_fixture_dir finalizer on darwin, if it is needed?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No that strong opinions, but perhaps we could take a look if it would become more messier than it's good for if we'd combine this with test/docker/entrypoint.sh with appropriate conditionals to avoid duplicating a bunch of things.

@yedayak yedayak marked this pull request as draft October 18, 2025 20:21
@yedayak yedayak force-pushed the macos-ci branch 3 times, most recently from a927bc8 to 9414d6d Compare October 25, 2025 21:08
Instead of running bash commands, copy directories and create symlinks
using pure python.
This should hopefully make macos GHA not fail to delete these
directories.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@scop scop scop left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

AltStyle によって変換されたページ (->オリジナル) /