Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Comments

Build platform-specific wheels containing libmagic#294

Open
ddelange wants to merge 59 commits intoahupp:master from
ddelange:abi3-wheels
Open

Build platform-specific wheels containing libmagic #294
ddelange wants to merge 59 commits intoahupp:master from
ddelange:abi3-wheels

Conversation

@ddelange
Copy link
Contributor

@ddelange ddelange commented Sep 6, 2023 •
edited
Loading

Hi @ahupp 👋

This PR builds self-contained wheels as discussed in #233. For Windows users, this renders python-magic-bin from @julian-r obsolete.

pip install these wheels

  1. @fpgmaas published the unofficial python-magic-standalone PyPI package based on this branch. 
    pip install python-magic-standalone
  2. Alternatively, pip can install these wheels from the GitHub Release assets from my fork: 
    # space separated list of URLs for --find-links
export PIP_FIND_LINKS='https://github.com/ddelange/python-magic/releases/expanded_assets/0.4.28.post9'
pip install --force-reinstall python-magic

The wheels diff:

- python-magic-0.4.27.tar.gz
+ python-magic-0.4.28.tar.gz
- python_magic-0.4.27-py2.py3-none-any.whl
+ python_magic-0.4.28-py2.py3-none-macosx_10_9_x86_64.whl
+ python_magic-0.4.28-py2.py3-none-macosx_11_0_arm64.whl
+ python_magic-0.4.28-py2.py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ python_magic-0.4.28-py2.py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
+ python_magic-0.4.28-py2.py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl
+ python_magic-0.4.28-py2.py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
+ python_magic-0.4.28-py2.py3-none-musllinux_1_1_aarch64.whl
+ python_magic-0.4.28-py2.py3-none-musllinux_1_1_ppc64le.whl
+ python_magic-0.4.28-py2.py3-none-musllinux_1_1_s390x.whl
+ python_magic-0.4.28-py2.py3-none-musllinux_1_1_x86_64.whl
+ python_magic-0.4.28-py2.py3-none-win32.whl
+ python_magic-0.4.28-py2.py3-none-win_amd64.whl

The wheels:

  • Come bundled with libmagic (.dylib on mac, .so on nix, .ddl on win) - no additional user action needed
    • latest libmagic (5.46)
    • skipping i686 linux wheels as they don't pass pytest ref ca4def3
  • Are forward compatible, they will install on cpython 2.7 and all current and future cpython 3.5+ distributions
  • Are built for the major cpu architectures (and also for alpine linux)
    • (削除) Linux architectures limited by availability: https://pkgs.org/search/?q=file-libs (削除ここまで) now building from source on linux
    • Windows ARM64 is not included currently, but MSYS2 built libmagic is available
  • Are uploaded as GitHub Release assets, as well as upload to PyPI, whenever a GitHub Release is (pre)released.
    • 🚩 @ahupp for this, you still need to add wheels.yml as trusted publisher here

CI/CD

dists build with official cibuildwheel on GitHub Actions, and they build in parallel:
image

  • The PyPI publish step is failing here ^ because my fork is not a trusted publisher
  • For any system not covered by these wheels, pip will fall back to the source distribution, which will check for proper libmagic on the systen at install time.

fix #137, fix #288, fix #225, fix #276, fix #248, fix #87, fix #139, fix #233, fix #73, fix #60, fix #34, fix #293, fix #233, fix #278, fix #262, fix #248, fix #238, fix #145, fix #61, fix #12, fix #295, fix #311, fix #312, fix #313, fix #321, fix #332, fix #249, fix #333, fix #350, fix #351, fix #287, fix #335

TzuriTeshuba, wordhui, and cm253 reacted with thumbs up emoji jspraul, declension, youngcho22, martina-oefelein, dAnjou, jean-humann, mariosasko, patacca, martin-liu, kdebrab, and 13 more reacted with hooray emoji Privat33r-dev, wordhui, nmichlo, and cm253 reacted with rocket emoji
@ddelange ddelange changed the title (削除) Build ABI3 wheels containing libmagic (削除ここまで) (追記) Build platform-specific wheels containing libmagic (追記ここまで) Sep 7, 2023
@ddelange ddelange force-pushed the abi3-wheels branch 2 times, most recently from a98f13b to dc9c393 Compare September 7, 2023 07:26
@ddelange ddelange force-pushed the abi3-wheels branch 12 times, most recently from d672b91 to 14f7dbb Compare September 7, 2023 10:03
Copy link

This is nice! Hope this will be merged soon!

Just ran into issues with my library being not usable by Mac and Windows users because I rely on python-magic. If there are wheels, I don't need to find a workaround or replace the library :)

python-magic-bin did not work for some of them, by the way.

with:
files: dist/*

- name: Upload to PyPI
Copy link

@stumpylog stumpylog Sep 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A small improvement here might be to use the PyPa Action instead: https://github.com/pypa/gh-action-pypi-publish

The big advantage is trusted publishing, instead of storing a password or token as a secret

apirogov reacted with thumbs up emoji
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's cool, thanks for sharing!

@ahupp shall I make that change and you set it up on PyPI side?

Copy link
Contributor Author

@ddelange ddelange Oct 26, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like trusted publishing is the way to go. I recently got this email:

Hi ddelange!
Earlier this year, we announced that PyPI would require all users to enable a form of two-factor authentication on their accounts by the end of 2023.

Keeping your PyPI account secure is important to all of us. We encourage you to enable two-factor authentication on your PyPI account as soon as possible.

What forms of 2FA can I use?
We currently offer two main forms of 2FA for your account:

Security device including modern browsers (preferred) (e.g. Yubikey, Google Titan)
Authentication app (e.g. Google Authenticator)
Once one of these secure forms is enabled on your account, you will also need to use either Trusted Publishers (preferred) or API tokens to upload to PyPI.

What do I do if I lose my 2FA device?
As part of 2FA enrollment, you will receive one-time use recovery codes. One of them must be used to confirm receipt before 2FA is fully active. Keep these recovery codes safe - they are equivalent to your 2FA device. Should you lose access > to your 2FA device, use a recovery code to log in and swap your 2FA to a new device.

Read more aboutrecovery codes.

Why is PyPI requiring 2FA?
Keeping all users of PyPI is a shared responsibility we take seriously. Strong passwords combined with 2FA is a recognized secure practice for over a decade.

We are requiring 2FA to protect your account and the packages you upload, and to protect PyPI itself from malicious actors. The most damaging attacks are account takeover and malicious package upload.

To see this and other security events for your account, visit your account security history.

Read more on this blog post.

If you run into problems, read the FAQ page. If the solutions there are unable to resolve the issue, contact us via support@pypi.org.

Thanks,
The PyPI Admins

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ahupp so the last thing for you to do is adding this repo as trusted publisher to https://pypi.org/manage/project/python-magic/settings/publishing/

Copy link
Owner

ahupp commented Sep 28, 2023

This is huge, thank you! Apology for the delay I thought I'd commented earlier but guess not. I'll look this over soon; I didn't quite understand how bad the binary dep situation was expecially on windows.
ashtonpaul, DeflateAwning, and Redoubts reacted with eyes emoji

Copy link

@ahupp @stumpylog could we have this one merged (and released) by the end of the year please ?
patacca, ashtonpaul, and nguaman reacted with thumbs up emoji

Copy link

Can this please be merged? Is this library actively maintained? Seems like a serious improvement.
ddelange, cclauss, nightlark, summersz, 0dminnimda, and cm253 reacted with heart emoji

Copy link
Contributor

@cclauss cclauss left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to get this landed.

@ddelange ddelange mentioned this pull request Aug 18, 2025
Copy link

fpgmaas commented Sep 4, 2025

Any updates here @ahupp ? I think many users are eagerly waiting for this to be available on PyPI. Even having a prerelease available on PyPI would be great!
piranna and ashtonpaul reacted with thumbs up emoji

Copy link

fpgmaas commented Sep 24, 2025 •
edited
Loading

Since this PR has been open for quite a while and there are no indications of it being merged anytime soon, I took the liberty of uploading the wheels to PyPI under the name python-magic-standalone, see https://pypi.org/project/python-magic-standalone/. It's sole purpose is to make it possible for users to install @ddelange's work directly from PyPI. I don't intend to maintain this fork, and I added a clear disclaimer to the README. It can be installed with any of the following commands, depending on the tool you use;

# pip
pip install python-magic-standalone
# uv
uv add python-magic-standalone
# poetry
poetry add python-magic-standalone

Copy link
Contributor

cclauss commented Sep 24, 2025

@fpgmaas, please consider building wheels for Python 3.14, as it is less than two weeks away.

Copy link
Contributor Author

ddelange commented Sep 24, 2025 •
edited
Loading

this branch / these wheels are forward compatible, no need for changes in the build 👍
fpgmaas and cclauss reacted with thumbs up emoji

Copy link
Contributor Author

Updated the PR description. Thanks @fpgmaas 🎉
fpgmaas reacted with thumbs up emoji

Copy link

@CodeXex123 CodeXex123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

cm253 reacted with rocket emoji
Copy link
Contributor Author

@ddelange ddelange Oct 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

49982d2 is a git cherry-pick from https://github.com/thc1102/python-magic to fix #287 / #335

Copy link

cm253 commented Nov 26, 2025

Is this PR still waiting for something? :)
zeevro, javad94, and Nriver reacted with thumbs up emoji cclauss reacted with laugh emoji mikicz reacted with confused emoji

Copy link

fpgmaas commented Dec 7, 2025

Hi @ahupp, just checking in to see if there are any updates on this PR. It seems quite some users are waiting for this to be merged. Are you still trying to determine if this should be merged or not, or are you simply lacking the time to properly review? I am aware that maintaining a project can be a bit of a timesink.

If time is the constraint, is there anything the community can do to help out? Is it maybe worth looking for maintainers for this project to help you manage it?
Nriver, cm253, jarrod-cummings, and risicle reacted with thumbs up emoji

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

AltStyle によって変換されたページ (->オリジナル) /