-
-
Notifications
You must be signed in to change notification settings - Fork 328
NotADirectoryError when installing from source distribution #1180
Unanswered
thegamecracks
asked this question in
Problem
-
When attempting to install the package specifically using its source distribution rather than the built wheels, the following error occurs:
×ばつ Preparing metadata (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [26 lines of output] Traceback (most recent call last): File "..\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module> main() File "..\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 152, in prepare_metadata_for_build_wheel whl_basename = backend.build_wheel(metadata_directory, config_settings) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Temp\pip-build-env-qtfs6sj6\overlay\Lib\site-packages\hatchling\build.py", line 58, in build_wheel return os.path.basename(next(builder.build(directory=wheel_directory, versions=['standard']))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Temp\pip-build-env-qtfs6sj6\overlay\Lib\site-packages\hatchling\builders\plugin\interface.py", line 147, in build build_hook.initialize(version, build_data) File "..\AppData\Local\Temp\pip-build-env-qtfs6sj6\overlay\Lib\site-packages\hatch_build_scripts\plugin.py", line 51, in initialize run(cmd, cwd=str(work_dir), check=True, shell=True) # noqa: S602 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 548, in run with Popen(*popenargs, **kwargs) as process: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 1026, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "..\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 1538, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ NotADirectoryError: [WinError 267] The directory name is invalid [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed ×ばつ Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details.">
>pip install reactpy --no-binary reactpy Collecting reactpy Using cached reactpy-1.0.2.tar.gz (123 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... error error: subprocess-exited-with-error ×ばつ Preparing metadata (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [26 lines of output] Traceback (most recent call last): File "..\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module> main() File "..\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 152, in prepare_metadata_for_build_wheel whl_basename = backend.build_wheel(metadata_directory, config_settings) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Temp\pip-build-env-qtfs6sj6\overlay\Lib\site-packages\hatchling\build.py", line 58, in build_wheel return os.path.basename(next(builder.build(directory=wheel_directory, versions=['standard']))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Temp\pip-build-env-qtfs6sj6\overlay\Lib\site-packages\hatchling\builders\plugin\interface.py", line 147, in build build_hook.initialize(version, build_data) File "..\AppData\Local\Temp\pip-build-env-qtfs6sj6\overlay\Lib\site-packages\hatch_build_scripts\plugin.py", line 51, in initialize run(cmd, cwd=str(work_dir), check=True, shell=True) # noqa: S602 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 548, in run with Popen(*popenargs, **kwargs) as process: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 1026, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "..\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 1538, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ NotADirectoryError: [WinError 267] The directory name is invalid [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed×ばつ Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details.
After some debugging, it appears that hatch is trying to build the JavaScript client and assumes src/js/ is present, but in the source distribution that directory is missing. Here is the build configuration in question:
reactpy/src/py/reactpy/pyproject.toml
Lines 127 to 136 in 3a3ad3f
[[tool.hatch.build.hooks.build-scripts.scripts]]
work_dir = "../../js"
out_dir = "reactpy/_static"
commands = [
"npm ci",
"npm run build"
]
artifacts = [
"app/dist/"
]
Right now, both the sdist and wheels uploaded on PyPI contain the reactpy/_static/ files generated by npm run build (inspector), but hatch is configured to build the client even when the source distribution already has the output files. This isn't a major issue since reactpy's wheels aren't dependent on system architecture, but it would be nice to have this part of the build process fixed (or at least documented).
The ideal solution would be to include the JS client's source code in the sdist and only build it when installing from source / generating the wheel. I thought of suggesting this:
+ [tool.hatch.build.targets.sdist.force-include] + "../../js" = "js" + + [tool.hatch.build.targets.wheel.force-include] + "reactpy/_static" = "reactpy/_static" # bypass .gitignore - [[tool.hatch.build.hooks.build-scripts.scripts]] - work_dir = "../../js" + [[tool.hatch.build.targets.wheel.hooks.build-scripts.scripts]] + work_dir = "js"
However it breaks installing from source (pip install src/py/reactpy) as pip skips building the sdist and therefore won't copy js/ to the correct place. I also don't know if there's an alternative to force-include which can map paths outside of the project root while respecting .gitignore. Absent that, it would be easy to accidentally pollute the sdist with node_modules.
In theory a symbolic link to src/js/ could remedy this along with slightly different changes to pyproject.toml, specifically the use of only-include:
+ [tool.hatch.build.targets.sdist] + only-include = ["js", "reactpy"] + + [tool.hatch.build.targets.wheel.force-include] + "reactpy/_static" = "reactpy/_static" # bypass .gitignore - [[tool.hatch.build.hooks.build-scripts.scripts]] - work_dir = "../../js" + [[tool.hatch.build.targets.wheel.hooks.build-scripts.scripts]] + work_dir = "js"
But while it works when generating the sdist / building from sdist (python -m build), the npm run build command fails when installing from source or building the wheel from source (hatch build/python -m build --wheel):
Details
×ばつ Preparing metadata (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [74 lines of output] added 249 packages, and audited 258 packages in 5s 89 packages are looking for funding run `npm fund` for details 3 moderate severity vulnerabilities To address all issues, run: npm audit fix Run `npm audit` for details. > build > npm --workspaces run build > event-to-object@0.1.2 build > tsc -b > @reactpy/client@0.3.1 build > tsc -b src/reactpy-vdom.tsx(3,28): error TS2307: Cannot find module 'event-to-object' or its corresponding type declarations. npm ERR! Lifecycle script `build` failed with error: npm ERR! Error: command failed npm ERR! in workspace: @reactpy/client@0.3.1 npm ERR! at location: ..\reactpy\src\py\reactpy\js\packages\@reactpy\client > build > vite build vite v3.2.7 building for production... transforming... âœ" 9 modules transformed. [vite]: Rollup failed to resolve import "event-to-object" from "../../../../js/app/node_modules/@reactpy/client/dist/reactpy-vdom.js". This is most likely unintended because it can break your application at runtime. If you do want to externalize this module explicitly add it to `build.rollupOptions.external` error during build: Error: [vite]: Rollup failed to resolve import "event-to-object" from "../../../../js/app/node_modules/@reactpy/client/dist/reactpy-vdom.js". This is most likely unintended because it can break your application at runtime. If you do want to externalize this module explicitly add it to `build.rollupOptions.external` at onRollupWarning (file:///../reactpy/src/js/node_modules/vite/dist/node/chunks/dep-2faf2534.js:45909:19) at onwarn (file:///../reactpy/src/js/node_modules/vite/dist/node/chunks/dep-2faf2534.js:45680:13) at Object.onwarn (file:///../reactpy/src/js/node_modules/rollup/dist/es/shared/rollup.js:23263:13) at ModuleLoader.handleResolveId (file:///../reactpy/src/js/node_modules/rollup/dist/es/shared/rollup.js:22158:26) at file:///../reactpy/src/js/node_modules/rollup/dist/es/shared/rollup.js:22119:26 npm ERR! Lifecycle script `build` failed with error: npm ERR! Error: command failed npm ERR! in workspace: undefined npm ERR! at location: ..\reactpy\src\py\reactpy\js\app Traceback (most recent call last): File "..\reactpy\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module> main() File "..\reactpy\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\reactpy\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 152, in prepare_metadata_for_build_wheel whl_basename = backend.build_wheel(metadata_directory, config_settings) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Temp\pip-build-env-4ebt3ux8\overlay\Lib\site-packages\hatchling\build.py", line 58, in build_wheel return os.path.basename(next(builder.build(directory=wheel_directory, versions=['standard']))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Temp\pip-build-env-4ebt3ux8\overlay\Lib\site-packages\hatchling\builders\plugin\interface.py", line 147, in build build_hook.initialize(version, build_data) File "..\AppData\Local\Temp\pip-build-env-4ebt3ux8\overlay\Lib\site-packages\hatch_build_scripts\plugin.py", line 51, in initialize run(cmd, cwd=str(work_dir), check=True, shell=True) # noqa: S602 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 571, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command 'npm run build' returned non-zero exit status 1. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed ×ばつ Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details.">
(.venv) ..\reactpy\src\py\reactpy>pip install . Processing ..\reactpy\src\py\reactpy Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... error error: subprocess-exited-with-error ×ばつ Preparing metadata (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [74 lines of output] added 249 packages, and audited 258 packages in 5s 89 packages are looking for funding run `npm fund` for details 3 moderate severity vulnerabilities To address all issues, run: npm audit fix Run `npm audit` for details. > build > npm --workspaces run build > event-to-object@0.1.2 build > tsc -b > @reactpy/client@0.3.1 build > tsc -b src/reactpy-vdom.tsx(3,28): error TS2307: Cannot find module 'event-to-object' or its corresponding type declarations. npm ERR! Lifecycle script `build` failed with error: npm ERR! Error: command failed npm ERR! in workspace: @reactpy/client@0.3.1 npm ERR! at location: ..\reactpy\src\py\reactpy\js\packages\@reactpy\client > build > vite build vite v3.2.7 building for production... transforming... âœ" 9 modules transformed. [vite]: Rollup failed to resolve import "event-to-object" from "../../../../js/app/node_modules/@reactpy/client/dist/reactpy-vdom.js". This is most likely unintended because it can break your application at runtime. If you do want to externalize this module explicitly add it to `build.rollupOptions.external` error during build: Error: [vite]: Rollup failed to resolve import "event-to-object" from "../../../../js/app/node_modules/@reactpy/client/dist/reactpy-vdom.js". This is most likely unintended because it can break your application at runtime. If you do want to externalize this module explicitly add it to `build.rollupOptions.external` at onRollupWarning (file:///../reactpy/src/js/node_modules/vite/dist/node/chunks/dep-2faf2534.js:45909:19) at onwarn (file:///../reactpy/src/js/node_modules/vite/dist/node/chunks/dep-2faf2534.js:45680:13) at Object.onwarn (file:///../reactpy/src/js/node_modules/rollup/dist/es/shared/rollup.js:23263:13) at ModuleLoader.handleResolveId (file:///../reactpy/src/js/node_modules/rollup/dist/es/shared/rollup.js:22158:26) at file:///../reactpy/src/js/node_modules/rollup/dist/es/shared/rollup.js:22119:26 npm ERR! Lifecycle script `build` failed with error: npm ERR! Error: command failed npm ERR! in workspace: undefined npm ERR! at location: ..\reactpy\src\py\reactpy\js\app Traceback (most recent call last): File "..\reactpy\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module> main() File "..\reactpy\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\reactpy\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 152, in prepare_metadata_for_build_wheel whl_basename = backend.build_wheel(metadata_directory, config_settings) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Temp\pip-build-env-4ebt3ux8\overlay\Lib\site-packages\hatchling\build.py", line 58, in build_wheel return os.path.basename(next(builder.build(directory=wheel_directory, versions=['standard']))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Temp\pip-build-env-4ebt3ux8\overlay\Lib\site-packages\hatchling\builders\plugin\interface.py", line 147, in build build_hook.initialize(version, build_data) File "..\AppData\Local\Temp\pip-build-env-4ebt3ux8\overlay\Lib\site-packages\hatch_build_scripts\plugin.py", line 51, in initialize run(cmd, cwd=str(work_dir), check=True, shell=True) # noqa: S602 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "..\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 571, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command 'npm run build' returned non-zero exit status 1. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed×ばつ Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details.
FWIW, in an unrelated package, sardine-web, I structured it so the JS client's source code was inside the package itself, but setup.py would build the client and remove the source code afterwards, leaving only the source code in the sdist and its compiled result in the wheel. In hindsight, it probably would have been easier had I left the client source code outside of the package and only included it in the sdist via MANIFEST.in. Given hatch's own mechanisms for controlling distributions, plus the convenience of hatch-build-scripts, it might be easier to move src/js/ beside the src/py/reactpy/reactpy/ package and follow the previous suggestion to pyproject.toml, rather than try to find a workaround with the current layout.
Beta Was this translation helpful? Give feedback.
All reactions
Replies: 2 comments 2 replies
-
Given hatch's own mechanisms for controlling distributions, plus the convenience of
hatch-build-scripts, it might be easier to movesrc/js/beside thesrc/py/reactpy/reactpy/package and follow the previous suggestion to pyproject.toml, rather than try to find a workaround with the current layout.
A word of caution, as of now this suggested layout would require js/node_modules/ to be deleted before creating a source distribution, as its presence can cause js/packages/ to be incorrectly omitted from the sdist (see pypa/hatch#1197 for more info). Turns out it wasn't easy...
Beta Was this translation helpful? Give feedback.
All reactions
0 replies
-
@thegamecracks Would you be interested in PRing your suggested fix?
Beta Was this translation helpful? Give feedback.
All reactions
2 replies
-
Yup, #1183 implements the last suggestion along with a custom build script to work around the node_modules conflict.
Beta Was this translation helpful? Give feedback.
All reactions
-
❤️ 1
-
Since I have current high interest in this same topic, I'll point out that #1183 mentioned above was closed in favor of #1191.
Beta Was this translation helpful? Give feedback.
All reactions
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment