ziglang/zig
261
5.9k
Fork
You've already forked zig
767

std.http.Client: Fix HTTP over Unix sockets #31742

Open
fardragon wants to merge 7 commits from fardragon/zig:fardragon/31135-http-unix-sockets into master
pull from: fardragon/zig:fardragon/31135-http-unix-sockets
merge into: ziglang:master
ziglang:master
ziglang:elfv2
ziglang:spork8
ziglang:restricted
ziglang:0.16.x
ziglang:panic-rewrite
ziglang:parking-futex-lockfree
ziglang:windows-Io-cleanup
ziglang:Io-watch
ziglang:ParseCommandLineOptions
ziglang:windows-async-files
ziglang:poll-ring
ziglang:debug-file-leaks-differently
ziglang:debug-file-leaks
ziglang:ProcessPrng
ziglang:elfv2-dyn
ziglang:jobserver
ziglang:threadtheft
ziglang:io-threaded-no-queue
ziglang:0.15.x
ziglang:Io.net
ziglang:comptime-allocator
ziglang:restricted-function-pointers
ziglang:cli
ziglang:wasm-linker-writer
ziglang:wrangle-writer-buffering
ziglang:sha1-stream
ziglang:async-await-demo
ziglang:fixes
ziglang:0.14.x
ziglang:ast-node-methods
ziglang:macos-debug-info
ziglang:make-vs-configure
ziglang:fuzz-macos
ziglang:sans-aro
ziglang:ArrayList-reserve
ziglang:incr-bug
ziglang:llvm-ir-nosanitize-metadata
ziglang:ci-tarballs
ziglang:ci-scripts
ziglang:threadpool
ziglang:0.12.x
ziglang:new-pkg-hash
ziglang:json-diagnostics
ziglang:more-doctests
ziglang:rework-comptime-mutation
ziglang:0.11.x
ziglang:ci-perf-comment
ziglang:stage2-async
ziglang:0.10.x
ziglang:autofix
ziglang:0.9.x
ziglang:aro
ziglang:hcs
ziglang:0.8.x
ziglang:0.7.x
Contributor
Copy link
  • Fixes std.http.Client.connectUnix
  • Adds http+unix scheme to allow passing percent encoded socket paths directly to std.http.Client.request. This is not standardized afaik, but the precedent is there in the form of python's requests-unixsocket or httpie. A way to differentiate between TCP and Unix URIs is needed in any case to properly handle HTTP redirects.
  • Runs subset of HTTP tests over unix sockets
  • Additionally fixes a somewhat related issue in libstd.Io.Threaded.addressUnixToPosix, where it would add an extra null byte at the end of an abstract socket path, leading to confusing connection errors

Closes #31135

* Fixes `std.http.Client.connectUnix` * Adds `http+unix` scheme to allow passing percent encoded socket paths directly to `std.http.Client.request`. This is not standardized afaik, but the precedent is there in the form of python's `requests-unixsocket` or `httpie`. A way to differentiate between TCP and Unix URIs is needed in any case to properly handle HTTP redirects. * Runs subset of HTTP tests over unix sockets * Additionally fixes a somewhat related issue in `libstd.Io.Threaded.addressUnixToPosix`, where it would add an extra null byte at the end of an abstract socket path, leading to confusing connection errors Closes #31135
Run part of HTTP server/client tests on unix sockets as well
Some checks failed
ci / aarch64-freebsd-debug (pull_request) Waiting to run
ci / aarch64-freebsd-release (pull_request) Waiting to run
ci / aarch64-netbsd-debug (pull_request) Waiting to run
ci / aarch64-netbsd-release (pull_request) Waiting to run
ci / riscv64-linux-debug (pull_request) Waiting to run
ci / riscv64-linux-release (pull_request) Waiting to run
ci / x86_64-freebsd-debug (pull_request) Failing after 54m2s
ci / x86_64-netbsd-debug (pull_request) Failing after 58m19s
ci / x86_64-netbsd-release (pull_request) Failing after 45m13s
ci / x86_64-freebsd-release (pull_request) Failing after 42m35s
ci / aarch64-macos-release (pull_request) Failing after 1h24m43s
ci / aarch64-linux-release (pull_request) Failing after 1h12m27s
ci / x86_64-linux-debug (pull_request) Failing after 59m22s
ci / aarch64-macos-debug (pull_request) Failing after 2h3m43s
ci / aarch64-linux-debug (pull_request) Failing after 2h2m39s
ci / x86_64-openbsd-release (pull_request) Successful in 1h2m27s
ci / x86_64-openbsd-debug (pull_request) Successful in 1h44m42s
ci / x86_64-windows-debug (pull_request) Failing after 1h32m32s
ci / x86_64-windows-release (pull_request) Failing after 59m33s
ci / x86_64-linux-release (pull_request) Failing after 2h21m30s
ci / x86_64-linux-debug-llvm (pull_request) Failing after 3h3m15s
ci / powerpc64le-linux-release (pull_request) Failing after 1h24m50s
ci / s390x-linux-release (pull_request) Successful in 1h28m32s
ci / s390x-linux-debug (pull_request) Successful in 2h52m1s
ci / powerpc64le-linux-debug (pull_request) Failing after 3h18m4s
ci / loongarch64-linux-release (pull_request) Failing after 1h37m28s
ci / loongarch64-linux-debug (pull_request) Failing after 2h57m49s
c940138478
Author
Contributor
Copy link

Looking into the failures, looks like I was a bit too optimistic about the temporary path length in the tests

Looking into the failures, looks like I was a bit too optimistic about the temporary path length in the tests
Address CI failures. Remove realPath usage. Put sockets in cwd with random names.
All checks were successful
ci / aarch64-macos-release (pull_request) Successful in 1h18m55s
ci / aarch64-linux-release (pull_request) Successful in 1h27m54s
ci / x86_64-netbsd-release (pull_request) Successful in 43m10s
ci / x86_64-linux-debug (pull_request) Successful in 1h29m38s
ci / x86_64-netbsd-debug (pull_request) Successful in 51m14s
ci / aarch64-macos-debug (pull_request) Successful in 1h36m22s
ci / x86_64-freebsd-release (pull_request) Successful in 41m33s
ci / x86_64-freebsd-debug (pull_request) Successful in 1h8m57s
ci / x86_64-windows-release (pull_request) Successful in 50m45s
ci / x86_64-windows-debug (pull_request) Successful in 1h11m24s
ci / x86_64-openbsd-release (pull_request) Successful in 1h12m9s
ci / x86_64-openbsd-debug (pull_request) Successful in 1h18m49s
ci / x86_64-linux-release (pull_request) Successful in 2h7m1s
ci / aarch64-linux-debug (pull_request) Successful in 2h27m57s
ci / s390x-linux-release (pull_request) Successful in 1h21m45s
ci / x86_64-linux-debug-llvm (pull_request) Successful in 3h37m43s
ci / s390x-linux-debug (pull_request) Successful in 3h8m4s
ci / powerpc64le-linux-release (pull_request) Successful in 1h23m50s
ci / powerpc64le-linux-debug (pull_request) Successful in 4h37m33s
ci / loongarch64-linux-debug (pull_request) Successful in 3h27m41s
ci / loongarch64-linux-release (pull_request) Successful in 1h46m20s
ci / aarch64-freebsd-release (pull_request) Successful in 2h53m25s
ci / aarch64-freebsd-debug (pull_request) Successful in 3h51m57s
ci / riscv64-linux-debug (pull_request) Has been skipped
ci / riscv64-linux-release (pull_request) Has been skipped
ci / aarch64-netbsd-debug (pull_request) Successful in 4h13m50s
ci / aarch64-netbsd-release (pull_request) Successful in 2h39m2s
73e3c8804d
andrewrk left a comment
Copy link

Thank you for the patch.

Thank you for the patch.
@ -14002,3 +14002,3 @@
// use a suffix which is usually the most relevant part of a path.
@memcpy(storage.un.path[0..path_len],a.path[a.path.len-path_len..]);
if(storage.un.path.len-path_len>0){
if(storage.un.path[0]!=0andstorage.un.path.len-path_len>0){
Owner
Copy link

this additional check needs a comment to explain its existence because it is not immediately obvous.

this additional check needs a comment to explain its existence because it is not immediately obvous.
Author
Contributor
Copy link

Added

Added
fardragon marked this conversation as resolved
Add comments
All checks were successful
ci / riscv64-linux-debug (pull_request) Has been skipped
ci / riscv64-linux-release (pull_request) Has been skipped
ci / x86_64-netbsd-release (pull_request) Successful in 47m19s
ci / x86_64-freebsd-release (pull_request) Successful in 50m57s
ci / x86_64-netbsd-debug (pull_request) Successful in 1h1m59s
ci / x86_64-freebsd-debug (pull_request) Successful in 1h3m37s
ci / aarch64-macos-release (pull_request) Successful in 1h31m58s
ci / x86_64-openbsd-release (pull_request) Successful in 1h18m19s
ci / x86_64-linux-debug (pull_request) Successful in 1h31m13s
ci / x86_64-openbsd-debug (pull_request) Successful in 1h23m11s
ci / aarch64-macos-debug (pull_request) Successful in 2h2m21s
ci / x86_64-windows-debug (pull_request) Successful in 1h23m9s
ci / aarch64-linux-debug (pull_request) Successful in 2h38m24s
ci / x86_64-windows-release (pull_request) Successful in 1h15m46s
ci / x86_64-linux-debug-llvm (pull_request) Successful in 2h51m3s
ci / x86_64-linux-release (pull_request) Successful in 3h29m0s
ci / powerpc64le-linux-release (pull_request) Successful in 1h42m39s
ci / aarch64-linux-release (pull_request) Successful in 1h26m17s
ci / loongarch64-linux-debug (pull_request) Successful in 3h25m1s
ci / powerpc64le-linux-debug (pull_request) Successful in 4h38m14s
ci / s390x-linux-release (pull_request) Successful in 2h36m23s
ci / s390x-linux-debug (pull_request) Successful in 4h0m5s
ci / aarch64-freebsd-debug (pull_request) Successful in 3h48m58s
ci / aarch64-freebsd-release (pull_request) Successful in 2h47m23s
ci / aarch64-netbsd-debug (pull_request) Successful in 4h21m57s
ci / aarch64-netbsd-release (pull_request) Successful in 2h26m35s
ci / loongarch64-linux-release (pull_request) Successful in 1h50m36s
196a289698
Merge remote-tracking branch 'origin/master' into fardragon/31135-http-unix-sockets
All checks were successful
ci / x86_64-netbsd-release (pull_request) Successful in 36m28s
ci / x86_64-freebsd-debug (pull_request) Successful in 57m22s
ci / x86_64-freebsd-release (pull_request) Successful in 46m38s
ci / aarch64-macos-release (pull_request) Successful in 1h39m47s
ci / x86_64-netbsd-debug (pull_request) Successful in 1h10m2s
ci / aarch64-macos-debug (pull_request) Successful in 2h20m54s
ci / aarch64-linux-release (pull_request) Successful in 1h28m50s
ci / aarch64-linux-debug (pull_request) Successful in 2h25m40s
ci / x86_64-openbsd-release (pull_request) Successful in 1h12m51s
ci / x86_64-openbsd-debug (pull_request) Successful in 1h21m37s
ci / x86_64-windows-release (pull_request) Successful in 1h20m38s
ci / x86_64-windows-debug (pull_request) Successful in 1h51m7s
ci / x86_64-linux-debug (pull_request) Successful in 1h9m57s
ci / x86_64-linux-release (pull_request) Successful in 2h32m59s
ci / x86_64-linux-debug-llvm (pull_request) Successful in 3h7m51s
ci / s390x-linux-release (pull_request) Successful in 1h18m23s
ci / s390x-linux-debug (pull_request) Successful in 2h56m9s
ci / powerpc64le-linux-release (pull_request) Successful in 1h44m13s
ci / powerpc64le-linux-debug (pull_request) Successful in 4h1m50s
ci / loongarch64-linux-release (pull_request) Successful in 2h7m2s
ci / loongarch64-linux-debug (pull_request) Successful in 3h18m57s
ci / aarch64-freebsd-debug (pull_request) Successful in 4h3m30s
ci / aarch64-freebsd-release (pull_request) Successful in 3h7m41s
ci / aarch64-netbsd-debug (pull_request) Successful in 3h37m9s
ci / aarch64-netbsd-release (pull_request) Successful in 2h43m8s
ci / riscv64-linux-debug (pull_request) Has been skipped
ci / riscv64-linux-release (pull_request) Has been skipped
02acc88cb7
Merge remote-tracking branch 'origin/master' into fardragon/31135-http-unix-sockets
All checks were successful
ci / x86_64-freebsd-release (pull_request) Successful in 48m17s
ci / x86_64-netbsd-debug (pull_request) Successful in 56m43s
ci / x86_64-freebsd-debug (pull_request) Successful in 1h5m39s
ci / x86_64-linux-debug (pull_request) Successful in 1h17m15s
ci / x86_64-windows-release (pull_request) Successful in 1h13m54s
ci / x86_64-openbsd-release (pull_request) Successful in 1h14m52s
ci / x86_64-windows-debug (pull_request) Successful in 1h24m4s
ci / x86_64-openbsd-debug (pull_request) Successful in 1h26m49s
ci / aarch64-linux-release (pull_request) Successful in 1h50m47s
ci / aarch64-macos-release (pull_request) Successful in 1h55m24s
ci / powerpc64le-linux-release (pull_request) Successful in 2h0m32s
ci / aarch64-macos-debug (pull_request) Successful in 2h4m45s
ci / x86_64-netbsd-release (pull_request) Successful in 45m56s
ci / aarch64-linux-debug (pull_request) Successful in 2h48m1s
ci / x86_64-linux-release (pull_request) Successful in 2h47m37s
ci / loongarch64-linux-release (pull_request) Successful in 1h45m40s
ci / x86_64-linux-debug-llvm (pull_request) Successful in 3h5m49s
ci / powerpc64le-linux-debug (pull_request) Successful in 3h30m32s
ci / loongarch64-linux-debug (pull_request) Successful in 3h53m6s
ci / s390x-linux-debug (pull_request) Successful in 5h16m25s
ci / s390x-linux-release (pull_request) Successful in 2h21m10s
ci / aarch64-freebsd-debug (pull_request) Successful in 3h47m5s
ci / aarch64-freebsd-release (pull_request) Successful in 2h57m17s
ci / aarch64-netbsd-release (pull_request) Successful in 3h20m26s
ci / aarch64-netbsd-debug (pull_request) Successful in 4h44m28s
ci / riscv64-linux-debug (pull_request) Has been skipped
ci / riscv64-linux-release (pull_request) Has been skipped
d43c3c6b8c
All checks were successful
ci / x86_64-freebsd-release (pull_request) Successful in 48m17s
Required
Details
ci / x86_64-netbsd-debug (pull_request) Successful in 56m43s
Required
Details
ci / x86_64-freebsd-debug (pull_request) Successful in 1h5m39s
Required
Details
ci / x86_64-linux-debug (pull_request) Successful in 1h17m15s
Required
Details
ci / x86_64-windows-release (pull_request) Successful in 1h13m54s
Required
Details
ci / x86_64-openbsd-release (pull_request) Successful in 1h14m52s
Required
Details
ci / x86_64-windows-debug (pull_request) Successful in 1h24m4s
Required
Details
ci / x86_64-openbsd-debug (pull_request) Successful in 1h26m49s
Required
Details
ci / aarch64-linux-release (pull_request) Successful in 1h50m47s
Required
Details
ci / aarch64-macos-release (pull_request) Successful in 1h55m24s
Required
Details
ci / powerpc64le-linux-release (pull_request) Successful in 2h0m32s
Required
Details
ci / aarch64-macos-debug (pull_request) Successful in 2h4m45s
Required
Details
ci / x86_64-netbsd-release (pull_request) Successful in 45m56s
Required
Details
ci / aarch64-linux-debug (pull_request) Successful in 2h48m1s
Required
Details
ci / x86_64-linux-release (pull_request) Successful in 2h47m37s
Required
Details
ci / loongarch64-linux-release (pull_request) Successful in 1h45m40s
ci / x86_64-linux-debug-llvm (pull_request) Successful in 3h5m49s
Required
Details
ci / powerpc64le-linux-debug (pull_request) Successful in 3h30m32s
Required
Details
ci / loongarch64-linux-debug (pull_request) Successful in 3h53m6s
ci / s390x-linux-debug (pull_request) Successful in 5h16m25s
Required
Details
ci / s390x-linux-release (pull_request) Successful in 2h21m10s
Required
Details
ci / aarch64-freebsd-debug (pull_request) Successful in 3h47m5s
ci / aarch64-freebsd-release (pull_request) Successful in 2h57m17s
ci / aarch64-netbsd-release (pull_request) Successful in 3h20m26s
ci / aarch64-netbsd-debug (pull_request) Successful in 4h44m28s
ci / riscv64-linux-debug (pull_request) Has been skipped
ci / riscv64-linux-release (pull_request) Has been skipped
This pull request has changes conflicting with the target branch.
  • lib/std/Uri.zig
  • lib/std/http/Client.zig
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u fardragon/31135-http-unix-sockets:fardragon-fardragon/31135-http-unix-sockets
git switch fardragon-fardragon/31135-http-unix-sockets
Sign in to join this conversation.
No reviewers
Labels
Clear labels
abi/f32
abi/ilp32
abi/sf
accepted
This proposal is planned.
arch/21k
arch/6502
arch/aarch64
arch/alpha
arch/amdgcn
arch/arc
arch/arc32
arch/arc64
arch/arm
arch/avr
arch/bfin
arch/bpf
arch/colossus
arch/cris
arch/csky
arch/dlx
arch/epiphany
arch/fr30
arch/frv
arch/hexagon
arch/hppa
arch/hppa64
arch/ia64
arch/kalimba
arch/kvx
arch/lanai
arch/lm32
arch/loongarch32
arch/loongarch64
arch/m32r
arch/m68k
arch/m88k
arch/mcore
arch/microblaze
arch/mips
arch/mips64
arch/mmix
arch/moxie
arch/mrisc32
arch/msp430
arch/nds32
arch/ns32k
arch/nvptx
arch/or1k
arch/powerpc
arch/powerpc64
arch/propeller
arch/riscv32
arch/riscv64
arch/rl78
arch/rx
arch/s390x
arch/sh
arch/sparc
arch/sparc64
arch/spirv
arch/spu
arch/tricore
arch/v850
arch/vax
arch/vc4
arch/ve
arch/wasm
arch/x86
arch/x86_64
arch/xcore
arch/xtensa
autodoc
The web application for interactive documentation and generation of its assets.
backend/c
The C backend outputs C source code.
backend/llvm
The LLVM backend outputs an LLVM bitcode module.
backend/self-hosted
The self-hosted backends produce machine code directly.
binutils
Zig's included binary utilities: zig ar, zig dlltool, zig lib, zig ranlib, zig objcopy, and zig rc.
breaking
Implementing this issue could cause existing code to no longer compile or have different behavior.
build system
The Zig build system - zig build, std.Build, the build runner, and package management.
debug info
An issue related to debug information (e.g. DWARF) produced by the Zig compiler.
docs
An issue with documentation, e.g. the language reference or standard library doc comments.
error message
This issue points out an error message that is unhelpful and should be improved.
frontend
Tokenization, parsing, AstGen, ZonGen, Sema, Legalize, and Liveness.
fuzzing
An issue related to Zig's integrated fuzz testing.
incremental
Reuse of internal compiler state for faster compilation.
lib/c
This issue relates to Zig's libc implementation and/or vendored libcs.
lib/compiler-rt
This issue relates to Zig's compiler-rt library.
lib/cxx
This issue relates to Zig's vendored libc++ and/or libc++abi.
lib/std
This issue relates to Zig's standard library.
lib/tsan
This issue relates to Zig's vendored libtsan.
lib/ubsan-rt
This issue relates to Zig's ubsan-rt library.
lib/unwind
This issue relates to Zig's vendored libunwind.
linking
Zig's integrated object file and incremental linker.
miscompilation
The compiler reports success but produces semantically incorrect code.
os/android
os/contiki
os/dragonfly
os/driverkit
os/emscripten
os/freebsd
os/fuchsia
os/haiku
os/hermit
os/hurd
os/illumos
os/ios
os/linux
os/maccatalyst
os/macos
os/managarm
os/netbsd
os/ohos
os/openbsd
os/plan9
os/redox
os/rtems
os/serenity
os/tvos
os/uefi
os/visionos
os/wasi
os/watchos
os/windows
proposal
This issue suggests language modifications. If it also has the "accepted" label then it is planned.
release notes
This issue or pull request should be mentioned in the release notes.
testing
This issue is related to testing the compiler, standard library, or other parts of Zig.
zig cc
Zig as a drop-in C-family compiler.
zig fmt
The Zig source code formatter.
zig reduce
The Zig source code reduction tool.
bounty
https://ziglang.org/news/announcing-donor-bounties
bug
Observed behavior contradicts documented or intended behavior.
contributor-friendly
This issue is limited in scope and/or knowledge of project internals.
downstream
An issue with a third-party project that uses this project.
enhancement
Solving this issue will likely involve adding new logic or components to the codebase.
infra
An issue related to project infrastructure, e.g. continuous integration.
optimization
A task to improve performance and/or resource usage.
question
No questions on the issue tracker; use a community space instead.
regression
Something that used to work in a previous version stopped working
upstream
An issue with a third-party project that this project uses.
use case
Describes a real use case that is difficult or impossible, but does not propose a solution.
No labels
abi/f32
abi/ilp32
abi/sf
accepted
arch/21k
arch/6502
arch/aarch64
arch/alpha
arch/amdgcn
arch/arc
arch/arc32
arch/arc64
arch/arm
arch/avr
arch/bfin
arch/bpf
arch/colossus
arch/cris
arch/csky
arch/dlx
arch/epiphany
arch/fr30
arch/frv
arch/hexagon
arch/hppa
arch/hppa64
arch/ia64
arch/kalimba
arch/kvx
arch/lanai
arch/lm32
arch/loongarch32
arch/loongarch64
arch/m32r
arch/m68k
arch/m88k
arch/mcore
arch/microblaze
arch/mips
arch/mips64
arch/mmix
arch/moxie
arch/mrisc32
arch/msp430
arch/nds32
arch/ns32k
arch/nvptx
arch/or1k
arch/powerpc
arch/powerpc64
arch/propeller
arch/riscv32
arch/riscv64
arch/rl78
arch/rx
arch/s390x
arch/sh
arch/sparc
arch/sparc64
arch/spirv
arch/spu
arch/tricore
arch/v850
arch/vax
arch/vc4
arch/ve
arch/wasm
arch/x86
arch/x86_64
arch/xcore
arch/xtensa
autodoc
backend/c
backend/llvm
backend/self-hosted
binutils
breaking
build system
debug info
docs
error message
frontend
fuzzing
incremental
lib/c
lib/compiler-rt
lib/cxx
lib/std
lib/tsan
lib/ubsan-rt
lib/unwind
linking
miscompilation
os/android
os/contiki
os/dragonfly
os/driverkit
os/emscripten
os/freebsd
os/fuchsia
os/haiku
os/hermit
os/hurd
os/illumos
os/ios
os/linux
os/maccatalyst
os/macos
os/managarm
os/netbsd
os/ohos
os/openbsd
os/plan9
os/redox
os/rtems
os/serenity
os/tvos
os/uefi
os/visionos
os/wasi
os/watchos
os/windows
proposal
release notes
testing
zig cc
zig fmt
zig reduce
bounty
bug
contributor-friendly
downstream
enhancement
infra
optimization
question
regression
upstream
use case
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
ziglang/zig!31742
Reference in a new issue
ziglang/zig
No description provided.
Delete branch "fardragon/zig:fardragon/31135-http-unix-sockets"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?