9
2
Fork
You've already forked platform
2

verify: fail fast and never hang #345

Manually merged
ryansquared merged 3 commits from fix/verify-app-source-preflight into main 2026年07月07日 14:57:51 +02:00

caution verify could waste 10+ minutes (or hang forever) before telling you something was wrong. Now it checks the cheap stuff first and bounds every network call.

  • Preflight the app source ref (git ls-remote) before building — missing branch fails in seconds, not minutes
  • Preflight enclave + framework archives too (HEAD request) — a 404'd commit fails fast
  • Added timeouts to the attestation call and archive downloads — they were unbounded and could hang forever
  • Guarded enclave-builder's ls-remote so it can't hang on a credential prompt
  • Hardened the commit-SHA matching + added tests
caution verify could waste 10+ minutes (or hang forever) before telling you something was wrong. Now it checks the cheap stuff first and bounds every network call. - Preflight the app source ref (git ls-remote) before building — missing branch fails in seconds, not minutes - Preflight enclave + framework archives too (HEAD request) — a 404'd commit fails fast - Added timeouts to the attestation call and archive downloads — they were unbounded and could hang forever - Guarded enclave-builder's ls-remote so it can't hang on a credential prompt - Hardened the commit-SHA matching + added tests
Add a git ls-remote preflight before the reproduce build so a missing
branch/commit fails in seconds instead of after minutes of archive
downloads and clone/fetch fallbacks.
Add connect/request timeouts to the attestation client and enclave/
framework archive downloads (were unbounded), guard enclave-builder's
ls-remote against credential-prompt hangs, HEAD-preflight enclave and
framework archives, and harden the app-source SHA match.
@ -1646,0 +1647,4 @@
// but never replies) can't hang the CLI forever. It does not cap body
// transfer, so large downloads are unaffected.
client: reqwest::Client::builder()
.connect_timeout(Duration::from_secs(30))

We can also do .read_timeout() to catch connections that die partway through. It's an idle read() timeout, not a deadline.

We can also do .read_timeout() to catch connections that die partway through. It's an idle read() timeout, not a deadline.
Author
Owner
Copy link

Added .read_timeout(90s) on the shared client. Has to stay above the attestation fetch's 60s per-request override, or it'd cut that request short

Added .read_timeout(90s) on the shared client. Has to stay above the attestation fetch's 60s per-request override, or it'd cut that request short

The read timeout is for "any data received". As long as we get any data every 30 seconds, even a byte, it won't terminate. It would still be terminated by a 60 seconds generic deadline.

The read timeout is for "any data received". As long as we get _any_ data every 30 seconds, even a byte, it won't terminate. It would still be terminated by a 60 seconds generic deadline.

Has to stay above the attestation fetch's 60s per-request override, or it'd cut that request short

read_timeout() is an idle timer, .timeout() is a deadline timer. Idle timers can always be shorter than deadline timers, since idle timers only count "time since last byte", whereas deadline timers count "time since first byte". Time since last byte should always be less-or-equal time since first byte.

> Has to stay above the attestation fetch's 60s per-request override, or it'd cut that request short read_timeout() is an idle timer, .timeout() is a deadline timer. Idle timers can always be shorter than deadline timers, since idle timers only count "time since last byte", whereas deadline timers count "time since first byte". Time since last byte should always be less-or-equal time since first byte.
Author
Owner
Copy link

.read_timeout(30s) for the idle case (half-dead connections), plus .timeout(60s) as the generic deadline.

`.read_timeout(30s)` for the idle case (half-dead connections), plus `.timeout(60s)` as the generic deadline.
@ -6364,0 +6409,4 @@
letresponse=matchself
.client
.head(url)
.timeout(Duration::from_secs(30))

Can we make use of read_timeout instead to not kill slow connections?

Can we make use of read_timeout instead to not kill slow connections?

read_timeout being added to the ClientBuilder, it's not available on RequestBuilder

read_timeout being added to the ClientBuilder, it's not available on RequestBuilder
Author
Owner
Copy link

Right, client-only. Leaving as is

Right, client-only. Leaving as is
ryansquared marked this conversation as resolved
@ -6364,0 +6469,4 @@
commit: &str,
branch: Option<&str>,
)-> Result<()>{
// Archive tarball URLs aren't ls-remote-able and 404 quickly on their own.

Could we run a HEAD against the url?

Could we run a HEAD against the url?
Author
Owner
Copy link

Done - /archive/*.tar[.gz] app-source URLs now fall back to preflight_archive_url (HEAD) instead of skipping

Done - `/archive/*.tar[.gz]` app-source URLs now fall back to preflight_archive_url (HEAD) instead of skipping
ryansquared marked this conversation as resolved
@ -6364,0 +6400,4 @@
/// unsupported HEAD (405), or a transport error is treated as inconclusive and
/// left for the real download to resolve. Non-HTTP sources (git URLs, local
/// paths) are skipped.
asyncfn preflight_archive_url(&self,label: &str,url: &str)-> Result<()>{

When adding new functions, please add associated error types that cover all unique variants of a function. This one should have something like:

#[derive(Clone, Debug, thiserror::Error)]#[error("The archive for commit {commit} was not available at {url} (HTTP {code})")]pubstruct PreflightArchiveUrlError{pubcommit: String,puburl: String,pubcode: StatusCode,}
When adding new functions, please add associated error types that cover all unique variants of a function. This one should have something like: ```rust #[derive(Clone, Debug, thiserror::Error)] #[error("The archive for commit {commit} was not available at {url} (HTTP {code})")] pub struct PreflightArchiveUrlError { pub commit: String, pub url: String, pub code: StatusCode, } ```
Author
Owner
Copy link

Added PreflightArchiveUrlError { label, url, code: StatusCode }

Added `PreflightArchiveUrlError { label, url, code: StatusCode }`
ryansquared marked this conversation as resolved
@ -6364,0 +6468,4 @@
git_url: &str,
commit: &str,
branch: Option<&str>,
)-> Result<()>{

Same as above, please add an error type covering all unique error paths. Looks like there's only one, so it can also be a struct. If you have code has multiple spots where it returns an error, please use an enum. If you are returning an error caused by another function's error, please use #[source] if additional context would be helpful, or #[from] if no additional context would help, and it's not an abstract error type - when using #[from], you can use the ? sigil without issue.

Same as above, please add an error type covering all unique error paths. Looks like there's only one, so it can also be a struct. If you have code has multiple spots where it returns an error, please use an enum. If you are returning an error caused by another function's error, please use `#[source]` if additional context would be helpful, or `#[from]` if no additional context would help, and it's not an abstract error type - when using `#[from]`, you can use the `?` sigil without issue.
Author
Owner
Copy link

Added AppSourceBranchMissingError (single path → struct)

Added `AppSourceBranchMissingError (single path → struct)`
ryansquared marked this conversation as resolved
@ -6364,0 +6550,4 @@
listing: &str,
commit: &str,
branch: Option<&str>,
)-> std::result::Result<bool,String>{

This should also have a ClassifyAppSourceRefsError { commit: String, branch: Option<String> }.

This should also have a `ClassifyAppSourceRefsError { commit: String, branch: Option<String> }`.
Author
Owner
Copy link

Done. ClassifyAppSourceRefsError { commit: String, branch: String }. Made branch a String (not Option) since the only error arm is Some(branch) && !present. Didn't #[source]-chain it under AppSourceBranchMissingError — main prints the full Caused by: chain, so it'd just repeat branch/commit; the caller consumes it into the richer error instead.

Done. `ClassifyAppSourceRefsError { commit: String, branch: String }`. Made branch a String (not Option) since the only error arm is Some(branch) && !present. Didn't #[source]-chain it under AppSourceBranchMissingError — main prints the full _Caused by_: chain, so it'd just repeat branch/commit; the caller consumes it into the richer error instead.

please include #[source] AppSourceBranchMissingError, as we may add additional context fields to it later, and that may be missed.

please include `#[source] AppSourceBranchMissingError`, as we may add additional context fields to it later, and that may be missed.
Author
Owner
Copy link

Done: AppSourceBranchMissingError #[source]-ing ClassifyAppSourceRefsError (the outer error wraps the inner one, since only it carries the url)

Done: `AppSourceBranchMissingError #[source]`-ing `ClassifyAppSourceRefsError` (the outer error wraps the inner one, since only it carries the url)
@ -159,0 +173,4 @@
/// bounds an unresponsive mirror and a total timeout bounds a stalled transfer,
/// so a missing or hung source fails in minutes at worst instead of hanging
/// forever — reqwest's default client (`reqwest::get`) has neither bound.
fn archive_http_client()-> Result<reqwest::Client>{

just return reqwest's error type or use .expect("codepath introduces no variables")

just return reqwest's error type or use `.expect("codepath introduces no variables")`
Author
Owner
Copy link

Fixed .expect("archive HTTP client config is static"). Returns reqwest::Client now, so both call sites lose ?

Fixed `.expect("archive HTTP client config is static")`. Returns `reqwest::Client` now, so both call sites lose `?`
ryansquared marked this conversation as resolved
- verify preflight: convert bail!/context to thiserror error types
- bound shared reqwest client with a read timeout
- make archive_http_client infallible (static config)
vkobel force-pushed fix/verify-app-source-preflight from 8408451c39 to 5ca4b39144 2026年07月03日 17:36:52 +02:00 Compare
vkobel force-pushed fix/verify-app-source-preflight from 5ca4b39144 to b12d139832 2026年07月07日 09:57:25 +02:00 Compare
ryansquared manually merged commit 9aebefb51c into main 2026年07月07日 14:57:51 +02:00
Sign in to join this conversation.
No reviewers
Labels
Clear labels
Compat/Breaking
Breaking change that won't be backward compatible
Component/Enclave-Builder
This issue affects Enclave Builder (git-push and CLI)
Component/Infrastructure
This issue affects generic Caution hosted infra components
Deploy/BYOC
Relates to Caution Bring-your-own-compute
Deploy/Full-managed
Relates to Caution fully managed cloud
Deploy/Self-hosted
Relates to Caution deployed on self-hosted infra
Interface/API
This issue affects the API
Interface/CLI
This issue affects the command line interface
Interface/Git
This issue affects the Git `push` interface
Interface/Web
This issue affects the web dashboard
Kind/Bug
Something is not working
Kind/Documentation
Documentation changes
Kind/Enhancement
Improve existing functionality
Kind/Feature
New functionality
Kind/Security
This is security issue
Kind/Testing
Issue or pull request related to testing
Priority
Critical
The priority is critical
Priority
High
The priority is high
Priority
Low
The priority is low
Priority
Medium
The priority is medium
Reviewed
Confirmed
Issue has been confirmed
Reviewed
Duplicate
This issue or pull request already exists
Reviewed
Invalid
Invalid issue
Reviewed
Won't Fix
This issue won't be fixed
Status
Abandoned
Somebody has started to work on this but abandoned work
Status
Blocked
Something is blocking this issue or pull request
Status
Need More Info
Feedback is required to reproduce issue or to continue work
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
caution/platform!345
Reference in a new issue
caution/platform
No description provided.
Delete branch "fix/verify-app-source-preflight"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?