-
Notifications
You must be signed in to change notification settings - Fork 84
feat: Emit worker pool size as a DEBUG log event during init on Lambda Managed Instances - #219
Open
vip-amzn wants to merge 3 commits into
Open
feat: Emit worker pool size as a DEBUG log event during init on Lambda Managed Instances #219vip-amzn wants to merge 3 commits into
vip-amzn wants to merge 3 commits into
Conversation
...a Managed Instances
maxday
commented
Sep 3, 2026
Member
Nice! My one real concern is the change in JsonFormatter.format: it now passes a raw dict message straight into a JSON encoder that has no fallback, and this applies to every log call, not just the new event.
So if anyone logs a dict that contains a non-serializable value (a datetime, Decimal, bytes, a custom object, etc.), formatting will raise and the log line gets dropped, whereas today getMessage() just stringifies it safely.
Could we add default=str to the encoder (or a try/except that falls back to the old behavior) so we don't regress existing dict logging?
maxday
commented
Sep 3, 2026
Member
Also, the sink that init_logging() opens is never closed, I think the fd leaks for FramedTelemetryLogSink since handlers.clear() doesn't close it?
Address PR review feedback: - Add default=str to the JSON log encoder so dict messages (and extra attributes) containing non-serializable values are stringified instead of raising and dropping the log record. - Close the log sink opened by init_logging deterministically after the parent's handler is removed, instead of relying on GC.
maxday
maxday
approved these changes
Sep 3, 2026
RAPID wires the runtime main process's stdout/stderr to the log egress at spawn, so the parent does not need to dial the telemetry FD provider socket — that socket exists for the forked workers, which continue to redirect in run_single. Addresses PR review feedback.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
On Lambda Managed Instances,
the runtime sizes and spawns a worker pool during initialization to handle
multiple concurrent invocations per execution environment. Today there is no
way to observe the effective pool size: it can be influenced by configuration,
but nothing confirms what the runtime actually resolved.
This change adds a one-time structured JSON log event, emitted at DEBUG level
during runtime initialization, reporting the resolved worker pool size and the
maximum concurrency the execution environment supports. Both values are
included so the worker count is not mistaken for the supported concurrency.
Example (visible when the function's application log level is DEBUG or lower):
{ "timestamp": "...", "level": "DEBUG", "message": { "event": "runtime_worker_pool_initializing", "workerCount": 17, "executionEnvironmentMaxConcurrency": 34 } }Design notes:
so it only appears when the application log level is set to DEBUG or lower.
Default configurations see no new log lines and no added CloudWatch cost.
already initialized during bootstrap.
path is untouched.
(e.g.
filter message.event = "runtime_worker_pool_initializing").See also: Python runtime for Lambda Managed Instances
Testing
correct schema and values, suppressed at INFO and above, and not emitted on
the single-concurrency path.
execution environment at DEBUG, never appears at INFO or above, and repeated
sequential/concurrent invocations neither re-emit the event nor affect
normal invocation logs.