Improve concurrency handling, fix part of CI pipeline and more #39

Original file line number	Diff line number	Diff line change
Expand Up		@@ -5,6 +5,15 @@ This directory contains the DevContainer configuration for developing FineCode i
		- `devcontainer.json`: Main configuration file for VS Code DevContainers.
		- `docker-compose.devcontainer.yml`: Compose service definition for the main workspace container.

	## Node.js

	Node.js 22 is a runtime dependency of `setup_system` handlers that install
	npm-distributed tools (e.g. pi coding agent, which rejects Node older than
	22.19.0). The base image already ships `nvm` itself (no Node version installed), so
	`setup-node.sh` installs Node 22 through that pre-existing `nvm` in
	`postCreateCommand`. Because Node is installed via nvm under
	the `vscode` user, `npm install -g` works without sudo.

		## Local observability stack (opt-in)

		The devcontainer includes the repository-level `docker-compose.otel.yml`, but all of
Expand All		@@ -24,6 +33,26 @@ WAL events are recorded on disk regardless of whether the stack is running, so y
		bring it up later and ingest the history retroactively. See
		[ADR-0052](../../finecode_internal_docs/adr/0052-observability-stack-opt-in-via-compose-profile.md).

	## Persistent WM server

	`FINECODE_WM_AUTOSTART=1` is set in `.env.example`, so `postStartCommand` runs
	`start-wm-server --detach --keep-alive` on every container start, via
	`start-wm-server.sh`. The workspace stays warm across commands instead of rebuilding
	its config and runners each time. Comment the variable out to go back to a server
	per client; the script is also a no-op when the `dev_workspace` venv does not exist
	yet.

	Changing the variable needs the container recreated, not reopened — Compose
	resolves `.env` into a container's environment only when that container is created
	(the same caveat as `FINECODE_OTLP_ENDPOINT` above; see [Developing
	FineCode](../docs/guides/developing-finecode.md#local-observability-stack) for the
	rebuild commands).

	What comes with a keep-alive server — resident extension runners, a fixed log level,
	and why a server started lazily after a crash is not one — is described under
	[`start-wm-server`](../docs/cli.md#start-wm-server). Re-run
	`sh .devcontainer/start-wm-server.sh` to get the persistent one back.

		## Optional private internal docs mount

		The workspace service supports an optional bind mount for private internal docs.
Expand Down

23 changes: 22 additions & 1 deletion .devcontainer/devcontainer.json

Show comments View file Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -6,7 +6,28 @@
		],
		"service": "workspace",
		"shutdownAction": "stopCompose",
	"postCreateCommand": "sh .devcontainer/post-create-dev-workspace.sh && sh .devcontainer/setup-shell.sh",
	// Node.js is required by the pi coding agent, which setup_system installs via
	// npm. Pi's preflight rejects anything older than 22.19.0, so Debian's own
	// nodejs package is not an option, and so is the node devcontainer feature: the
	// base image already ships nvm itself, and running the feature on top of that
	// hits a BuildKit bug writing into nvm's cache dir. setup-node.sh installs Node
	// via that pre-existing nvm instead, at container-create time. See
	// setup-node.sh for the full explanation.
	// remoteEnv (not containerEnv): containerEnv is baked into the compose
	// service's own environment, including its `sleep infinity` entrypoint process —
	// and "${containerEnv:PATH}" does not self-resolve there, so it overwrote PATH
	// with the literal unexpanded string and broke everything, including `sleep`.
	// remoteEnv only applies to devcontainer-managed processes (postCreateCommand,
	// terminals), where the substitution does work.
	"remoteEnv": {
	"NVM_DIR": "/usr/local/share/nvm",
	"PATH": "/usr/local/share/nvm/current/bin:${containerEnv:PATH}"
	},
	"postCreateCommand": "sh .devcontainer/setup-node.sh && sh .devcontainer/post-create-dev-workspace.sh && sh .devcontainer/setup-shell.sh",
	// postStart, not postCreate: a stopped container loses its WM server, so this
	// has to run again on every start. The venv it needs may not exist yet on the
	// first create; the script exits cleanly in that case.
	"postStartCommand": "sh .devcontainer/start-wm-server.sh",
		"customizations": {
		"vscode": {
		"extensions": [
Expand Down

3 changes: 3 additions & 0 deletions .devcontainer/docker-compose.devcontainer.yml

Show comments View file Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -11,6 +11,9 @@ services:
		# both COMPOSE_PROFILES=otel and FINECODE_OTLP_ENDPOINT in .env — the ready
		# collector URL is provided there, so developers never compose it themselves.
		- FINECODE_OTLP_ENDPOINT=${FINECODE_OTLP_ENDPOINT:-}
	# Read only by .devcontainer/start-wm-server.sh; passed through here so the
	# value set in .env is visible to processes inside the container.
	- FINECODE_WM_AUTOSTART=${FINECODE_WM_AUTOSTART:-}
		# uv's default cache (~/.cache/uv) lives on the container's overlay
		# filesystem, a different device than this bind-mounted workspace. uv
		# dedupes installs by hardlinking from its cache into each venv, but
Expand Down

24 changes: 24 additions & 0 deletions .devcontainer/setup-node.sh

Show comments View file Open in desktop

Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,24 @@
	#!/bin/sh
	# Installs Node via the nvm already baked into the base image.
	#
	# We do NOT use the ghcr.io/devcontainers/features/node feature for this:
	# the base image already ships nvm itself (no Node version installed yet)
	# at /usr/local/share/nvm, owned by vscode:nvm. Running the node feature on
	# top of that hits its "NVM already installed" code path, which skips the
	# ownership fixup it only does for a freshly created nvm dir and calls
	# `nvm install` directly. That specific command works fine under a normal
	# `docker run`, but fails with "Permission denied" writing into
	# /usr/local/share/nvm/.cache when run inside a BuildKit RUN layer, which
	# does not reproduce ownership from the base image's read-only layer
	# correctly on copy-up. Running nvm install here, in postCreateCommand
	# (a normal container process, not a build layer) sidesteps the bug.
	set -eu

	export NVM_DIR="/usr/local/share/nvm"
	export NVM_SYMLINK_CURRENT=true
	# shellcheck source=/dev/null
	. "$NVM_DIR/nvm.sh"

	umask 0002
	nvm install 22
	nvm alias default 22

29 changes: 29 additions & 0 deletions .devcontainer/start-wm-server.sh

Show comments View file Open in desktop

Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,29 @@
	#!/bin/sh
	# Start a persistent shared FineCode WM server at container start, so the
	# workspace stays warm for the life of the container. What that costs is
	# documented in docs/cli.md, "Autostarting a persistent server".
	#
	# FineCode itself never reads FINECODE_WM_AUTOSTART — this script is its only
	# consumer, and it turns it into an explicit `--keep-alive` on the server it
	# starts, because keep-alive must never be ambient.
	set -eu

	if [ -z "${FINECODE_WM_AUTOSTART:-}" ] \|\| [ "${FINECODE_WM_AUTOSTART}" = "0" ]; then
	exit 0
	fi

	VENV_PYTHON=".venvs/dev_workspace/bin/python"
	if [ ! -x "$VENV_PYTHON" ]; then
	echo "FineCode WM autostart: $VENV_PYTHON not found, skipping." >&2
	exit 0
	fi

	# Never fatal: the workspace is fully usable without it, because every client
	# starts a server on demand anyway. Failing container start over a warm cache
	# would be a worse outcome than the cold start it is trying to avoid.
	if "$VENV_PYTHON" -m finecode start-wm-server --detach --keep-alive; then
	echo "FineCode WM autostart: shared server running."
	else
	echo "FineCode WM autostart: no shared server reachable yet; it may still be" \
	"starting, and clients attach to it when it is." >&2
	fi

6 changes: 6 additions & 0 deletions .env.example

Show comments View file Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -20,3 +20,9 @@ DEVCONTAINER_INTERNAL_DOCS_PATH=
		# use it to start/stop the collector while the endpoint stays armed.
		# COMPOSE_PROFILES=otel
		# FINECODE_OTLP_ENDPOINT=http://otel-lgtm:4317

	# Start a persistent shared WM server when the devcontainer starts, so config and
	# extension runners survive between commands. Comment out to get a server per
	# client instead. What comes with it: docs/cli.md, "Autostarting a persistent
	# server".
	FINECODE_WM_AUTOSTART=1

5 changes: 2 additions & 3 deletions .github/workflows/ci-cd.yml

Show comments View file Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -132,14 +132,13 @@ jobs:
		if: ${{ !cancelled() && steps.install.outcome == 'success' }}
		run: \|
		source .venvs/dev_workspace/${{ matrix.venv_bin }}/activate
	# TODO: test with all supported python versions
		python -m finecode run --log-level="$FINECODE_LOG_LEVEL" run_tests
		shell: bash

		- name: Publish to TestPyPI and verify
		if: runner.os == 'Linux' && github.event_name == 'workflow_dispatch' && inputs.publish_testpypi
		env:
	FINECODE_CONFIG_PUBLISH_AND_VERIFY_ARTIFACT__INIT_REPOSITORY_PROVIDER__REPOSITORIES: '[{"name": "testpypi", "url": "https://test.pypi.org/"}]'
	FINECODE_CONFIG_PUBLISH_AND_VERIFY_ARTIFACT__INIT_REPOSITORY_PROVIDER__REPOSITORIES: '[{"name": "testpypi", "index_url": "https://test.pypi.org/simple/", "upload_url": "https://test.pypi.org/legacy/"}]'
		FINECODE_CONFIG_PUBLISH_AND_VERIFY_ARTIFACT__INIT_REPOSITORY_PROVIDER__CREDENTIALS_BY_REPOSITORY: '{"testpypi": {"username": "${{ secrets.TESTPYPI_USERNAME }}", "password": "${{ secrets.TESTPYPI_PASSWORD }}"}}'
		run: \|
		source .venvs/dev_workspace/${{ matrix.venv_bin }}/activate
Expand All		@@ -154,7 +153,7 @@ jobs:
		- name: Publish to PyPI and verify
		if: runner.os == 'Linux' && startsWith(github.ref, 'refs/tags/')
		env:
	FINECODE_CONFIG_PUBLISH_AND_VERIFY_ARTIFACT__INIT_REPOSITORY_PROVIDER__REPOSITORIES: '[{"name": "pypi", "url": "https://pypi.org/"}]'
	FINECODE_CONFIG_PUBLISH_AND_VERIFY_ARTIFACT__INIT_REPOSITORY_PROVIDER__REPOSITORIES: '[{"name": "pypi", "index_url": "https://pypi.org/simple/", "upload_url": "https://upload.pypi.org/legacy/"}]'
		FINECODE_CONFIG_PUBLISH_AND_VERIFY_ARTIFACT__INIT_REPOSITORY_PROVIDER__CREDENTIALS_BY_REPOSITORY: '{"pypi": {"username": "${{ secrets.PYPI_USERNAME }}", "password": "${{ secrets.PYPI_PASSWORD }}"}}'
		run: \|
		# TODO: make sure git tag exists (for manual trigger)
Expand Down

Navigation Menu

Uh oh!

Uh oh!

Improve concurrency handling, fix part of CI pipeline and more #39

Are you sure you want to change the base?

Uh oh!

Improve concurrency handling, fix part of CI pipeline and more #39

Filter by extension

Uh oh!

Diff view

Diff view

Uh oh!

There are no files selected for viewing

Uh oh!

Uh oh!