Forgejo Runner Infrastructure with Code Signing

A Terraform configuration that deploys a secure, auditable build and signing infrastructure for Forgejo Actions using AWS.

Overview

This infrastructure provides a unified Forgejo runner with dual signing capabilities:

1. Code Signing: Signs executable code and kernel modules using AWS KMS with RSA-4096 keys
2. RPM/Package Signing: Signs RPM packages using GPG backed by AWS KMS

The runner operates in a public subnet with no inbound access, comprehensive logging, and public transparency logs.

Architecture

┌─────────────────────────────────────────────────────────────┐
│ AWS VPC │
│ │
│ ┌──────────────────────────────────────────────────┐ │
│ │ Public Subnet │ │
│ │ │ │
│ │ ┌──────────────────┐ │ │
│ │ │ Unified Runner │ │ │
│ │ │ │ │ │
│ │ │ - Code Signing │ │ │
│ │ │ - RPM Signing │ │ │
│ │ │ - KMS Access │ │ │
│ │ └────────┬─────────┘ │ │
│ │ │ │ │
│ │ ┌────────▼─────────┐ │ │
│ │ │ Internet Gateway │ │ │
│ │ └────────┬─────────┘ │ │
│ └─────────────────────┼──────────────────────────────┘ │
│ │ │
└────────────────────────┼─────────────────────────────────────┘
 │
 ┌─────────┴─────────┐
 │ │
 ┌────▼─────┐ ┌────▼────────┐
 │ Public │ │ AWS │
 │ Internet │ │ Services │
 └──────────┘ │ (via VPC │
 │ Endpoints) │
 └─────────────┘
 │
 ├─ KMS (2x Signing Keys)
 ├─ Secrets Manager
 ├─ S3 (Public Logs)
 └─ CloudWatch Logs

Note: Instances are in a public subnet with security groups that block all inbound traffic. Outbound connectivity is via Internet Gateway.

Features

Security

• No Inbound Access: Security groups block all inbound connections
• KMS-Based Code Signing: Private keys never leave AWS KMS
• Encrypted Storage: All EBS volumes encrypted at rest
• IMDSv2 Enforced: Metadata security hardened
• IAM Least Privilege: Minimal permissions per instance
• VPC Endpoints: Secure access to AWS services and reduced data transfer costs

Transparency & Auditability

• Public Transparency Logs: All signing operations logged to public S3 bucket
• CloudTrail Integration: Complete audit trail of KMS operations
• CloudWatch Logging: Real-time monitoring of signing activities
• S3 Access Logs: Track transparency log access
• Versioned Logs: Immutable record with S3 versioning

Automation & Build Environment

• Terraform IaC: Reproducible infrastructure as code
• Automated Registration: Runners auto-register with Forgejo
• User Data Scripts: Complete instance configuration
• Systems Manager Access: SSH-less access for management
• Rootless Docker: Containerized builds without root privileges
• Comprehensive Tooling: Kernel build tools, RPM packaging, signing utilities

Quick Start

Option 1: Manual Deployment

1. Review Prerequisites

   cat PREREQUISITES.md
   

2. Configure Deployment

   cp terraform.tfvars.example terraform.tfvars
   # Edit terraform.tfvars with your values
   

3. Deploy Infrastructure

   terraform init
   terraform plan
   terraform apply
   

4. Complete Setup

   # Follow post-deployment steps in DEPLOYMENT.md
   cat DEPLOYMENT.md
   

Option 2: CI/CD Deployment with Forgejo Actions

Deploy automatically using Forgejo Actions:

1. Review Prerequisites

   cat CICD_DEPLOYMENT.md
   

2. Setup Backend and Authentication

   # Create S3 backend for Terraform state
   ./setup-terraform-backend.sh my-terraform-state-bucket
   # Choose authentication method (OIDC recommended or AWS credentials)
   # See CICD_DEPLOYMENT.md for detailed steps
   

3. Push to Repository

   git push origin main
   # Workflow automatically runs terraform plan and apply
   

See CICD_DEPLOYMENT.md for complete CI/CD setup instructions.

Security Hardening: Disable CA Key Access

After initial deployment, the CA (Certificate Authority) KMS key is accessible to the runner instance for certificate generation. Once certificates are created and uploaded to S3, you should disable CA key access as a security best practice.

The CA key is only needed during initial setup. Normal signing operations only require the code signing and GPG signing keys.

Workflow

1. Deploy with CA access enabled (default):

   terraform apply
   # ca_setup_mode = true (default)
   

2. Wait for certificates to be generated:

   # Check that certificates exist in S3
   aws s3 ls s3://YOUR-TRANSPARENCY-BUCKET/certs/
   # Should show: ca-cert.pem and code-signing-cert.pem
   

3. Disable CA key access:

   # In terraform.tfvars, add:
   echo 'ca_setup_mode = false' >> terraform.tfvars
   # Or for CI/CD, set the secret:
   # TF_VAR_ca_setup_mode = false
   

4. Apply the change:

   terraform apply
   # IAM policy will be updated to remove CA key access
   # Instance will be replaced with new permissions
   

Benefits:

• Limits blast radius if instance is compromised
• CA key can only be used to sign new code signing certificates, not for daily operations
• Follows principle of least privilege

Note: If you need to regenerate certificates (e.g., for rotation), temporarily set ca_setup_mode = true, apply, then set it back to false.

Documentation

• PREREQUISITES.md : Requirements and pre-deployment setup
• DEPLOYMENT.md : Complete deployment and operational guide
• CICD_DEPLOYMENT.md : CI/CD deployment with Forgejo Actions
• SECRETS_REFERENCE.md : Complete reference for CI/CD secrets
• ARCHITECTURE.md : Detailed technical architecture
• VERIFICATION.md : How to verify signatures
• SECURITY.md : Security best practices and hardening
• QUICK_REFERENCE.md : Quick command reference

Infrastructure Components

Network Layer

• VPC with public subnet
• Internet Gateway for outbound connectivity
• VPC Endpoints for AWS services (S3, KMS, Secrets Manager, CloudWatch)
• Security groups with no inbound access

Compute Layer

• EC2 instance (Fedora Cloud Base)
  • Unified runner with KMS access for both code and GPG signing
  • Handles kernel module signing (pesign) and RPM signing (GPG)
  • Rootless Docker support for containerized builds
• Systems Manager for SSH-less access
• Automated forgejo-runner installation and registration

Security Layer

• KMS asymmetric key (RSA-4096) for code signing
• IAM roles with least privilege policies
• Secrets Manager for Forgejo registration token
• CloudTrail for KMS API logging

Storage & Logging

• S3 bucket for public transparency logs (with public read access)
• S3 bucket for build artifacts (static website hosting)
• S3 bucket for CloudTrail logs
• S3 access logging bucket
• CloudWatch log groups for KMS operations
• EBS volumes with encryption

Signing Operations

Code Signing (KMS)

The runner uses AWS KMS to sign binaries and kernel modules:

/usr/local/bin/kms-sign input-file output-signature

Features:

• RSA-4096 signatures
• SHA-256 hashing
• Automatic transparency logging
• Public key available via KMS API
• Works with pesign for kernel module signing

RPM Signing (GPG with KMS)

The runner uses GPG backed by AWS KMS for package signing:

/usr/local/bin/rpm-sign package.rpm

Features:

• GPG private key backed by AWS KMS (never leaves HSM)
• GPG public key automatically stored in S3 for distribution
• Automatic transparency logging
• Standard RPM signature format
• PKCS#11 integration via gnupg-pkcs11-scd
• Key persistence: existing GPG key fetched from S3 on instance recreation

Public Key Distribution

Both signing keys have their public components available for verification:

Code Signing Certificate (x509):

# Download from S3
curl https://YOUR-BUCKET.s3.REGION.amazonaws.com/certs/code-signing-cert.pem -o code-signing-cert.pem
# Or get directly from KMS
aws kms get-public-key --key-id YOUR-KEY-ID --query PublicKey --output text | base64 -d > public-key.der
openssl rsa -pubin -inform DER -in public-key.der -outform PEM -out public-key.pem

GPG Public Key:

# Download from S3
curl https://YOUR-BUCKET.s3.REGION.amazonaws.com/gpg-keys/gpg-signing-key.gpg -o gpg-signing-key.gpg
# Import for RPM verification
rpm --import gpg-signing-key.gpg
# Or import into GPG keyring
gpg --import gpg-signing-key.gpg

Container Support (Rootless Docker)

The runner includes rootless Docker for secure containerized builds:

Features:

• Runs as non-root user (forgejo-runner)
• No elevated privileges required
• Compatible with Forgejo Actions container workflows
• Pre-configured with proper environment variables

Using in workflows:

name:Container Buildon:[push]jobs:build:runs-on:self-hostedsteps:- uses:actions/checkout@v3- name:Build containerrun:docker build -t myimage:latest .- name:Run containerrun:docker run --rm myimage:latest

Docker is available at:

• Socket: unix:///home/forgejo-runner/.docker/run/docker.sock
• User service: systemctl --user status docker (as forgejo-runner)

Build Artifacts Hosting

A dedicated S3 bucket is configured for hosting build artifacts as a static website:

Features:

• Public read access for artifact downloads
• Static website hosting enabled
• Runner has write permissions to upload artifacts
• CORS enabled for web access

Using in workflows:

name:Build and Deployon:[push]jobs:build:runs-on:self-hostedsteps:- uses:actions/checkout@v3- name:Buildrun:make build- name:Upload to S3run:| aws s3 cp ./dist/ s3://YOUR-ARTIFACTS-BUCKET/builds/${{ github.sha }}/ --recursive
 # Create/update latest link
 echo "${{ github.sha }}" > latest.txt
 aws s3 cp latest.txt s3://YOUR-ARTIFACTS-BUCKET/builds/latest.txt

Accessing artifacts:

• Direct URL: https://YOUR-BUCKET.s3.REGION.amazonaws.com/builds/COMMIT_SHA/file.rpm
• Website URL: http://YOUR-BUCKET.s3-website-REGION.amazonaws.com/builds/COMMIT_SHA/

Transparency Logs

All signing operations are automatically logged to a public S3 bucket:

s3://your-bucket/code-signing/YYYY/MM/DD/timestamp-hash.json
s3://your-bucket/rpm-signing/YYYY/MM/DD/timestamp-hash.json

Each log entry includes:

• Timestamp (UTC)
• Operation type
• Key identifier (KMS Key ID or GPG fingerprint)
• Input file hash (SHA-256)
• Output file reference
• Instance ID

These logs are publicly readable and immutable (S3 versioning enabled).

Example Workflows

Code Signing Workflow

name:Build and Signon:[push]jobs:sign:runs-on:code-signingsteps:- uses:actions/checkout@v3- name:Buildrun:make build- name:Signrun:/usr/local/bin/kms-sign ./binary ./binary.sig

RPM Signing Workflow

name:Build and Sign RPMon:[push]jobs:sign:runs-on:self-hostedsteps:- uses:actions/checkout@v3- name:Build RPMrun:rpmbuild -ba spec.file- name:Sign RPMrun:/usr/local/bin/rpm-sign ~/rpmbuild/RPMS/x86_64/*.rpm

Cost Estimate

Monthly costs (us-east-1, approximate):

Costs can be reduced by:

• Using smaller instance types (t3.small) - save 15ドル/month
• Implementing Reserved Instances - save 30-40%
• S3 lifecycle policies for old logs

Monitoring

Access logs and metrics:

# View KMS signing logs
aws logs tail /aws/kms/forgejo-runner-code-signing --follow
# List transparency logs
aws s3 ls s3://your-bucket/code-signing/ --recursive
# Check runner status
aws ssm start-session --target <instance-id>
sudo systemctl status forgejo-runner

Security Considerations

• KMS private keys never leave AWS
• No inbound access (use Systems Manager for management)
• All operations logged and auditable
• Public transparency logs enable third-party verification
• Least privilege IAM policies
• Encrypted storage and transmission
• Security groups deny all inbound traffic

Updating

Updating Runner Version

By default, the infrastructure installs the latest version of forgejo-runner. To update to a newer version, simply recreate the instances:

terraform apply -replace=aws_instance.code_signing_runner

To pin to a specific version, edit terraform.tfvars:

forgejo_runner_version = "12.8.2"

Then apply:

terraform apply

Note: This will recreate the instance. GPG keys are KMS-backed and will be automatically reconfigured.

Troubleshooting

Common issues and solutions are documented in DEPLOYMENT.md.

Quick diagnostics:

# Check instance status
terraform output code_signing_instance_id | xargs -I {} \
 aws ec2 describe-instance-status --instance-ids {}
# View instance console output
terraform output code_signing_instance_id | xargs -I {} \
 aws ec2 get-console-output --instance-id {}
# Check runner service
aws ssm start-session --target $(terraform output -raw code_signing_instance_id)
sudo journalctl -u forgejo-runner -f

Maintenance

Regular maintenance tasks:

• Review transparency logs weekly
• Update forgejo-runner monthly (automatically uses latest by default)
• Review IAM policies quarterly
• Monitor CloudWatch for anomalies
• Review KMS key policies annually
• Test signature verification periodically

Teardown

To destroy all infrastructure:

terraform destroy

See DEPLOYMENT.md for detailed teardown procedures.

Files

.
├── README.md # This file
├── PREREQUISITES.md # Setup requirements
├── DEPLOYMENT.md # Deployment guide
├── CICD_DEPLOYMENT.md # CI/CD deployment guide
├── SECRETS_REFERENCE.md # CI/CD secrets reference
├── ARCHITECTURE.md # Technical architecture details
├── VERIFICATION.md # Signature verification guide
├── SECURITY.md # Security best practices
├── TRANSPARENCY_LOG.md # Transparency log details
├── QUICK_REFERENCE.md # Quick command reference
├── terraform.tfvars.example # Example configuration
├── versions.tf # Terraform and provider versions
├── variables.tf # Input variables
├── outputs.tf # Output values
├── backend.tf # S3 backend configuration
├── vpc.tf # VPC and networking
├── iam.tf # IAM roles and policies
├── kms.tf # KMS key and CloudTrail
├── secrets.tf # Secrets Manager
├── s3.tf # S3 buckets
├── ec2.tf # EC2 instances
├── aws-oidc-setup.tf # OIDC provider setup for CI/CD
├── setup-terraform-backend.sh # Script to create S3 backend
├── user-data-code-signing.sh # Code signing instance setup
├── user-data-rpm-signing.sh # RPM signing instance setup
├── Makefile # Convenience commands
├── .gitignore # Git ignore patterns
└── .forgejo/
 └── workflows/
 ├── terraform.yml # Main CI/CD workflow (OIDC)
 ├── terraform-simple.yml # Simple workflow (AWS credentials)
 └── oidc-setup.yml # OIDC provider setup workflow

License

This configuration is provided as-is for use with Forgejo Actions. Adapt as needed for your requirements.

Support

For issues:

• Review documentation in PREREQUISITES.md and DEPLOYMENT.md
• Check AWS CloudWatch logs
• Consult Forgejo documentation: https://forgejo.org/docs/
• Review Terraform AWS provider docs: https://registry.terraform.io/providers/hashicorp/aws/

Contributing

This is a reference implementation. Feel free to customize for your specific needs:

• Adjust instance types for your workload
• Add additional signing methods
• Implement custom transparency log formats
• Add monitoring and alerting
• Integrate with your CI/CD pipelines