Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Serverless directory listing frontend for Termux mirrors on AWS Lambda@Edge.

License

Notifications You must be signed in to change notification settings

bitnarri/DirectoryListing

Repository files navigation

Termux Mirror Index - Lambda@Edge Frontend

πŸš€ Overview

  • Objective: Provides a high-performance, flicker-free web UI for browsing a Termux APT mirror repository.
  • Core Technologies: AWS Lambda@Edge (Node.js/Python), Vue.js (App Shell), DynamoDB.
  • Key Feature: Delivers faster directory listings than the S3 ListObjects API by using DynamoDB metadata.
  • Build: Supports automated minification and deployment via AWS CodeBuild/CodePipeline.

✨ Key Features

  • Fast Listings: Reads from DynamoDB Global Tables for geo-proximate metadata queries.
  • Flicker-Free UI: Combines a server-rendered skeleton with a client-side Vue.js application.
  • Optimized Loading:
    • CSS: Critical CSS (layout, fonts) is inlined; non-critical CSS (hover, focus) is externalized.
    • JavaScript: The main client script (client.min.js) is externalized and loaded with defer.
    • Fonts: Optimized with preload, local(), and font-display: fallback.
  • Automated Build: build.mjs script handles minification, configuration injection, and packaging.
  • Dual Runtimes: Concurrently builds L@E functions for both Node.js and Python runtimes.

πŸ“‚ Project Structure

/
|-- src/ # Source code
| |-- lambda_template.js # Node.js L@E template (pre-injection)
| |-- lambda_template.py # Python L@E template (pre-injection)
| |-- client.js # Client-side Vue app source
| |-- style.css # Inlined CSS source (layout, fonts)
| |-- mirror.css # External CSS source (hover, focus, etc.)
|
|-- .dockerignore # Docker build ignore file
|-- .gitignore # Git ignore file
|-- build.mjs # Node.js-based build script
|-- buildspec.yml # AWS CodeBuild specification
|-- config.json # (Ignored) Build configuration values
|-- Dockerfile # Dockerfile for isolated builds
|-- LICENSE.md # Project license
|-- package.json # Node.js dependencies and build scripts
|-- package-lock.json # Node.js dependency lock file
|-- requirements.txt # Python build dependencies (python-minifier)
|-- sample_config.json # Example configuration file
|
|-- dist/ # (Ignored) Build artifacts
|-- client.min.js # Minified client Vue app
|-- mirror.min.css # Minified external CSS
|-- index.mjs # Minified Node.js L@E function
|-- lambda_handler.py # Minified Python L@E function
|-- index_js.zip # Node.js deployment package
|-- lambda_function_py.zip # Python deployment package

πŸ› οΈ Build Process

Prerequisites
  • Docker: (Recommended) To build in a containerized environment.
  • Or (Manual Setup):
  • Node.js (v22+)
  • npm
  • Python (v3.9+)
  • pip

Configuration

  • Create config.json: Copy sample_config.json to config.json.

  • Edit config.json: Fill in your AWS resources (table name, regions).

  • Security: config.json is ignored by Git.

    • Local Builds: The build.mjs script reads config.json.
    • AWS CodeBuild: Set environment variables (e.g., TABLE_NAME, REGION_PRIMARY) to override local file settings.
  • CodeBuild Environment Variable

Name Value (Example) Type
S3_DEPLOY_BUCKET my-lambda-deploy-bucket-us-east-1 PLAINTEXT
S3_STATIC_BUCKET S3-Static-Assets PLAINTEXT
TABLE_NAME DynamoDBGlobalTableName PLAINTEXT
REGION_PRIMARY us-west-1 PLAINTEXT
REGION_SECONDARY eu-north-1 PLAINTEXT
CACHE_AGE 600 PLAINTEXT

Install dependencies

# Install Node.js build dependencies (terser, csso)
npm install
# Install Python build dependencies (pyminify)
pip install -r requirements.txt

Run Build

docker build --output ./dist .
# Option 2: Using Local Environment
# Requires all prerequisites to be installed locally
npm run build

Build Artifacts (in dist directory)

  • index.mjs (Minified Node.js L@E function)
  • lambda_function.py (Minified Python L@E function)
  • client.min.js (Minified client Vue app)
  • mirror.min.css (Minified external CSS)
  • (CodeBuild) index_mjs.zip (Node.js deployment package)
  • (CodeBuild) lambda_function_py.zip (Python deployment package)

πŸ—„οΈ (Note) Backend Data Sync

This L@E frontend depends on metadata in DynamoDB and file objects in S3. This data is kept in sync with an official Termux mirror via a periodic backend process.

Data Sync Architecture Example:

  • Automation: AWS Systems Manager (SSM) Automation & State Manager
  • Environment: An on-demand EC2 instance
  • Trigger: SSM State Manager invokes an Automation document on an hourly schedule (e.g., cron(0 0 */1 * * ? *)).

Automation flow:

  1. Start Instance: SSM Automation starts EC2 instance.
  2. Remote Sync: An rsync process (see here) syncs the official mirror to the local EC2 directory.
  3. DynamoDB Sync: A Python script parses the rsync output to perform an incremental update (add/modify/delete) on the DynamoDB Global Table.
  4. S3 Sync: aws s3 sync ... --delete syncs the local directory to the primary S3 origin bucket.
  5. Cache Invalidation: aws cloudfront create-invalidation ... purges the CloudFront cache for /*.
  6. Stop instance: SSM Automation stops the EC2 instance to save costs.

High-Availability Architecture

  • Database: DynamoDB Global Tables allow the L@E function to read from the nearest AWS region (e.g., REGION_PRIMARY or REGION_SECONDARY).
  • Storage: S3 Cross-Region Replication (CRR) combined with a CloudFront Origin Group enables automatic failover to a secondary S3 bucket if the primary origin fails.

πŸš€ Deployment Strategy

Deployment consists of two independent parts: static assets and the Lambda@Edge function.

1, Deploy Static Assets (S3 + CloudFront Behaviors)

These are files loaded by the browser, not the L@E function (e.g., client.min.js, mirror.min.css, roboto.woff2). 1. Create S3 bucket (for static assets): Create S3 bucket such as my-static-assets. (Equal to CodeBuild environment variable S3_STATIC_BUCKET)

  1. S3 bucket: Create an S3 bucket for static assets (e.g., S3-Static-Assets, corresponding to S3_STATIC_BUCKET).
  2. Upload: After running npm run build, upload dist/client.min.js, dist/mirror.min.css, and other assets (like roboto.woff2) to this bucket.
  3. Add CloudFront Origin: Add this S3 bucket as a new Origin in your CloudFront distribution (e.g., S3-Static-Assets).
  4. Add CloudFront Cache Behaviors:
    • Go to the Behaviors tab and Create Behavior.
    • Path Pattern: /client.min.js
    • Origin: Select the S3-Static-Assets origin
    • Cache Policy: Managed-CachingOptimized (or a custom policy with a long TTL).
    • Viewer Protocol Policy: Redirect HTTP to HTTPS
    • Allowed HTTP Methods: GET, HEAD.
    • Repeat this process to create separate behaviors for /mirror.min.css, /roboto.woff2, etc.

2. Deploy Lambda@Edge Function (L@E + CloudFront Default Behavior)

The L@E function handles Default (*) requests to generate the dynamic HTML.

  1. S3 bucket (Lambda): Create an S3 bucket in the us-east-1 (N. Virginia) region to store the L@E function's ZIP file (e.g., my-lambda-deploy-bucket-us-east-1). This corresponds to S3_DEPLOY_BUCKET.
  2. Upload ZIP: After npm run build, upload the L@E package (dist/lambda_function_py.zip or dist/index_js.zip) to this us-east-1 bucket.
  3. Update Lambda function (in us-east-1):
    • In the AWS Lambda console, switch to the us-east-1 region.
    • Architecture: Must be x86_64.
    • Runtime: Node.js 22.x or Python 3.13
    • Update Code: Use the "Upload from Amazon S3 location" option and provide the S3 URL to the .zip file you just uploaded.
    • Publish Version: Click Actions > Publish new version. (e.g., version :17). CloudFront cannot use $LATEST.
    • Associate with CloudFront:
      • In the CloudFront console, Edit your distribution's Default (*) Behavior.
      • Under Function associations > Viewer Request, Select the new Lambda function ARN with the specific version number you just published (e.g., ...:my-lambda-function:17).

Contribution

This project is not currently accepting contributions.


License

AltStyle γ«γ‚ˆγ£γ¦ε€‰ζ›γ•γ‚ŒγŸγƒšγƒΌγ‚Έ (->γ‚ͺγƒͺγ‚ΈγƒŠγƒ«) /