🐳 🎉 dockerized version of awscli-local #90

$ docker compose build && docker compose push
[+] Building 4.8s (17/17) FINISHED docker-container:kind_khayyam
 => [awslocal internal] load build definition from Dockerfile 0.0s
 => => transferring dockerfile: 998B 0.0s
 => WARN: InvalidDefaultArgInFrom: Default value for ARG ${BUILDER_IMAGE} results in empty or invalid base image name (line 3) 0.0s
 => WARN: InvalidDefaultArgInFrom: Default value for ARG ${BUILDER_IMAGE} results in empty or invalid base image name (line 16) 0.0s
 => [awslocal internal] load metadata for docker.io/library/python:3.13.2-alpine3.21 3.8s
 => [awslocal auth] library/python:pull token for registry-1.docker.io 0.0s
 => [awslocal internal] load .dockerignore 0.0s
 => => transferring context: 2B 0.0s
 => [awslocal builder 1/6] FROM docker.io/library/python:3.13.2-alpine3.21@sha256:323a717dc4a010fee21e3f1aac738ee10bb485de4e7593ce242b36ee48d6b352 0.0s
 => => resolve docker.io/library/python:3.13.2-alpine3.21@sha256:323a717dc4a010fee21e3f1aac738ee10bb485de4e7593ce242b36ee48d6b352 0.0s
 => [awslocal internal] load build context 0.0s
 => => transferring context: 9.57kB 0.0s
 => CACHED [awslocal service 2/5] RUN apk update && apk add aws-cli==2.22.10-r0 bash zip curl 0.0s
 => CACHED [awslocal service 3/5] WORKDIR /app/site-packages 0.0s
 => CACHED [awslocal builder 2/6] WORKDIR /usr/src/app 0.0s
 => CACHED [awslocal builder 3/6] RUN python3 -m venv /venv 0.0s
 => CACHED [awslocal builder 4/6] RUN pip install --upgrade pip 0.0s
 => CACHED [awslocal builder 5/6] COPY requirements.txt requirements.txt 0.0s
 => CACHED [awslocal builder 6/6] RUN pip install --no-cache-dir -r requirements.txt 0.0s
 => CACHED [awslocal service 4/5] COPY --from=builder /venv /venv 0.0s
 => [awslocal service 5/5] COPY bin/awslocal-docker /venv/bin/awslocal 0.1s
 => [awslocal] exporting to docker image format 0.8s
 => => exporting layers 0.0s
 => => exporting manifest sha256:d9c4ae2efb3cb07909e31189afdf2c449558d56e31494fa5e302ca1e65a5478a 0.0s
 => => exporting config sha256:c1802bab7ea93cdafe600f4603e84c67b1e248fb95c91dcf6149c286e98f438e 0.0s
 => => sending tarball 0.7s
 => [awslocal] importing to docker 0.0s
 => => loading layer 8aff40f8e9b8 3.55kB / 3.55kB 0.0s
[+] Pushing 8/8
 ✔ Pushing marcellodesales/awscli-local:0.22.0.7: 8aff40f8e9b8 Pushed 6.1s 
 ✔ Pushing marcellodesales/awscli-local:0.22.0.7: 8fae2133b79d Layer already exists 2.9s 
 ✔ Pushing marcellodesales/awscli-local:0.22.0.7: 56c23dc9a76d Layer already exists 2.8s 
 ✔ Pushing marcellodesales/awscli-local:0.22.0.7: da53f50a8a75 Layer already exists 2.9s 
 ✔ Pushing marcellodesales/awscli-local:0.22.0.7: 336e6a290569 Layer already exists 2.8s 
 ✔ Pushing marcellodesales/awscli-local:0.22.0.7: 53d9c097c68d Layer already exists 4.6s 
 ✔ Pushing marcellodesales/awscli-local:0.22.0.7: 052b772c7a04 Layer already exists 4.6s 
 ✔ Pushing marcellodesales/awscli-local:0.22.0.7: 08000c18d16d Layer already exists 4.6s 
(.venv)

🏃 Running

Entrypoint already executes awslocal

$ docker compose run awslocal
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
 aws help
 aws <command> help
 aws <command> <subcommand> help
aws: error: the following arguments are required: command

✅ Testing

Created a lambda function as a hello world
Start localstack with a network bridged

🔧 Localstack Deployment

services:
 localstack:
 container_name: localstack
 image: dockerhub.docker.artifactory.viasat.com/localstack/localstack:4.2.0
 ports:
 - "4566:4566" # LocalStack main endpoint
 - "4571:4571" # Internal communication
 user: root
 environment:
 - SERVICES=lambda,s3,cloudwatch,iam,events
 - DEBUG=1
 - DOCKER_HOST=unix:///var/run/docker.sock
 - HOST_TMP_FOLDER=/tmp/localstack
 volumes:
 - /var/run/docker.sock:/var/run/docker.sock
 - ./data:/var/lib/localstack
 - ./s3-hello-world-lambda-function:/lambda-functions
 networks:
 - local-aws
networks:
 local-aws:
 driver: bridge
 name: local-aws

🔧 AWS Lambda Deploy example

Locally create a deploy and run compose service that connects to localstack through the bridge network
Create a .env properties with the required inputs

LAMBDA_LOCAL_DIR=s3-hello-world-lambda-function
LAMBDA_FUNCTION_NAME=s3-hello-world
LAMBDA_TRIGGER_NAME=daily-lambda-trigger
LAMBDA_HANDLER_FUNCTION=lambda_function.handler
LAMBDA_PYTHON_VERSION=python3.11
LOCALSTACK_HOST=localstack:4566

Then, use the deploy

networks:
 #####
 ##### First, docker compose up -d is required to get the local aws network created
 ##### See the healthcheck section below
 #####
 local-aws:
 external: true
services:
 s3-hello-world:
 image: dockerhub.docker.artifactory.viasat.com/marcellodesales/awscli-local:0.22.0.7
 working_dir: /viasat/platform/vionix/aws
 volumes:
 - ${CURRENT_DIR:-.}:/viasat/platform/vionix/aws
 networks:
 - local-aws
# env_file: ./s3-hello-world-lambda-function/local.env
 environment:
 LAMBDA_LOCAL_DIR: ${LAMBDA_LOCAL_DIR}
 LAMBDA_FUNCTION_NAME: ${LAMBDA_FUNCTION_NAME}
 LAMBDA_TRIGGER_NAME: ${LAMBDA_TRIGGER_NAME}
 LAMBDA_HANDLER_FUNCTION: ${LAMBDA_HANDLER_FUNCTION}
 LAMBDA_PYTHON_VERSION: ${LAMBDA_PYTHON_VERSION}
 LOCALSTACK_HOST: ${LOCALSTACK_HOST}
 entrypoint: ["/bin/bash"]
 command:
 - -c
 - |
 env
 echo "Checking if localstack is up and running... curl -I http://localstack:4566/_localstack/health"
 curl -I http://localstack:4566/_localstack/health || exit 1

 echo "Create AWS Role from policies/lambda-role-policy.json"
 awslocal iam create-role \
 --role-name lambda-executor \
 --assume-role-policy-document file://policies/lambda-role-policy.json

 echo "Create the lambda zip file ${LAMBDA_LOCAL_DIR}/function.zip"
 cd ${LAMBDA_LOCAL_DIR}
 zip -r function.zip .

 set -x
 awslocal lambda delete-function --function-name ${LAMBDA_FUNCTION_NAME} 

 echo "Creating the lambda function from ${LAMBDA_LOCAL_DIR}/function.zip"
 awslocal lambda create-function \
 --function-name ${LAMBDA_FUNCTION_NAME} \
 --runtime ${LAMBDA_PYTHON_VERSION} \
 --role arn:aws:iam::000000000000:role/lambda-executor \
 --handler ${LAMBDA_HANDLER_FUNCTION} \
 --zip-file fileb://function.zip

 echo "Creating a cloudwatch event trigger ${LAMBDA_TRIGGER_NAME} schedule ${LAMBDA_SCHEDULE_EXPRESSION}"
 # Optional: Create a CloudWatch event trigger
 awslocal events put-rule \
 --name "${LAMBDA_TRIGGER_NAME}" \
 --schedule-expression "rate(1 day)"

 echo "Add an event target for the function for the CloudWatch example"
 awslocal events put-targets \
 --rule "${LAMBDA_TRIGGER_NAME}" \
 --targets "Id"="1","Arn"="arn:aws:lambda:us-east-1:000000000000:function:${LAMBDA_FUNCTION_NAME}"

 echo "Invoke Lambda function locally"
 awslocal lambda invoke \
 --function-name ${LAMBDA_FUNCTION_NAME} \
 --payload '{"key": "value"}' \
 output.json
 echo "Show output"
 cat output.json

Then, execute the docker-compose-deploy to verify

$ docker compose -f docker-compose-deploy.yaml run s3-hello-world
WARN[0000] The "LAMBDA_SCHEDULE_EXPRESSION" variable is not set. Defaulting to a blank string. 
WARN[0000] Found orphan containers ([localstack vionix-platform-aws-localstack-deploy-s3-hello-world-run-4104d6790002 vionix-platform-aws-localstack-deploy-s3-hello-world-run-5e0c801fa498 vionix-platform-aws-localstack-deploy-s3-hello-world-run-ce3ab7aa1306 vionix-platform-aws-localstack-deploy-s3-hello-world-run-0ac955708867 vionix-platform-aws-localstack-deploy-s3-hello-world-run-42c0598bb72d vionix-platform-aws-localstack-deploy-s3-hello-world-run-263653088c3d vionix-platform-aws-localstack-deploy-s3-hello-world-run-943a0e66b789 vionix-platform-aws-localstack-deploy-s3-hello-world-run-c89c8e03e05b]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. 
[+] Running 1/1
 ✔ s3-hello-world Pulled 5.2s 
PYTHON_SHA256=d984bcc57cd67caab26f7def42e523b1c015bbc5dc07836cf4f0b63fa159eb56
HOSTNAME=f4de90798643
PYTHON_VERSION=3.13.2
LAMBDA_LOCAL_DIR=s3-hello-world-lambda-function
LAMBDA_TRIGGER_NAME=daily-lambda-trigger
LAMBDA_FUNCTION_NAME=s3-hello-world
PWD=/viasat/platform/vionix/aws
HOME=/root
LAMBDA_HANDLER_FUNCTION=lambda_function.handler
GPG_KEY=7169605F62C751356D054A26A821E680E5FA6305
LAMBDA_PYTHON_VERSION=python3.11
TERM=xterm
SHLVL=1
PATH=/usr/bin:/venv/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
LOCALSTACK_HOST=localstack:4566
_=/usr/bin/env
Checking if localstack is up and running... curl -I http://localstack:4566/_localstack/health
HTTP/1.1 200 OK
Server: TwistedWeb/24.3.0
Date: 2025年3月26日 13:50:02 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 2
Create AWS Role from policies/lambda-role-policy.json
An error occurred (EntityAlreadyExists) when calling the CreateRole operation: Role with name lambda-executor already exists.
Create the lambda zip file s3-hello-world-lambda-function/function.zip
updating: requirements.txt (stored 0%)
updating: Dockerfile (deflated 35%)
updating: lambda_function.py (deflated 41%)
updating: output.json (deflated 2%)
+ awslocal lambda delete-function --function-name s3-hello-world
+ echo 'Creating the lambda function from s3-hello-world-lambda-function/function.zip'
Creating the lambda function from s3-hello-world-lambda-function/function.zip
+ awslocal lambda create-function --function-name s3-hello-world --runtime python3.11 --role arn:aws:iam::000000000000:role/lambda-executor --handler lambda_function.handler --zip-file fileb://function.zip
{
 "FunctionName": "s3-hello-world",
 "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:s3-hello-world",
 "Runtime": "python3.11",
 "Role": "arn:aws:iam::000000000000:role/lambda-executor",
 "Handler": "lambda_function.handler",
 "CodeSize": 1324,
 "Description": "",
 "Timeout": 3,
 "MemorySize": 128,
 "LastModified": "2025-03-26T13:50:11.657043+0000",
 "CodeSha256": "BJ8ec5spa6q3gyk8i0Hg28f067v759u26XIzaiUKfl4=",
 "Version": "$LATEST",
 "TracingConfig": {
 "Mode": "PassThrough"
 },
 "RevisionId": "14d05e41-7e0d-41ce-9c6a-98988e55c20e",
 "State": "Pending",
 "StateReason": "The function is being created.",
 "StateReasonCode": "Creating",
 "PackageType": "Zip",
 "Architectures": [
 "x86_64"
 ],
 "EphemeralStorage": {
 "Size": 512
 },
 "SnapStart": {
 "ApplyOn": "None",
 "OptimizationStatus": "Off"
 },
 "RuntimeVersionConfig": {
 "RuntimeVersionArn": "arn:aws:lambda:us-east-1::runtime:8eeff65f6809a3ce81507fe733fe09b835899b99481ba22fd75b5a7338290ec1"
 },
 "LoggingConfig": {
 "LogFormat": "Text",
 "LogGroup": "/aws/lambda/s3-hello-world"
 }
}
+ echo 'Creating a cloudwatch event trigger daily-lambda-trigger schedule '
Creating a cloudwatch event trigger daily-lambda-trigger schedule 
+ awslocal events put-rule --name daily-lambda-trigger --schedule-expression 'rate(1 day)'
{
 "RuleArn": "arn:aws:events:us-east-1:000000000000:rule/daily-lambda-trigger"
}
+ echo 'Add an event target for the function for the CloudWatch example'
Add an event target for the function for the CloudWatch example
+ awslocal events put-targets --rule daily-lambda-trigger --targets Id=1,Arn=arn:aws:lambda:us-east-1:000000000000:function:s3-hello-world
{
 "FailedEntryCount": 0,
 "FailedEntries": []
}
+ echo 'Invoke Lambda function locally'
Invoke Lambda function locally
+ awslocal lambda invoke --function-name s3-hello-world --payload '{"key": "value"}' output.json
{
 "StatusCode": 200,
 "ExecutedVersion": "$LATEST"
}
+ echo 'Show output'
Show output
+ cat output.json
{"statusCode": 200, "body": "Hello from S3 Lambda example called with {'key': 'value'}"}% (.venv)

marcellodesales added 4 commits

March 25, 2025 16:14

@marcellodesales


 🐳 🎉 initial version of the cli dockerized

@marcellodesales


 🐳 ♻️ generate image for linux/amd64 + refact

4c34ee1

* Add specific versions in docker-compose
* Add correct entrypoint execution

@marcellodesales


 🐳 ♻️ add support for aws-cliv2 + awslocal bin

769cfd0

The binary needs to use the venv python installed to avoid
using the one from the system. A new version must remove
this step to avoid confusion and end up having 2 python
installations in the same docker image.
For now, since there are multiple python instances, the
system is pointing to an older version, while venv has
a newer version python 3.13. So, the awslocal-docker
has the proper call as the order of PATH needs to be
respected so that the AWS-cliv2 is loaded beore aws-v1
added by the python bindings.

@marcellodesales


 🐍 📦 add python awscli needed by awslocal

d9bf038

>>> import awscli
Traceback (most recent call last):
 File "<python-input-0>", line 1, in <module>
 import awscli
ModuleNotFoundError: No module named 'awscli'
>>>

@marcellodesales marcellodesales changed the title ~~(削除) 🐳 🎉 initial version of the cli dockerized (削除ここまで)~~ (追記) 🐳 🎉 dockerized version of awscli-local (追記ここまで)

Mar 26, 2025

mattviasat

mattviasat reviewed

Mar 26, 2025

View reviewed changes

bin/awslocal-docker

@@ -0,0 +1,269 @@

#!/venv/bin/python3

Copy link

@mattviasat mattviasat Mar 26, 2025 •

edited

Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like you are copy/pasting the entire file just to change the shebang. That will be difficult to maintain and seems unnecessary.
Since you already change the path with PATH="/venv/bin:$PATH", it should automatically use the virtualenv because /venv/bin/python is the first python that will be found. E.g.

$ python3 -m venv /tmp/venv
$ PATH="/tmp/venv/bin:$PATH" which python
/tmp/venv/bin/python

I would remove bin/awslocal-docker

Copy link

Author

@marcellodesales marcellodesales Mar 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the problem here is to support both aws-cliv2 and the aws-cli python bindings... Couldn't find a way to specify them, but I will try to remove it...

mattviasat

mattviasat reviewed

Mar 26, 2025

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

mattviasat

mattviasat reviewed

Mar 26, 2025

View reviewed changes

Dockerfile

# Add awslocal to site-packages/bin, with priority to awsv2 bin

# As awscli is added for the bindings, v1 will be also added

COPY bin/awslocal-docker /venv/bin/awslocal

Copy link

@mattviasat mattviasat Mar 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need the copy of the file

Suggested change

COPY bin/awslocal-docker /venv/bin/awslocal

COPY bin/awslocal /venv/bin/awslocal

Copy link

Author

@marcellodesales marcellodesales Mar 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the whole problem here is that the image has python 3.13 installed with the command venv creation... I will remove that step and see if it's possible to keep a single installation and, as a result, decrease the image size.

@marcellodesales @mattviasat


 🐳 add --no-cache-dir for pip upgrade Dockerfile

c3c6044

Co-authored-by: mattviasat <129442476+mattviasat@users.noreply.github.com>

@mattviasat

Copy link

mattviasat commented Apr 1, 2025

The other thing I would say is that it might be easier to just call awslocal inside the existing localstack image, rather than creating a new image that just containers awslocal.

@marcellodesales

Copy link

Author

marcellodesales commented Apr 2, 2025

The other thing I would say is that it might be easier to just call awslocal inside the existing localstack image, rather than creating a new image that just containers awslocal.

Thank you for all your suggestions... I just need a smaller image to account for SBOM, security, etc... This is specific for a cloudNative platform and I don't need to pull the entire localstack image for the use cases that I have...

@mattviasat

Copy link

mattviasat commented Apr 7, 2025

Thank you for all your suggestions... I just need a smaller image to account for SBOM, security, etc... This is specific for a cloudNative platform and I don't need to pull the entire localstack image for the use cases that I have...

I think there is probably a simpler way that results in a smaller image. You could use python/python3.x image and just

python3 -m venv /venv/
/venv/bin/python3 -m pip install --no-cache-dir awscli-local
/venv/bin/awslocal help

@marcellodesales


 🆕 :earth: Add terraform-local implementation tflocal

da44848

@mattviasat

Copy link

mattviasat commented May 20, 2025

The other thing I would say is that it might be easier to just call awslocal inside the existing localstack image, rather than creating a new image that just containers awslocal.

Thank you for all your suggestions... I just need a smaller image to account for SBOM, security, etc... This is specific for a cloudNative platform and I don't need to pull the entire localstack image for the use cases that I have...

If you are running localstack in one docker container, than i believe you should already have the image stored locally.
If you need to interact with an existing localstack container you can either:

exec into the running container to run a command. awslocal is installed in localstack/localstack and you know it will be the right version.
Use init hooks to create as stack when the containers comes up https://docs.localstack.cloud/references/init-hooks/#lifecycle-stages-and-hooks
or if you insist on running a different container, docker run -e awslocal localstack/localstack <my command here>
you would need to set up the ports and allow it to talk to the original localstack/localstack container, but it should work.

Labels

None yet

2 participants

@marcellodesales @mattviasat

Uh oh!

🐳 🎉 dockerized version of awscli-local #90

Are you sure you want to change the base?

🐳 🎉 dockerized version of awscli-local #90

Uh oh!

Conversation

@marcellodesales marcellodesales commented Mar 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🎉 New Version

🏗️ Build

🏃 Running

✅ Testing

🔧 Localstack Deployment

🔧 AWS Lambda Deploy example

Uh oh!

@mattviasat mattviasat Mar 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

@marcellodesales marcellodesales Mar 27, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

@mattviasat mattviasat Mar 26, 2025

Choose a reason for hiding this comment

Uh oh!

@marcellodesales marcellodesales Mar 27, 2025

Choose a reason for hiding this comment

Uh oh!

mattviasat commented Apr 1, 2025

Uh oh!

marcellodesales commented Apr 2, 2025

Uh oh!

mattviasat commented Apr 7, 2025

Uh oh!

mattviasat commented May 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

@marcellodesales marcellodesales commented Mar 25, 2025 •

edited

Loading

@mattviasat mattviasat Mar 26, 2025 •

edited

Loading