This code should help to jump start PySpark with Anaconda on AWS using Terraform.

This project is no longer actively maintained and has been archived on Jan 7, 2025.

1. Install Terraform on Mac: brew install terraform
2. Adjust the scripts (bootstrap_actions.sh and pyspark_quick_setup.sh) in scripts if necessary
3. Set parameters in terraform.tfvars
4. Start cluster:

terraform init
terraform apply

5. Destroy cluster:

terraform destroy

• Configure AWS on your local machine: aws configure
• AWS instance cost for eu-central-1

• Dat Tran, github: datitran

See LICENSE for details.