hardware.dark.fi webshop

website for the Dark.Fi's hardware webshop.

python-based website integrating a webshop (Stripe, NOWPayments) and git-based workflow to update website content:

• python 3.12
• FastAPI
• custom parser for md + yaml documents

development

settings.toml

rename settings.sample.toml to settings.toml and fill out fields:

• git_repo: set the path to the content folder (it can also be a relative path)
• document_match: list of file extension to match
• document_exclude: list of files to exclude
• block_types: set necessary block types (for the content document); for starting, the default ones should be enough; the website's parser read from this list of block types to validate if a given block is "allowed"
• local_db.filepath: path to local DB, eg file that contains the orders
• order_upload.filepath: path to folder where new XLSX file order are exported to (these files are used to upload a shipping order on the Swiss Post website)
• currency.default: default currency for Stripe
• currency.now_payments: default currency for NOWPayments

.env

copy the file env.sample and rename it .env. then update its content accordingly:

• ENV: set if environment is development or production (it affects some of the code logic)

• CACHE: set enable or disable to cache CSS, JS and font files

• CSRF_SECRET_KEY: generate a secret (a random string of characters) for the CSRF protection, see https://stackoverflow.com/a/54433731; for example:

  python
  >>> import secrets
  >>> secrets.token_hex(16)
  

• SESSION_SECRET_KEY: see item above for how to generate a random string

• STRIPE_SECRET_KEY: Stripe API key for production

• STRIPE_TEST_SECRET_KEY: Stripe API key for dev environment. Put here your TEST API key. Beware that data in test environment differs from production

• STRIPE_ENDPOINT_SECRET: Stripe webhook secret

  • important: use keys generated from Test mode while working on the website

• NOWPAYMENTS_API_KEY: NOWPayments API key

• NOWPAYMENTS_IPN: NOWPayments IPN

• NOWPAYMENTS_SANDBOX_API_KEY: NOWPayments Sandbox API key

• NOWPAYMENTS_SANDBOX_IPN: NOWPayments Sandbox IPN

• MAIL_USERNAME: email address

• MAIL_PASSWORD: email password

• MAIL_FROM: email sender address

• MAIL_FROM_NAME: email sender name

• MAIL_PORT: email port

• MAIL_SERVER: email server

• MAIL_STARTTLS: True|False

• MAIL_SSL_TLS: True|False

• MAIL_USE_CREDENTIALS: True|False

• MAIL_VALIDATE_CERTS: True|False

• CPU_CORE: set computer's number of CPU cores, on Linux run nproc --all; this is used by the build_image_cache.py command to generate images of various sizes for the frontend.

setup Python

• python 3.12
• make virtual environment: python -m venv env
• activate virtual environment: source env/bin/activate
• install packages with pip install -r requirements.txt

packages

whenever installing a new package, save it to requirements.txt by doing:

pip freeze > requirements.txt

server

to run the local web-server, type:

./dev-server.sh 

to run the production server, do:

./server.sh

create image cache

we display images in various sizes so that the browser can pick the best image in terms of quality / size ratio — for instance by not downloading a high-res image prepared for a big screen if the browser request comes from a small screen device (eg. a phone). to deal with this, there a CLI script called build_image_cache.py at the root of the repo. it offers two commands:

before generating images make sure that the cache folder exists in static/images, otherwise images won't get written

• build-all-images: this build all images across the content repo. if an image exists already in the ./static/images/cache/ folder, it will skip it. you can either remove it from the cache folder or rename the image to a new name to force-recreate it. run this command like so: python build_image_cache.py build-all-images '<path/to/content-repo>
• build-images: this command is used in the git post-receive hook in the git bare repo (on the VPS). it checks if the last pushed commit contains also image files, and if so it creates the necessary sizes of each image for the website. check the post-receive files for the details

example:

python build_image_cache.py build-all-images <path/to/content-folder>

test stripe webhook (locally)

unlike with NOWPayments, we can locally test that the Stripe webhook works as intended — for instance that it triggers the order status update, the product inventory update, and the email confirmation.

to do so:

• first install the Stripe CLI
• then open a new shell and run:

stripe listen --load-from-webhooks-api --forward-to http://127.0.0.1:5014

where http://127.0.0.1:5014 is your local server port (which is the one set in dev-server.sh). this command listen to our "online" Stripe webhook for any event, and redirects them all to our local environment.

when you run Stripe CLI locally, it might print the webhook signature valid for our local environment — make sure to add it or replace it to the field STRIPE_TEST_ENDPOINT_SECRET in the .env file.

some refs:

• Receive Stripe events in your webhook endpoint
• Resolve webhook signature verification errors

aftewards, run whatever you need to test.

server setup

installation

• make new unix user from root (sudo adduser)
• add ssh pubkey to new user's home under ~/.ssh/authorized_keys, set permissions (https://superuser.com/a/925859)
• add newly created user under sudo group (sudo adduser <user> sudo)
• update /etc/ssh/ssh_config by changing or adding values to the following:

# disable ssh login for root user
PermitRootLogin no
ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM no
# remove password-based login for any user
AuthenticationMethods publickey
PubkeyAuthentication yes
# limit ssh access to specified users (IMPORTANT: this will block any other previously authorized user to login via ssh)
AllowUsers <user-1> <user-2>
# disable empty password
PermitEmptyPasswords no
# disable X11Forwarding
X11Forwarding no

then restart sshd with sudo systemctl restart sshd

• install ufw and enable SSH + Nginx
• install git
• install pyenv (https://bgasparotto.com/install-pyenv-ubuntu-debian):
  • sudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \ libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \ xz-utils tk-dev libffi-dev liblzma-dev
  • curl -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash
  • pyenv install 3.12
• make git bare repo for source code and add post-receive hook:

#!/bin/bash
GIT_WORK_TREE=/var/www/hardware.darkfi git checkout -f main

• make hook executable with chmod +x hooks/post-receive
• make folder for website at /var/www/hardware.darkfi; create a new git remote pointing to <user>@<server-ip>:www-code.git and then push to this remote (it should copy all files over to /var/www/darkfi-webshop)
• install nginx and copy sample file to /etc/nginx/site-available/; enable nginx site by making a symbolic link of it to /etc/nginx/site-enabled/: sudo ln -s /etc/nginx/site-available/hardware.darkfi.nginx /etc/nginx/site-enabled/hardware.darkfi.nginx
• make new file /etc/systemd/system/darkfi.service and copy sample
• sudo systemctl enable darkfi, sudo systemctl start darkfi

Tests

Tests are using pytest and playwright for browser automation. To run the tests make sure that the test data is uploaded to your instance and the dev server is running:

pytest tests/ will run all the tests from the folder. To run specific tests you need to point pytest path to your test set.

To test email functionality make sure to include MAIL_TO_TEST variable in .env. This will be an email that will be getting test messages.