SciOp

collecting at-risk data in torrent rss feeds

Running a SciOp Instance

These instructions are for executing the sciop tool.

Installing Dependencies

With pip

Make a python virtual environment and activate it:

python -m venv ~/.envs/sciop
. ~/.envs/sciop/bin/activate

Make sure pip is reasonably new:

pip install --upgrade pip

Install dependencies and then sciop itself in-place as an editable requirement:

pip install -e .

With PDM

Install dependencies, automatically creating a virtual environment by default

pdm install

Running SciOp

Create a configuration starting from the sample:

cp .env.sample .env
$EDITOR .env

Two fields must be set:

• SCIOP_ENV: one of dev, test or prod. One should only make a sciop instance publicly available in prod mode. dev mode is for local development purposes, as is test. You must ensure that you do not re-use the same db between a dev/test and prod instance, e.g. if the SCIOP_DB location is explicitly set to something other than the defaults.
• SCIOP_SECRET_KEY: must be a securely-generated random hex value. A key can be generated with openssl rand -hex 32.

With pip

sciop

With pdm

pdm run start

Contributing

We use pdm to build and interact with the code in this repo for contributions. This workflow is slightly different than simply executing the tool.

Setup Development Environment

pdm can be installed at the top level, without entering a virtual environment:

pip install pdm

However, it is also possible to install pdm within an existing venv.

Then, pdm can install dependencies, implicitly creating a venv if not already within one:

pdm install

Testing Your Changes

To run the code within the worktree:

pdm run start

To fix formatting and imports:

pdm run format

To run lint:

pdm run lint

To run automated testing:

pdm run test

Changes can then be submitted as a pull request against this repository on Codeberg.

DB Migrations

Any changes to the database must have corresponding migrations.

Migrations can be autogenerated with alembic

alembic -c ./src/sciop/migrations/alembic.ini revision -m "{migration-slug}" --autogenerate

Where {migration-slug} is some description of the changes made in the migration.

The migration generator compares the current state of the ORM models to the current state of the database, so the database must be equal to the state at the last migration.

To get a clean database, before you generate the migration, remove your development database and create a new one using alembic

alembic -c ./src/sciop/migrations/alembic.ini upgrade head

Then generate the migration with the command above.

Migrations can be tested with pytest

python -m pytest tests/test_migrations.py
# or
pdm run pytest tests/test_migrations.py

To test trickier migrations, you might want to create a version of the db in the previous state to compare what happens after the migration.

rm db.dev.sqlite
git switch main
pdm run start
# wait for startup... then quit
git switch {feature-branch}
pdm run migrate

License

EUPL v1.2

Vendored Software

This project includes the following vendored software:

• htmx - Zero-Clause BSD
• form-json - GPL 3.0
• fastapi_rss - MIT