Labradorite dreamed by openai's Dall-E2: A brown labrador shooting a lazer from a blue crystal on its forehead, digital art

Labradorite rightfully belongs to the group of wisdom crystals, because it promotes a free flow of mental energy as it awakens intuition and allows us to get mentally unstuck.

- every esoteric out there

Ruby 3.4 SQLite FTS5

• Rationale
• Features / User Info
• Technical Prerequisites
• Architecture
• Folder Structure
• Major Dependencies
• Run/Deploy with Docker
  • Docker Compose
• Development
• Run it
• Follow-up: Inspiration, Knowledge
  • SQLite FTS5
  • Rails Multi-Model Search

Most Notetaking apps do either too much (Notion), too little (Apple Notes) and lock me in (all options ever).

I wanted something that does just-enoughTM, in the spirit of a flat file cms. Let me try and wrap it up in a few bullets:

• ownership of the notes (flat files)
  markdown, yaml, attachments all in one place in directories
• easy on the eyes/mind
  I don't want to see all the old stuff I may not need anymore
• a search that shows me snips in the notes
  not just the notes in a sidebar
• a very simple tagging support
• basic file uploads
  keeping the originals side by side with notes

I like to keep things simple, so I once again fell for Roda as the Web Layer. SQLite FTS5 with Extralite provides excellent full-text search without external dependencies. Pure Ruby + SQLite = simple deployment anywhere!

Search Engine

• SQLite FTS5 full-text search with BM25 ranking
• In-memory database by default for faster performance
• Configurable storage: Use DATABASE_TYPE=file for persistent search index
• Multi-field search across title, tags, and content
• Smart snippets showing matching content with context

Markdown Editor

• Preview
• Supports auto-formatting with/around saving (on page refresh)

Attachments

• supported attachments for uploads
  • txt
  • pdf
  • md
  • png
  • jpg
  • jpeg
  • heic
  • webp
  • yml
  • yaml
  • json
  • gpx

Screenshots

• Ruby v3.4+ are tested and ready to roll
• SQLite3 (included with most systems)

Little experiment on how it rolls:

• all GET gets HTML
• all POST sends FORMS/JSON gets JSON back
• all links/navigation are plain HTML a tags 🤯

• lib: Library Code / Business Logic
  • lib/search_index/: SQLite FTS5 search implementation
• views: ERB Views
• memos: the Memos/Notes in subdirectories YYYY/MM/DD/random-string
• public: css/js/media
• search_index.db: SQLite FTS5 search index (optional, for persistent storage)

• Roda
• Sequel + Extralite (SQLite FTS5)
• RedCarpet

The included Dockerfile and the docker-compose.yml are more a stub to build upon.

Currently the Dockerfile support passing the USERNAME as an build-arg.

• $ docker build . --build-arg USERNAME=$(whoami) -t labba
• $ docker run --rm -p9292:9292 labba

Either this way, or you adapt the Dockerfile to your needs.

Let's say your username on the server you run (or your machine) is johndoe, then your Dockerfile should look something like this:

# in your version of the Dockerfile, replace labradorite at the ARG USERNAME
ARG USERNAME=johndoe

That will help you both secure the service and have the files being created with the proper rights on the host system.

build and/or up using: $ USERNAME=$(whoami) docker-compose build

Volumes

• this memos directory has all your created content in it. Having it mounted from your host should make easy for you to backup (if you don't use a private gitrepo for that 😉)
• search_index.db: Only needed if using DATABASE_TYPE=file for persistent search index
• Note: The search index is rebuilt automatically from memo files on startup

The project now runs on any platform supporting Ruby + SQLite (Linux, macOS, Windows). Docker deployment is still supported for convenience.

Database Configuration

• Default: In-memory database (faster, rebuilt on startup)
• Persistent: Set DATABASE_TYPE=file for file-based storage
• Custom path: Use DATABASE_PATH=/custom/path.db with file mode

Testing

• $ rake test runs the test suite
• $ rake reset_default_memos resets the files to the repos defaults (e.g. for testing)
• $ rake reset_memos clears ALL memos

Before pushing code, you should always run rake reset and rake test,
therefore have a repo you work on AND another you use JUST for running the software.

Example docker-compose file:

version: '3'
services:
 app:
 user: "${UID}:${GID}"
 environment:
 - USERNAME=yourusername # REPLACE THIS WITH YOUR VPS user's USERNAME
 - DATABASE_TYPE=file # Optional: use 'file' for persistent search index
 - DEFAULT_RECENT_MEMOS_COUNT=25 # Optional: number of recent memos to show on homepage
 image: ghcr.io/simonneutert/labradorite-notebook:v0.2.0
 # ports:
 # - 9292:9292
 command: bundle exec rackup -o0 -Eproduction
 volumes:
 - ./memos:/home/labradorite/memos:cached
 - ./search_index.db:/home/labradorite/search_index.db # Only if using DATABASE_TYPE=file

https://www.sqlite.org/fts5.html

https://stevepolito.design/blog/search-across-multiple-models-in-rails/

Gist: a searchable table is to be added and kept updated using commit_hooks