Posty - Mastodon Archive tool

This tool was designed to create a full export of your Public and Unlisted posts from your Mastodon archive, and convert it to a static HTML site. The generated site is very easy to modify, relying on basic semantic HTML and a simple CSS library.

If you want to use an online version of this tool, there's one running on the Island One network's dev server at https://posty.1sland.social.

Create a Standalone Site from your Mastodon Archive!

• Uses SimpleCSS for styling. Edit the extracted main.css to customize for your situation. SimpleCSS automatically uses your system preferences to determine whether to show a light or dark theme.
• Low Javascript, operates primarily as HTML/CSS. Search page uses DataTables.
• Runs using CommandBox running open-source CFML under the hood.
• Posts, replies, polls, images (and included alt text applied directly to the images themselves).

If you just want to build a quick archive, you're welcome to use the hosted version but if you'd prefer to run the entire thing yourself, choose either the Docker or CommandBox method below.

Features

Posty isn't much of a CLI, it's a web application, even if you're just running it on your own machine, you'll be opening it in your local web browser in order to generate an HTML site from your static archive. It has several useful features:

• Generates a static HTML site of your Mastodon Public and Unlisted posts, bundling a simple css framework that is easy to modify.
• Maintains references to the original status IDs in naming your individual post HTML files.
• Supports Locales: You can generate your HTML site with whatever locale you want.
• Supports Language Encoding: The HTML for your posts will be decorated with the lang attribute matching the language selected for the post, which should help with language translation software.
• Localization applies across the entire application, with translations for many languages built-in.
• Optional Tags directory (Tag Cloud) and Search pages.
• Optional: Generate with SES status urls for hrefs (ie., https://yoursite.social/@user/1234) instead of direct references to .html files.
• Host as an Archiving service for others
• Host as a Dedicated Archiving Service for a retiring Mastodon server. Configure the .env file to be a dedicated archive location for your server, and then ask your users to upload their archives into Posty. Once your server is retired, you can convert Posty to a static HTML site, maintaining previous status URL references.

Getting Started - Docker

The easiest way to run Posty is with Docker.

1. Clone this repo to a directory somewhere. cd to this directory and create an env file: cp .env.example .env
2. Run docker compose up -d
3. That's it. You'll be running on localhost at port 9901. You can change the port in the server.json, and then docker compose restart.

When you're done, type docker compose stop to stop the container.

If you want to skip all of that and just run docker commands, do like so:

docker login registry.digitalocean.com -u dop_v1_b6e98cf81cf372623c4d3e26a3bd082554594cf3eaf02e8b01c560ff519e24a0 -p dop_v1_b6e98cf81cf372623c4d3e26a3bd082554594cf3eaf02e8b01c560ff519e24a0
docker pull registry.digitalocean.com/oliphant/posty:latest
docker run -d --name posty \
 -p 9901:9901 \
 -v posty_archive:/app/archive \
 -v posty_stage:/app/stage \
 -v posty_db:/app/db \
 registry.digitalocean.com/oliphant/posty:latest

Getting Started - CommandBox

1. Clone this repo to a directory somewhere. cd to this directory and create an env file: cp .env.example .env
2. Download and Install CommandBox. (On MacOs, I recommend doing it via Homebrew.)
3. Run commandbox in the terminal with box. While you're in there, run update --system to grab the latest and greatest.
4. Inside of the CommandBox prompt:

cd /yourdirectory/posty
install
start

5. That's it. You'll be running on localhost at port 9901. You can change the port in the server.json, and then restart in CommandBox.

When you're done in the commandbox prompt, type stop to stop the server and then type quit or exit to exit the box terminal.

Uploading your Archive

Access the site at http://127.0.0.1:9901/

In the browser, you'll be prompted to upload your Mastodon archive .zip file. Once uploaded, you should see a preview of your profile and all of your posts, and can include or exclude individual posts and replies as you see fit. From there, follow the prompts, and you can quickly generate, browse, and download a zip archive of your static site that can be hosted anywhere.

That's all you need to know if you just want to run this locally, generate an archive, and be done with it.

Read further if you want to host your own archive service for Mastodon, or are curious of how it works under the hood.

Running As a Hosted Service

You can run this as an archiving service (so others can upload their archives here, too). Here are instructions on how to run this as a service on Linux.

You are welcome to use the provided /utils/posty.service in the repo, as there are a few commented examples.

By default, Posty runs as an archiving service for anyone and everyone who has an archive to upload. But you can also run it as a dedicated archive service for a specific instance.

Running as a Dedicated Hosted Service

If you want to limit this to a specific domain, add the following in your .env file:

LIMIT_TO=yourserver.social

This will force the server into 'Dedicated' Archive mode and only archives from yourserver.social (in this example) can be uploaded. Other archives may be uploaded, but will be presented with an error message once the archive has been read, and all data will be purged from the server.

Beyond this, there's a few changes to default behavior:

1. Instead of creating a folder under /archive named user@yourserver.social the @yourserver.social part will be removed, the user will be saved using only their handle.
2. SES urls for post references will be enabled by default.
3. Because of the folder name change, the rewrites can be shortened and omit the instance from the handle. So you can reference the archive at https://archive.yourserver.social/@user.

If you ever wish to change this and allow general uploads of archives again, remember that archives will be uploaded using their full handle.

Running as a Static HTML Archive

Once you've collected all the archives, it's time to disable the application server and host as static HTML.

In CommandBox you'll need to stop and then server forget. In Docker, you'll want to docker compose stop and docker compose rm -f to ensure everything is reset.

If you're running this as a service, you'll want to edit your posty.service file to enable the server-web.json option. If running as Docker, you'll want to edit the docker-compose.yml file to uncomment the server-web.json option.

Once the server is back up, you'll only be running the webserver, not the application server. You'll be serving the index.html as your root page's content as well. All of the rewrites and web server settings will remain functional.

Updating Posty

From your Posty directory, do a git pull and then:

In Docker: Most of the time, docker restart will be sufficient. But if you want to ensure the latest CommandBox updates, you can always blow away and rebuild everything: docker compose stop && docker compose rm -f && docker compose pull && docker compose up -d

In CommandBox: restart or with the system service systemctl restart posty.

Note: If you're running CommandBox, you can also do a 'quick reload'.

Instead of doing restart or systemctl restart posty simply pull down the code as usual, and then hit Posty with a reload parameter. (If the server.json or box.json files change, however, you should do a full restart rather than a quick reload.)

?reload=commandbox (commandbox is the default password to reload the application, but use whatever cfconfig_adminPassword you have defined in your .env file).

When there are updates to the .env.example file and new envs are added, you must add them to your own .env file or else Posty won't start. You can run box dotenv populate --new via CommandBox to update your .env file, or do it manually. Once the env is present in your .env file, you should be able to restart Posty as normal.

Customization

The /assets folder at the root holds the entire template for static HTML pages, and the css and js assets (in their associated folders) get copied directly into the the archive, at the root folder.

This means that if you modify the main.css you'll not only be able to affect the look and feel of Posty as a whole, but that same CSS file will be copied into each archive as well. You can upload an archive and make changes to the CSS, refreshing the display page to see what the archives will look like.

If you want to have all archives match the theme of your server, this is your opportunity to edit and style the main.css until the archives look exactly the way you want, then let people into the application to generate their own static archives from this template you've created for your instance.

Changes to this file won't impact any existing archives, that's the thing about static HTML, once you've generated it, there's no changes unless you upload again, so you're encouraged to work with this until it looks the way you want.

Thankfully, Posty has been designed to facilitate that entire workflow, re-uploading and re-generating archives to pull in style updates is just part of it.

Changing the backend code, including the HTML generated, that's a different story.

Most of the templating for that, at least for the pages themselves, is in /views/extract/partials/.

But individual posts are generated via a means I'm not entirely proud of...namely string concatenation, and that's in /model/services/Generator.cfc in the renderPost function.

I hope to refactor this later to make templating a bit easier. You can see some examples of better templating in the Post renderer.

Then there's localization. All of that is in the /localization folder, either in the translations.json or in the folder structure itself, but that will require a separate topic for later.

Important Folders - The Application

This app will populate the ./stage, ./archive, and ./db folders with exported data and metadata. The /archive folder has all of the generated static archives, with the folder name saved as the user's handle, ie., user@site.example.

The stage folder is used when first uploading and extracting the archive. That folder exists until you choose your posts and click 'Generate', at which point it will copy image files from that directory down to the archive folder and wipe out the stage folder once the copy of those files is complete and the outbox.json and actor.json have been read into memory.

A metadata.json file is created in the db folder at this time with a saved identifier shared in the user's cookie. This is what prevents a malicious user from uploading another archive for the same user and overwriting their work. It's not a true login, but more of a 'first-come, first served' to protect an existing archive from getting overwritten by anyone but the person who first uploaded it.

You'll then be on the 'Publish' page, which will give you the option to open the archive in a new tab and review it, download a zipfile of the archive, or click a link to Delete all data. If you choose to delete all data, the metadata.json will be deleted from the db folder and the archive will be deleted from the archive folder.

You're welcome to test out that entire function from start to finish a few times. I certainly have.

How Does It Work?

The app runs an embedded JBOSS Undertow web server, talking to a Lucee which is a free, open source implementation of CFML (ColdFusion Markup Language). Under the hood, it's all a servlet running on Java 21 (configured in the server.json and box.json), and it interprets .cfm files kind of like JSP interprets .jsp files (and you should technically be able to run both on here, but that's beyond the scope of this document). Everything gets compiled down to java under the hood on the first request and that's all that's happening here. We're executing Java, compiled down from CFML, and CommandBox wraps all of this like a CLI attached to a bundled application and webserver with access to its own java libraries as well as the entire JVM.

It's pretty powerful out of the box, in other words, even before you add libraries and extensions to it.

All of this bundled tech is only necessary for generating static archives, and is a bit overkill for that purpose. Once generated, the static HTML/CSS site itself can be hosted anywhere pretty cheaply as a static site, or simply browsed locally by dragging the index.html file into your web browser. That part is pretty old school, and designed to be easy to modify.

Development/Contributing/Customizing

Once you have CommandBox running, if you're in development mode (see the .env file) you can reload the application dynamically as you make changes to the code. The MVC framework used is called "Framework One" or FW/1.

The controllers, views layouts and model folders and the Application.cfc (which bootstraps the application and has built-in request and other handlers) are the primary code of the application, and with the exception of the Application.cfc (which is standard to every CFML application anywhere) the other folders are FW/1 standards, and that's where you can review the code or make changes.

What do you mean "It's the Postiest?"

It means: of all the posty post generators out there, posting your posty posts, this one is the postiest of all.