Watch 3

Star 6

HTTPS

Download ZIP Download TAR.GZ Download BUNDLE Open with VS Code Open with VSCodium Open with Intellij IDEA

as this codebase is still extremely unfinished, have this article i've been writing alongside instead of a README. it's probably more useful than anything a README for this could contain right now

also see the csharp-dev branch
also see KittyShopper/ois#1

creating a generic activitypub server is a horrible idea i would not wish upon my worst enemy

(Note: this is being written concurrently with the actual implementation. You will see me thinking out loud, possibly realizing mistakes, changing my mind partway through, and the overall development process in roughly the same order it happened)

I want to act on an idea i had and slowly refined in my head for a while now, on making an AP server whose only job is to handle actors, leaving all the indexing of reply trees and other "actual functionality" to other services an actor could log in to (if it sounds familiar, this is a C2S server. Or an ActivityPDS if you don't care about worthless protocol wars and like to keep up with what the other side is doing)

To make C2S actually work in real life, I will be ruthless in creating bespoke extensions wherever necessary to get to a level of functionality I am happy with. My end goal is to federate with most other contemporary server implementations (I frankly don't care about any existing C2S clients, if any actually exist), and to not regress on UX compared to other AP implementations (I won't have the polish of Mastodon, but that ideally won't be the protocol's fault), not hypothetical future implementations that handle spherical as:Notes in a subject-predicate-object shaped vacuum, using a client that smells like it predates Myspace.

Aside from the generic "identity server" part, I am also going to attempt an implementation of the other side, the Client of C2S. Which is how I plan to deal with "nobody implementing my extensions" problem. After all, standards (and extensions) are worthless without any implementations. And thinking of standards without having at least a prototype tends to end up with horrendous junk. See: ActivityPub. (And I want to, you know, actually use the damn thing)

To make things a bit more consistent, I will be referring to the generic part as an "identity server" and to the part that has most of the logic you'd expect from a contemporary AP instance as an "application server". (UPDATE: I kinda fumbled the ball on this) I think the Client-to-Server nomenclature is flawed in that it brings up images of phone clients connecting directly to generic servers, which, while possible (and will be supported), is a terrible way to implement any nontrivial functionality. When I refer to "clients" however, I mean both application servers and things like mobile clients.

The identity server's responsibilities will be:

• Hosting ActivityStreams objects and collections (including the actors owning them), and any local media blobs they may refer to.
• HTTP signature key management
• WebFinger
• Delivering activities created by clients (outbox)
• Forwarding external activities to application servers (inbox)
  • May implement a websocket extension to the inbox to let non-server clients listen to the inbox in a more efficient way than polling the collection
• Side effects from follows and blocks, as those are global to an actor.
• For clients which do not have an application server component, proxying object fetches with the necessary HTTP signatures.
  • These will be cached, and the cache will be warmed up by remote as:Create and as:Updates. Mainly to prevent "read receipts" at least for the first month or so before they expire from the cache.
  • Verifying the actor's credentials, and proxying the actor's requests to application servers, potentially through the same as:proxyUrl endpoint as above.

The last one may be a bit interesting. This is another design choice I stole from Atproto PDSs (and one they seem to dislike, given their push for backend-for-frontend), and it has some really interesting properties:

• When actors are allowed to migrate between identity servers transparently, they function as a surprisingly well client-side ISP-level censorship avoidance technique.
• The PDS/identity server, like a web browser, can choose to intercept requests and responses and modify them according to the user's wishes.
• Clients which support multiple application servers can send requests to all of them while only keeping track of a single session, allowing interactions that are more seamless, and potentially opening the door for application servers which only handle specific functionality as a sort-of "plugin" to another.
  • In fact, clients can make application servers into optional extensions on top of a pure-C2S base. This is an area I'm contemplating exploring with my own C2S client.
• It may help application servers reference dynamically generated objects in objects they create, while allowing them past the same-origin policies of most contemporary implementations.

Oh, while i mentioned it, If possible, I would like to implement enough of FEP-ef61 to make a complete implementation possible in a backwards compatible manner. Aside from the whole transparent migrations part, it ends up solving the currently unsolved "actor's namespace" problem while letting applications have some control over IDs pre-as:Create. (Which is a useful property if you're creating multiple objects referring to one another at the same time)

Tech Stack

I will likely be using C# and .NET Core for the server bits. Despite it's reputation it can produce some surprisingly light and fast servers as demonstrated by projects like Iceshrimp.NET or AppViewLite. And by doing even less I can likely get something even lighter than that, at least for the identity server bit. It's also a reasonable language that doesn't suck that much to write (not sorry, Go), and doesn't make me think too hard (sorry, Rust. I don't have any stimulants nor a CPU with more than 8 threads).

To make deployment as easy as possible for the identity server, I'm thinking of stealing a decision from the reference Atproto PDS implementation, and creating separate SQLite databases per user. Separating the databases per user helps with the concurrent write problem, as one person can only do so much so fast. And any instance-global data should change infrequently enough for storing them all on a shared SQLite database to not be a problem.

The SQLite support on EF Core is looking like a bit of a pain, with things like JSONB support missing. Postgres would've been way more fun to use here.

Right now, I'm also caching remote actors' keys on an SQLite database as well. I have a hunch that this may end up being a bottleneck. Given SQLite infamously can't write concurrently, authorized fetch on a large enough boost can cause a LOT of requests per second, where any cache misses would need to be downloaded and cached. Something like https://github.com/koculu/ZoneTree on "Async-Compressed Mode", https://github.com/litedb-org/LiteDB, or RocksDB may prove useful here (and for other caches).

UPDATE: Caches are now handled by RocksDB. Am contemplating replacing the per-user SQLite databases with another RocksDB as well given my data is not relational in a way the database layer can make sense of.

UPDATE: I am experimenting with Rust and dropping SQLite for Fjall (because the RocksDB bindings do not compile) for the identity server. The application server will likely stay as C# because it's main bottleneck should be database queries, and won't need to deal with extremely concurrent requests from large boosts or the like. Multiple identity servers can also share the same application server instance so I imagine it requiring slightly more server resources won't be too much of a problem, communities can share the instance (and given it'll be the primary place remote user moderation happens, I expect they will)

I do want to have authorized fetch here. Not because I believe it's a good way to implement denylists (it has quite a few flaws), but because it allows allowlist federation, object visibility enforcement beyond "unguessable IDs", and the ability to track which actors have seen which objects, so we can send as:Updates and as:Deletes to them. Since I'll need the overhead of AF anyway for those, I may use it for denylisting as well. (You can do the last part without validating any signatures, but then malicious requests could add anyone they want to those lists which may cause a denial of service. I'll still need to think about that)

As of writing I'm still a bit unsure on if i should store ActivityStreams objects in user databases or as separate blobs like media (filesystem or S3 depending on configuration). Collections will likely be handled separately in the database instead of being "rendered" into objects.

UPDATE: AS objects will be stored in a database as we'll need to dynamically update their contents when passing them along (so we can't just use a CDN on top of an S3 bucket). Maaybe the server could fetch them from a private bucket but at that point the API costs of S3 providers may be a problem.

The application server will use a Postgres database, as it will be dealing with a lot more data being written a lot more frequently, and queried in complex ways, compared to the identity server.

Am contemplating using a TypeScript backend to benefit from better integration with SvelteKit (which is what I'll be writing the web client's frontend in). I really do not like the backend-for-frontend approach this brings (and the "locking-down" of the client API if I use their RPC system), but there are quite a lot of user-visible benefits. (Reducing the amount of data sent to (and processed by) the client, for one. May even allow for completely JS-less access if we forego sending AP activities directly to the server as well)

It'd also make deployment of the client easier (one backend service to maintain instead of two, assuming we want server-side-rendering).

I am going with a TypeScript backend with a Solid frontend. I decided to evaluate it for various reasons (Svelte 5 still not having usable browser dev tools, Tanstack Query adapter being broken) and decided to stick with it. JSX is surprisingly more usable than I initiall thought, and it allows for a few neat tricks I don't know if Svelte can do as easily.

Authentication

Server-to-Server has more or less standardized on HTTP Signatures, and I have no intention to replace that (though I would really like to opportunistically upgrade to ed25519 if FEP-521a keys are present, which I'm not sure if anyone else other than me would implement support for in the HTTP Signatures path).

For Client-to-Server authentication, the authentication primer suggests OAuth 2.0. I've decided to implement it using the authorization code flow (requiring PKCE) (削除) via a (likely broken) implementation of OpenID Connect (but I don't expect application servers or clients to make use of that), as there was a good enough library for that. (削除ここまで) I've again stolen some design decisions from Atproto and used draft-parecki-oauth-client-id-metadata-document to let clients register themselves.

Authorization will be done through RFC 9396, with the exact types being documented in docs/oauth.md#rich-authorization-requests

Since implementing this I've became aware of FEP-d8c2 which specifies pretty much exactly what I did except using AP actors for the client ID. I see no reason why that part can't be implemented alongside the IETF draft until one of them "wins".

For the identity server proxying to application servers, I may just use the as:proxyUrl & HTTP Signatures as well.

I have no idea how secure my setup will actually be in use, and will probably need to think this through a bit deeper. In particular I haven't thought through forcing PAR or DPoP (削除) (and I don't think the latter is implemented in the server library I use, though it doesn't look too difficult to retrofit on top of it) (削除ここまで)

Authorization

I'm still thinking this through. It'll prolly be through OAuth scopes for C2S activity and endpoint authorization, (削除) combined with custom visibility extensions on objects (I want to explicitly separate collections and actors, otherwise as:to and as:cc (and as:bto and as:bcc) would be enough. I will convert between them though) (削除ここまで) enforced through authorized fetch where it's not visible to as:Public.

UPDATE: Thinking about it a bit more, we can handle collections just fine. May not need an extension after all

Limiting the @types of objects clients can as:Create (or activities they can send) is not effective as remotes may be "duck typing" objects based on their fields. Known as: namespaced activities can be specified, but anything else is fair game with perhaps a single "forward unknown activities" permission.

There'd likely be additional permissions for "handling your own objects" and "handling other clients' objects". And perhaps one "handling global objects" permission for the actor and their collections like as:featured?

For "which actors are allowed to modify which objects", I'm contemplating using as:attributedTo, on every object. Possibly with a fallback to "whose namespace does the object belong to" if as:attributedTo were to somehow get lost.

Concerns with nomadic identity in practice

Thinking it through a bit more, I may end up not going through with nomadic identity...

verificationMethod of all portable objects being the DID forces gateways to store the user's private key if it ever needs to send activities in behalf of the user.

This can be workable if you just... Don't send activities as the user. But I am worried about the limits this brings in practice. It's worth noting that "activities as the the user" includes such things as accepting follows automatically, or gateways rotating their own keys associated to that actor.

When HTTP signatures are necessary for communicating with other servers, each gateway that makes requests on behalf of an actor SHOULD use a separate secret key.

Stupid Idea: Let the client generate the server keys? No. Bad. This will let them abuse same-origin security by sending their own signed activities outside the identity server's control.

Also how do you deliver the private key over to new sessions without storing it? Encrypting it with a key derived from the account password? (Would run into problems when you change a user's password, especially if they forgot it!) Again, did:key is not rotatable if the gateway (or anyone else for that matter) sees that private key, and is the only thing stopping you from moving any actor to any gateway (who will let them in). And none of the other DID methods are required by the spec.

Also how do "private" objects behave? If they're signed then you have no deniability if someone Just Forwards Them Along Anyway. For public objects this may be less trouble, but you'd need to exclude private objects from portability, which would no longer make the migrations seamless.

Alternative to nomadic identity?

For seamless migrations, I wonder if asking people to place redirects to their own URLs would work. Then the identity server could use that prefix as the "actor's namespace". Identity server migration could then be done by switching that redirect to a different one. (And moving all the data over some other way)

This is wholly dependent on how different software implement same-origin of course. If they don't allow redirects between hosts even when the id is correct, some form of reverse proxying may be necessary. (If the identity server is aware it's being proxied, this should not impact HTTP Signatures as it can compensate for the different Host)

Unfortunately though this configuration is more or less permanent and can't be changed ever (e.g. if the custom domain expires, or if you decide to move an account to a custom prefix after creation)

Still, I may implement this as it's a lot more promising UX-wise (after the initial setup, which is fairly technical and unintuitive). And you can always run a Mastodon-style as:Move between accounts which is better than nothing.

The current "actor namespace" implementation allows any arbitrary domain to be served as long as it points to the instance and an actor with that hostname in it's namespace exists. Maybe just Pointing DNS At An Instance could be enough?

Limitations on objects

Object IDs, if specified, will be required to be namespaced under the "actor's namespace". I'm unsure what that means in practice yet but I'll likely just do {actorId}/{...restOfPath}.

To make sure there's a single, complete (i.e. not missing any fields), source of truth for all objects, I will not be allowing clients to send embedded objects in place of id references unless they have no id, or have their parent's id with a fragment.

If the objects are not signed then the identity server can choose to inline local objects itself, and this is something I'm planning to look into for performance (and compatibility) reasons. Since the identity server has the signing key it can, in theory, re-sign the inlined object but I'm worried about the performance of that. Could be cached though.

This is going to require me to handle JSON-LD unless I hate myself even more than that, (削除) which in my current tech stack means I'll get to have two incompatible JSON libraries in the same project. Fun for the whole family! (削除ここまで)

Collections

Given I will also be paginating collections dynamically, I will:

• Not support manual creation, updates, or deletes of as:CollectionPage
• Not support specifying as:first and as:last explicitly
• Not support clients signing collections
  • They can ask the identity server to sign the dynamic collection on their behalf, if that's something they want to do.
• Not support inline objects of any kind inside collections
  • This one can be implemented, but makes my life a whole lot easier to not. (How do you remove an inline object from a collection?)
    • Check equality of expanded and JSON-canonicalized string?
• Not treat objects with fragment IDs as collections (no as:Add, they should be updated with the parent object)

I don't expect any other limitations on collections. as:items on as:Create (and perhaps as:Update if I want to let clients overwrite the entire collection in one go) can be handled relatively well and the rest of the object can be stored as a "seed object" that then gets the pagination fields dynamically injected on render.

It may be interesting, for performance reasons, to just render as:items for collections with a low number of items. Pagination only happening when a second page is necessary could save on HTTP requests.

The only difference between ordered and unordered collections will be their @types.

UPDATE: The above is no longer true. In the Rust implementation ordered collections are 2x the size of unordered collections (which are automatically ordered by ID lexicographically). Should advocate for unordered collection use if ordering is strictly not necessary.

This is not unfortunately "fixable". Equalizing both collections would have a 4 byte overhead on all entries (basically nothing) but would ruin "is present in collection" performance (https://wetdry.world/@kopper/114820470606384593). The previous implementation was only equal not because ordered collections were smaller but because unordered collections were larger.

In terms of accessing collections, filtering such as FEP-6606 (potentially doable but likely to be slow), sorting any way other than from-the-start or from-the-end, or anything SparQL shaped (both basically impossible with the current architecture) is explicitly out of scope for the identity server. That kind of indexing should be done by a client. The decision to proxy API calls over as:proxyUrl means that a general-purpose "indexing server" can be logged into to do that kind of free-form indexing, and any clients which support interacting with it can call into it with their existing ois access token.

I am still unsure on if collection items should be filtered by visibility.

TODO: consider https://codeberg.org/fediverse/fep/src/branch/main/fep/5711/fep-5711.md for the collections the client creates

Actors

Clients must be forbidden from creating anything that looks like an actor with public keys (blocking anything with fields sec:publicKeyPem and sec:publicKeyMultibase ought to be enough, I think?), as if that's the case they can start sending signed requests without going through the checks of the identity server and exploit the same origin security model to interfere with other actors.

This check must be done both with and without the JSON-LD context applied to account for malicious contexts renaming fields to get around it for remotes which may or may not handle JSON-LD.

THIS IS NOT ENOUGH: SUPPORTING REMOTE JSON-LD CONTEXTS AT ALL FOR INCOMING OBJECTS IS AN ACTIVE VULNERABILITY https://socialhub.activitypub.rocks/t/am-i-understanding-something-wrong-or-is-json-ld-remote-context-support-a-gigantic-security-vulnerability-if-any-dumb-c2s-server-implementation-tries-to-federate-with-the-current-network/5439

Incoming Activities

If the activity objects themselves are not signed, the HTTP signature should be passed along with the headers the identity server used to validate it, so application servers can also validate the contents come from the described actor. FEP-1b12 as:Announce forwarding can not work as not all real world activities can be resolved from their @ids.

Outgoing Activities

Outgoing activities need a custom extension flag that marks them as "ephemeral", for things like as:Delete (so the database doesn't fill up with delete activities)

Additionally; we could treat all activities without any as:to/as:cc as ephemeral (for when the individual activities don't matter but their side effects do). If a client wishes to have a permanently available activity that's private to itself, it can have an as:to that only contains the actor and nobody else.

We'll likely want to implement an extension that lets clients batch multiple activities at once, so a post and it's likes/replies/shares collections can be created at the same time by a client.

Application Server bespoke API

I wonder if just sending object IDs to the user is a good idea. The client could then use their as:proxyUrl to load those objects from their identity server, which could act as a cache. The application server then wouldn't need to store anything it does not explicitly index.

Would help a LOT with the "C2S with server as a bonus" use case, alongside easier propagation of edits and deletes. May cause extra traffic for remote instances.

A problem: This may make identity servers more resource intensive as they now store remote data. Cleaning up the cache would be trivial though, compared to a contemporary implementation where remote objects and local objects are mixed with foreign keys in the database.

Another problem: This would require at least 1 extra request (assuming custom extension to batch request multiple objects at the same time). Would that impact UX in a significant way?

Object storage

Objects are stored as a pair of JSON-LD expanded object, and the value of the topmost @context. They get re-compacted on render time, which is necessary to allow the server to modify the object (injecting actor keys, dynamic collection pagination, inlining referenced objects) while still respecting it's context.

A downside of this approach is that nested objects that may have their own @contexts will not get them persisted. I don't think restriction matters here, as I expect clients to produce objects themselves with full awareness of their structure (and can therefore place all the necessary term definitions to the top-level context).

At least in the inlining case, deferring that until after compaction would help with preserving @contexts, as we can paste the compacted object in place directly, context and all.

An idea worth exploring is letting the client ask the response to be re-compacted to a context document it specifies. This could let clients who don't want to deal with LD processing offload it to the identity server, and get a predictable structure they can walk through like any other non-LD remote.

Remote collections

I may need to explicitly maintain some collections for remote users (for visibility based on delivery to collections). (削除) Given my current database schema is one file per local user I'm not entirely sure how I'm going to store those. Given the rarity of collection modifications, storing them all in a single SQLite for "everyone else" might be Good Enough? (削除ここまで)

If delivery to collections wasn't a thing I may have been able to get away with not handling remote collections in the identity server entirely. And given I don't know which collections people will use for visibility (wouldn't be very generic of me to limit it to as:following), this means remote users now have a way to store some amount of permanent data in the identity server. Blehg.

Update: They could do that already by Just Sending Activities (which would be permanently saved on the inbox), ya dimwit.

Problem: What happens if remotes do not explicitly send collection activities and use that collection for visibility? We'd need them to implement a "collection probe" endpoint. Could maybe implement Smithereen's extension on our end? But what about remotes which do not have any? Walking the collection would be too expensive.

Actually, given it's just for visibility to our actors, and to figure out which of our inboxes to deliver to, we could only track remote collection activities where as:object is one of our own actors. Explicitly documenting "we won't accurately deliver to remote collections" seems like a reasonable compromise here, no?

(If the remote collection has an as:attributedTo, we could deliver to those actors and hope they know what to do with it)

These collections won't need to be ordered or exposed through any API, as they'll only be used for internal "is in collection" queries. Anyone wanting to iterate remote collections should load them from the remote directly.

Interlude: JSON-LD

Took a brief break here to create my own JSON-LD implementation because I benchmarked dotNetRdf and thought it should not take milliseconds to handle basic JSON-LD (no networking involved in this number). Accidentally ended up being 6-7x faster with the same multiplier in memory allocation savings. Oops.

I was already not the biggest fan of JSON-LD, but implementing it brought up a brand new level of disappointment I did not expect to feel from a fucking W3C specification.

Disappointing spec things:

• Missing and unclear steps. Had to reference dotNetRdf itself more than i expected.
• Way too "JSONly-typed". There may be more performance to squeeze out by replacing the JSON data structures with more specialized stuff, and a more stable implementation by explicitly defining what can or can't be null, but the spec won't help you there because the only things it knows are "map", "array", and "string" and all three are interchangeable in a variable. Bonus JSON-LD implementor tip: You can replace this string concatenation with a single byte
• "IRI Expanding" IS NOT THE SAME THING AS "Run the IRI Expansion Algorithm" WHICH IS A HORRENDOUS FUCKING MISTAKE JUST EXPLICITLY SPECIFY VOCAB:TRUE EVERYWHERE IF YOU DON'T WANT TO MAKE IT THE DEFAULT
• (same with "IRI Compacting")

Disappointing algorithm thing (singular): Half the library being asynchronous because literally any part of expansion & compaction can call into context processor (which can load remote documents) is... Not Ideal when making things faster often requires tools which do not work in async methods. I had a go at mitigating this a bit, but unfortunately the existence of @type:@json makes this WAY more difficult than it ought to be, so I dropped it. Feel free to finish that effort if you think it could work. Just making things sync, by itself, brings a slight perf increase.

And I have to implement JSON-LD 1.1. Despite what ActivityStreams says, real world fedi does not work in 1.0 alone. This is also why JSON-LD.NET was out of the question, despite it being faster than dotNetRdf (though it still uses Newtonsoft.Json, so I'd end up replacing it either way)

And finally, to anyone else making similarly complex specs: Please have tests for the intermediate steps alongside tests for the complete algorithm start-to-finish. It's extremely painful having to write 1k lines in it's entirety before having some idea on how much you got correct or not. You should be able to automagically instrument yourself said tests by running your complete-algorithm tests with extra code to dump inputs and outputs.

System.Text.Json Wishlist

• JsonArray and JsonObject sorting in-place (LINQ .OrderBy allocates an extra array)
• Creating both of them with an initial capacity (resizing the internal JsonObject dictionary is a large chunk of time right now)
  • We are currently using some evil and extremely unsupported hacks to make this work, though it should really be an official feature
• JsonArray-from-enumerable constructor that does not require allocating an array (Can do without if the previous item exists)

Anyway, back to your regularly scheduled ActivityPub talk:

Caching of private objects

This is kind of a problem: Remote software which do not check HTTP signatures on GETs (such as Misskey) frequently completely forbid people from fetching private objects (as they can't verify if they're allowed to). Which we'd need to do if we ever expired those objects from our cache. So we can not expire any private objects. This is sad.

Thankfully, a VAST majority of objects are public, so this shouldn't be too big of a problem.

Also, we should evaluate remote software to determine if this really is as big of a problem or if it is just a handful of outliers. If so, we may not need to bother with this (or at least settle on extending their cache lifetime instead of outright keeping them around forever)

Deletions and activities of deleted objects

May need to store a map of objects and all the activities that refer to them, so we can delete the activities when deleting the object as well.

This is important for privacy reasons, as many instances (us included, for compatibility) will embed full object content inside the as:Create or as:Update activities themselves.

This map should only need to contain remote objects which aren't as:Public (and therefore need to be permanently cached). Local objects I'm not too worried about given the embedding of local objects in activities will likely be done dynamically (so, no privacy issues), but we'd still want to track them to prevent database rot.