Sciop was born to become a federated bittorrent tracker. To make that possible, we need to stabilize the single site, and an enormous part of that is building the moderation and curation tooling to make it safe, convivial, and understandable in a federated social context.
This project gathers a few related-but-separable issues that together would make a big step forward in the ability to operate as a real distributed archive.
Sciop is also a neat (in my opinion) way of doing full stack, and if you've never worked with fastAPI and htmx together I think you are in for a refreshing treat of low-fi, high-function web tech :).
These are sliced into ~afternoon size pieces for pairs, and fuller details including dependency structure is represented in the issues themselves. You can pick and choose any of these and work on them separately and leave the integration for later if there isn't time! a brief summary and description of how these issues relate:
- #236 - Report Items - (medium size) - A prerequisite to the other items. Unfortunately, malicious activity exists! We need to make it possible to report mislabeled, malicious, or broken uploads. This can be implemented against the existing items, and worked on in parallel to the other features like commenting - once this exists then it should be straightforward to expand it.
- #237 - Comment on items - (medium-large size) - It is a common need to be able to talk about datasets, uploads, etc! This is also very important for the next item, being able to coordinate ongoing scrapes. This will require more considered work to become in a final form, but merely being able to do it would be a huge first step.
- #29 - "Claims" for ongoing scrapes - (medium size) - Sciop is designed so that datasets and dataset parts can be created without uploads in them. This was intended to report datasets that need to be preserved and facilitate scraping as it happens, which is currently done on the forum. To do that, it needs to be possible to mark a dataset or part as a progression of "claimed" -> "in progress" -> "downloaded" -> "uploaded" statuses. There is an existing PR from a long ago version of sciop that has started this and may be of some inspiration. The issue for this will be filled out in time for the hackathon.
- #376 - Display unfilled uploads - (small size) - similarly, we want to be able to visually differentiate unfilled uploads, and have a separate view for datasets that have no uploads for people who want to do a scrape!
Become federated :)
We would like to speak ActivityPub, and we would like to be able to use it to disseminate datasets, uploads, and torrents between instances.
We would also like to implement one of the several FEPs related to mobile identity, such that someone is able to hold independent keys that allow them to prove they are the owner of an account on another instance and provide a signed copy of their user data: datasets, uploads, and any other metadata that would be known about by the instance.
To do this, we would like to not fake it by making relational representations of JSON-LD objects, but would like to operate JSON-LD first by making an oxigraph database for the AP-facing stuff, and then have the current sqlite database be just a web representation of the underlying triples graph.
Become a bittorrent tracker
This could/likely shoudl happen in two different ways:
- Integrating with an external tracker, e.g. as in #400 , if one might want a high volume tracker that makes the site behave like a typical bittorrent tracker
- Implementing a low-volume private tracker, where it is possible to not just share seeds among an individual instance, but also share seeds among 'neighbor' instances (that share all their seeds) or across extended social graph (similar to how one would have a follow/following relationship on the fedi and elsewhere, once can have a co-seeding relationship for private torrents).
Good curation tools! verified accounts, collections, groups
This means both the tooling for our current instance, which is a set of relatively straightforward moderation needs like reporting, etc. but also extended federation curation and moderation tools.
We need to contend with all the classic archival and libraries problems like provenance, duplications, derivations, multiple versions, and so on, but in a totally free-for-all context. Some of the essential ideas here are that we can't expect completeness or perfection, but it does need to be possible for an instance to add, fork, merge, and indicate sameness with other instance's datasets/torrents.
This can be implemented as a linked data overlay starting with some basic adjacency terms from skos for declaring things "sameAs" and so on, and extend from there.