- Rust 91.9%
- Nix 3.9%
- CSS 2.7%
- Dockerfile 1.5%
repo-slopscore
CLI + web app which gives a "slop score" for any public git repository resolvable
via https://.
Project overview
Repo-slopscore goes through the entire commit history of a repository (upper limit is 5000 commits currently, but I have ideas to change this in the future) and detects visible signs of AI/LLM tool usage in the commit history and the source tree. The tool uses partial cloning to reduce data usage (and potentially forge server load) by skipping blobs/blob contents entirely.
Additionally, aggressive caching is used to ensure that a repo that has been analyzed before does not need to get fully cloned again. Instead, the tool remembers the precise point in time (+commit hash) of the most up-to-date commit to the repo and uses this knowledge to only clone from that point onward the next time analysis is requested.
Commit history analysis
In the tools file, there's a big list of different LLM agents and tooling. Repo-slopscore analyzes every commit in the history, looking at
- Committer name
- Committer email address
- Commit messages
in various ways. If AI activity signals are detected here, they'll get pointed out. The tools file covers a good amount of known (to me, at least) AI agents, and is made even more accurate by broad-spectrum filtering for phrases such as "generated with/by/using", which are telltale signs of LLM activity.
Limitations
Repo-slopscore does not look into each file and try to catch AIs that way.
Instead, source-tree analysis checks for the existence of common AI files
or folders in the source tree, such as .cursor/, CLAUDE.md and others.
Repo-slopscore also does not use any APIs provided by git forge providers to
analyze repositories. Everything is done purely using git.
As a result, the tool cannot look up associated merged PRs and detect AI
usage in them. This is relevant when something is squash-merged, as it
obscures the usual Co-Authored-By: commit trailers.
Additionally, this tool is not guaranteed to report accurately whether something uses AI or not. The tool's ability to analyze is limited to the above mentioned signals. There can be false-positives.
Usage
Two subcommands are available: analyze, which performs the analyzing steps purely in the
terminal, and web, which hosts a (currently) barebones browser UI to interact with.
There are no rate limits in the web app. Ratelimiting should be handled by a reverse proxy (or similar), if you plan to expose this app publicly.
# scan a repository, persisting the result under ./data
cargo run -- analyze https://codeberg.org/polyphony/repo-slopscore
# same, but emit JSON
cargo run -- analyze --json https://codeberg.org/polyphony/repo-slopscore
# force a full re-analysis (skip incremental shallow-since clone)
cargo run -- analyze --force-full-analyze https://example.org/owner/repo
# run the web UI
cargo run -- web --bind 127.0.0.1:3000
The web UI lists every previously scanned repository, shows a recent
activity feed, and exposes a form at /scan to trigger a new scan.
The --data-dir flag (default ./data) controls where analysis records
are persisted.
Versioning and MSRV
Semantic Versioning 2.0.0 is used as the versioning scheme for this project.
License
This project is licensed under the Mozilla Public License 2.0. See the LICENSE file for details.
Community
Please familiarize yourself with our Code of Conduct before interacting with our community, and read our stance about AI contributions before contributing to this project.
- Zulip: Join our community on Zulip
- Website: Visit polyproto.org
- Email: Contact us at info@polyphony.chat
be gay, do crime! 🏳️🌈🏳️⚧️