ai-files: Search /commits endpoint to find more bots #2

0x4261756D commented 2025年12月22日 14:38:53 +01:00

By querying the last 100 commits and checking their author and commit message for AI bots many repos that slipped through the cracks no longer do. By only checking files if no committer was found the API usage should be lower on average.
This is not a replacement for #1 since both only show the last 100 commits/contributors. On e.g. https://github.com/home-assistant/core neither claude nor Copilot isn't one of the last 100 contributors but one of the last 100 commit (co-)authors but the inverse can also be true, i.e. a repo where AI bots are among the last 100 contributors but the last 100 commits were ethically sourced.

By querying the last 100 commits and checking their author and commit message for AI bots many repos that slipped through the cracks no longer do. By only checking files if no committer was found the API usage should be lower on average. This is not a replacement for https://codeberg.org/jacksonchen666/userscripts/issues/1 since both only show the last 100 commits/contributors. On e.g. https://github.com/home-assistant/core neither claude nor Copilot isn't one of the last 100 contributors but one of the last 100 commit (co-)authors but the inverse can also be true, i.e. a repo where AI bots are among the last 100 contributors but the last 100 commits were ethically sourced.

jacksonchen666 commented 2025年12月22日 16:55:56 +01:00

Oh hi someone, thanks for the unexpected PR?!

I'm personally more keen on contributor list rather than last commits because commits are more volatile. Although I did not give it proper thought, so this isn't a final decision. (I also haven't looked at the PR proper and probably won't for a bit)

Oh hi someone, thanks for the unexpected PR?! I'm personally more keen on contributor list rather than last commits because commits are more volatile. Although I did not give it proper thought, so this isn't a final decision. (I also haven't looked at the PR proper and probably won't for a bit)

0x4261756D commented 2025年12月23日 22:53:13 +01:00

I also have an implementation of #1 now (https://codeberg.org/0x4261756D/userscripts/src/branch/contributor_and_commit_check) but it depends on this code. Should I update this PR or make a new one? I can also try to disentangle the contributor code from the commit stuff so they can be two clean separate PRs.

I also have an implementation of https://codeberg.org/jacksonchen666/userscripts/issues/1 now (https://codeberg.org/0x4261756D/userscripts/src/branch/contributor_and_commit_check) but it depends on this code. Should I update this PR or make a new one? I can also try to disentangle the contributor code from the commit stuff so they can be two clean separate PRs.

jacksonchen666 commented 2025年12月30日 21:21:54 +01:00

Hi, my attention on this has been quite sporadic, but I currently have the contributors+commits check userscript installed for testing (more on that on the 4th paragraph).

API rate limits are my biggest concern with these extra checks. Only 60/hour is available without authentication (and we are without authentication), and after some thought, I think the script should only do 2 per repo at best to allow browsing at an average of 30 repos per hour (2 minutes per repo, which is probably enough time to check instruction files, give up on that repo, and then find another repo). The current script without anything else would already do 3 requests max, and probably 2 requests for the average project (1 for files at project root, 1 for .github/). A commits check (1 API) and a contributors check (1 API) would already increase that to 4 API requests per repo if nothing was found (and it was commits, contributors, then files).

As for mitigating that issue, I'm not sure. One idea I have is to allow for configuration to enable/disable checks (and probably default to contributors then files, commits not default on because pretty volatile), something that I'll implement after this PR. Another is adding support for authenticating with GitHub tokens for 5000/hour, but even with 1000/hour it's 16 requests max per 60 repo per hour, which should be plenty. But that does require having a GitHub account, and identifies you to GitHub.

Now, I've been testing the contributors+commits checking script. I've just left it installed since the last time I tried working on this PR. I have used it and have managed to catch this: github.com/Futsch1/medTimer@1423c75d It does look like a true positive to me, but it wasn't something I was expecting (github.

Should I update this PR or make a new one? I can also try to disentangle the contributor code from the commit stuff so they can be two clean separate PRs.

Do whatever I guess. You could force-push this PR with both checks and also disentangle them somewhat, doesn't have to be separate PRs.

And an actual review the contributors+commits check thing:

• Maybe good idea to move the whole multiple checks thing tying in with Promise.resolve into its own async function, instead of just adding on top of what already exists
• Add links to commits or commit hash references when something commits is found

I probably have more things, but I think I forgot them. So I'm sending this comment now.

Oh and thank you for all your work on this so far!

Hi, my attention on this has been quite sporadic, but I currently have the contributors+commits check userscript installed for testing (more on that on the 4th paragraph). API rate limits are my biggest concern with these extra checks. Only 60/hour is available without authentication (and we are without authentication), and after some thought, I think the script should only do 2 per repo at best to allow browsing at an average of 30 repos per hour (2 minutes per repo, which is probably enough time to check instruction files, give up on that repo, and then find another repo). The current script without anything else would already do 3 requests max, and probably 2 requests for the average project (1 for files at project root, 1 for `.github/`). A commits check (1 API) and a contributors check (1 API) would already increase that to 4 API requests per repo if nothing was found (and it was commits, contributors, then files). As for mitigating that issue, I'm not sure. One idea I have is to allow for configuration to enable/disable checks (and probably default to contributors then files, commits not default on because pretty volatile), something that I'll implement after this PR. Another is adding support for authenticating with GitHub tokens for 5000/hour, but even with 1000/hour it's 16 requests *max* per 60 repo per hour, which should be plenty. But that does require having a GitHub account, and identifies you to GitHub. Now, I've been testing the contributors+commits checking script. I've just left it installed since the last time I tried working on this PR. I have used it and have managed to catch this: https://github.com/Futsch1/medTimer/commit/1423c75d It does look like a true positive to me, but it wasn't something I was expecting (github. > Should I update this PR or make a new one? I can also try to disentangle the contributor code from the commit stuff so they can be two clean separate PRs. Do whatever I guess. You could force-push this PR with both checks and also disentangle them somewhat, doesn't have to be separate PRs. And an actual review the contributors+commits check thing: - Maybe good idea to move the whole multiple checks thing tying in with [`Promise.resolve`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/resolve) into its own async function, instead of just adding on top of what already exists - Add links to commits or commit hash references when something commits is found I probably have more things, but I think I forgot them. So I'm sending this comment now. Oh and thank you for all your work on this so far!

jacksonchen666 commented 2026年01月05日 02:57:56 +01:00

Pingy? Would it be fine if I take over this PR? (I also have a bunch of small code changes I kinda want to do, so I feel like I should probably do all of that instead of relaying all requested changes to you)

Pingy? Would it be fine if I take over this PR? (I also have a bunch of small code changes I kinda want to do, so I feel like I should probably do all of that instead of relaying all requested changes to you)

0x4261756D commented 2026年01月05日 23:48:56 +01:00

Hi, sorry for the late reply. You can take over the PR no problem, would you like it in the current state or should I push the commits+contributors branch onto it? Either is fine by me.

Hi, sorry for the late reply. You can take over the PR no problem, would you like it in the current state or should I push the commits+contributors branch onto it? Either is fine by me.

jacksonchen666 commented 2026年01月08日 04:57:23 +01:00

It appears that I may have made a comment which was never sent. Either way, commits+contributors stuff please

It appears that I may have made a comment which was never sent. Either way, commits+contributors stuff please

0x4261756D commented 2026年01月08日 05:50:03 +01:00

Alrighty, done.

Alrighty, done.