• Doing what I can to protect and perpetuate US public science data.

  • Joined on 2025年05月07日
tripleshrimp pushed to main at tripleshrimp/sciop-cli 2025年10月03日 01:02:06 +02:00
6b5cbbe3b4 Merge pull request 'added scrub filter to tarballs to maintain privacy as default' (#34) from tripleshrimp/sciop-cli:tripleshrimp-tarball-scrub into main
8b5f86cdfb fix spacing
09c85b1fee added scrub filter to tarballs to maintain privacy as default
Compare 3 commits »
tripleshrimp pushed to tripleshrimp-tarball-scrub at tripleshrimp/sciop-cli 2025年10月03日 00:50:03 +02:00
8b5f86cdfb fix spacing
tripleshrimp created pull request Safeguarding/sciop-cli#34 2025年10月03日 00:38:19 +02:00
added scrub filter to tarballs to maintain privacy as default
tripleshrimp created pull request tripleshrimp/sciop-cli#1 2025年10月03日 00:35:52 +02:00
added scrub filter to tarballs to maintain privacy as default
tripleshrimp created branch tripleshrimp-tarball-scrub in tripleshrimp/sciop-cli 2025年10月03日 00:32:37 +02:00
tripleshrimp pushed to tripleshrimp-tarball-scrub at tripleshrimp/sciop-cli 2025年10月03日 00:32:37 +02:00
09c85b1fee added scrub filter to tarballs to maintain privacy as default
tripleshrimp pushed to main at tripleshrimp/sciop-cli 2025年10月03日 00:01:00 +02:00
b99b3dec5d
 changelog
014c10a9e5 Merge pull request 'Add "pack"/"unpack"' (#21) from tripleshrimp/sciop-cli:pack-datasets into main
dfd94e98eb Merge branch 'main' into pack-datasets
c63c251645 Merge pull request 'decrease weight of v1 overhead in hybrid torrents' (#33) from decrease-overhead-weight into main
bbfe02a07d
 changelog
Compare 30 commits »
tripleshrimp pushed to main at tripleshrimp/torrent-ranger 2025年08月28日 03:08:12 +02:00
1328df0e38 Update README.md
tripleshrimp pushed to main at tripleshrimp/torrent-ranger 2025年08月28日 02:22:21 +02:00
74a90e7314 Update README.md
tripleshrimp created repository tripleshrimp/torrent-ranger 2025年08月28日 01:37:00 +02:00
tripleshrimp commented on pull request Safeguarding/sciop-scraping#25 2025年08月12日 08:40:01 +02:00
Add shrimptools NPR spider

The atc functions are for the alternate web pages like this:

https://www.npr.org/programs/all-things-considered/archive

Not wired in right now but there in case we need them

tripleshrimp commented on pull request Safeguarding/sciop-cli#21 2025年07月09日 09:12:32 +02:00
Add "pack"/"unpack"

ok this actually runs now lol

tripleshrimp pushed to pack-datasets at tripleshrimp/sciop-cli 2025年07月09日 09:10:27 +02:00
1abdda54c9 actual fix
tripleshrimp commented on pull request Safeguarding/sciop-cli#21 2025年07月09日 08:19:12 +02:00
Add "pack"/"unpack"

oops hold on got distracted and forgot a couple test runs

tripleshrimp commented on pull request Safeguarding/sciop-cli#21 2025年07月09日 04:16:47 +02:00
Add "pack"/"unpack"

OK I made a few changes just based on the main proposed fixes

tripleshrimp pushed to pack-datasets at tripleshrimp/sciop-cli 2025年07月09日 04:14:31 +02:00
b80a447fdb expanded cases for depth and leaf, new fixture for heuristic
tripleshrimp commented on pull request Safeguarding/sciop-cli#21 2025年07月09日 03:14:33 +02:00
Add "pack"/"unpack"

.... oh my god. Duh. Not sure how that didn't click for me. In that case we could just allow for a -1, and then throw in the skip_threshold check for the depth mode, to consolidate. Leaf mode...

tripleshrimp commented on pull request Safeguarding/sciop-cli#21 2025年07月09日 03:10:07 +02:00
Add "pack"/"unpack"

In our current implementation, we select the exact level to compress, then compress every folder at that exact level. Therefore, everything below that level will be included in the archives...

tripleshrimp commented on pull request Safeguarding/sciop-cli#21 2025年07月01日 06:36:52 +02:00
Add "pack"/"unpack"

OK I think these test should cover our desired cases, let me know if there's anything I need to change/resolve

tripleshrimp pushed to pack-datasets at tripleshrimp/sciop-cli 2025年07月01日 06:34:20 +02:00
6d93f90781 updates to test_pack, also slight change to pack for system file skipping