1
0
Fork
You've already forked academia-preserver
0
Tool for preserving academic papers from Academia.edu through automated downloading
  • Python 92%
  • Shell 4.6%
  • HTML 3.4%
2025年02月13日 08:05:04 +01:00
cdc_archiver added a code.gov repo cloner 2025年02月05日 23:44:26 +01:00
code_gov added a code.gov repo cloner 2025年02月05日 23:44:26 +01:00
data_dot_gov quick and dirty scraper for data.gov 2025年02月01日 17:13:41 -05:00
data_ed_gov added a ED datasets bash script 2025年02月13日 08:04:55 +01:00
general_archiver added a ED datasets bash script 2025年02月13日 08:04:55 +01:00
nlm_nih_downloader changes to implementation; JPG download works, TIFF not, movie download starts playback in background, audio download not working 2025年02月10日 11:23:12 +00:00
urls added a ED datasets bash script 2025年02月13日 08:04:55 +01:00
.gitignore added a code.gov repo cloner 2025年02月05日 23:44:26 +01:00
LICENSE multi keyword 2025年01月21日 14:20:01 -05:00
main.py save to file and stuff 2025年01月28日 17:22:53 -05:00
readme.md save states 2025年01月21日 14:53:05 -05:00

Academia.edu Paper Preserver

A robust Python tool for preserving academic papers from Academia.edu through automated downloading. This tool allows researchers and archivists to bulk download papers based on keywords, with features for resuming interrupted downloads and managing large collections.

Purpose

This tool was created to help:

  • Preserve academic research that might become unavailable
  • Create local archives of research papers for offline access
  • Batch download papers related to specific research topics
  • Ensure research accessibility and preservation

Features

  • Multiple Keyword Search: Download papers from multiple research areas simultaneously
  • Resume Capability: Safely interrupt and resume downloads
  • Progress Tracking: Visual progress bars for both overall and individual download progress
  • Smart Naming: Organized filename structure including author, title, and keywords
  • User-Friendly Interface: Clear prompts and progress indicators
  • State Preservation: Maintains download state for recovery from interruptions

Installation

  1. Clone the repository:
git clone https://github.com/micr0-dev/academia-preserver.git
cd academia-preserver
  1. Install required packages:
pip install requests tqdm rich aiohttp

Usage

  1. Run the script:
python main.py
  1. Follow the interactive prompts:
    • Enter search keywords (comma-separated)
    • Specify output directory
    • Choose maximum number of papers to download

Example Input:

Enter search keywords: transgender, queer theory, gender studies
Enter output directory: my_papers
Maximum number of papers to download: all

Resuming Downloads

If a download is interrupted, you can resume it:

  1. Run the script again
  2. Choose 'yes' when asked about resuming
  3. The download will continue from where it left off

File Organization

Downloaded papers are saved with the following naming convention:

AuthorName-PaperTitle-Keyword-UniqueID.extension

Example:

John_Smith-Gender_Theory_Analysis-transgender-a1b2c3d4.pdf

Important Notes

  • Be mindful of Academia.edu's terms of service
  • Consider rate limiting and server load
  • Large downloads might take significant time
  • Ensure adequate disk space for downloads
  • Check your internet connection stability for large downloads

Error Handling

The script handles various error conditions:

  • Network interruptions
  • Server timeouts
  • Invalid responses
  • File system issues

If an error occurs, the script will:

  1. Save the current state
  2. Display error information
  3. Allow resuming from the last successful download

Contributing

Contributions are welcome! Please feel free to submit pull requests with improvements or bug fixes.

License

This project is licensed under the OVERWORKED LICENSE (OWL) v1.0. See the LICENSE file for details.

Disclaimer

This tool is for research and archival purposes. Users are responsible for ensuring compliance with Academia.edu's terms of service and respecting copyright restrictions.