-
Notifications
You must be signed in to change notification settings - Fork 60
All issues are fixed and pertained to the latest changes.#64
All issues are fixed and pertained to the latest changes. #64soumyadipkarforma wants to merge 7 commits into
Conversation
- Converted 'isro_scrape/scraping.ipynb' to a robust Python script 'isro_scrape/scrape.py'. - Implemented data deduplication and key normalization. - Added a GitHub Actions workflow '.github/workflows/daily_scrape.yml' for daily midnight UTC updates. - Enhanced the scraper to clean mission names and technical details. - Updated 'api/spacecraft_missions.js' with improved comments and naming. - Added a direct link to spacecraft missions on 'index.html'. - Cleaned up redundant API endpoint files.
...r-and-automation Feature/modernize scraper and automation
AdityaAsopa
commented
Mar 12, 2026
Hey @soumyadipkarforma — the GitHub Actions workflows here look great, especially the daily scrape automation. This maps directly to a gap in the repo's data pipeline.
Worth flagging: my PR #65 also touches spacecraft_missions.json and api/spacecraft_missions.js as part of a data normalization effort. There will likely be a merge conflict on those files if either PR goes in first — just something for maintainers to be aware of when sequencing these.
The daily scrape automation in particular would pair well with the normalization pipeline I added in #65 (scripts/normalize_data.py) — if the scraper runs daily, the normalization step could be added to the workflow to keep the data consistently structured. Might be worth combining or referencing each other.
No description provided.