-
Notifications
You must be signed in to change notification settings - Fork 60
Normalize data schemas, enrich datasets, and fix API handler quality #65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
99 changes: 90 additions & 9 deletions
README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,12 +1,93 @@ | ||
| # ISRO 🚀 API | ||
| # ISRO API | ||
|
|
||
| Open Source API for Launched Spacecrafts & Rockets data of ISRO | ||
| Open Source API for ISRO spacecraft, launcher, and mission data. | ||
|
|
||
| ## API End-Points | ||
| Spacecraft: [/api/spacecrafts](https://isro.vercel.app/api/spacecrafts) | ||
| - Launchers: [/api/launchers](https://isro.vercel.app/api/launchers) | ||
| - Customer Satellites: [/api/customer_satellites](https://isro.vercel.app/api/customer_satellites) | ||
| - Centres: [/api/centres](https://isro.vercel.app/api/centres) | ||
| ### Mission End-Points | ||
| - Spacecraft Missions: [/api/spacecraft_missions](https://isro.vercel.app/api/spacecraft_missions) | ||
| **Live:** [isro.vercel.app](https://isro.vercel.app) | ||
|
|
||
| ## API Endpoints | ||
|
|
||
| | Endpoint | Description | Records | | ||
| |----------|-------------|---------| | ||
| | [`/api/spacecrafts`](https://isro.vercel.app/api/spacecrafts) | All ISRO spacecrafts with launch date, vehicle, orbit type, and status | 113 | | ||
| | [`/api/launchers`](https://isro.vercel.app/api/launchers) | Launch vehicles classified by family (SLV, ASLV, PSLV, GSLV, LVM-3) | 81 | | ||
| | [`/api/customer_satellites`](https://isro.vercel.app/api/customer_satellites) | Foreign satellites launched by ISRO, with ISO dates and normalized countries | 75 | | ||
| | [`/api/centres`](https://isro.vercel.app/api/centres) | ISRO research centres across India | 44 | | ||
| | [`/api/spacecraft_missions`](https://isro.vercel.app/api/spacecraft_missions) | Detailed mission data: mass, power, orbit, payloads, stabilization, status | 65 | | ||
|
|
||
| ## Response Format | ||
|
|
||
| All endpoints return JSON with a consistent wrapper: | ||
|
|
||
| ```json | ||
| { | ||
| "spacecrafts": [ | ||
| { | ||
| "id": 1, | ||
| "name": "Aryabhata", | ||
| "launch_date": "1975年04月19日", | ||
| "launch_vehicle": "C-1 Intercosmos", | ||
| "mission_type": "Scientific/ Experimental", | ||
| "orbit_type": null, | ||
| "mass_kg": 360.0, | ||
| "status": "decommissioned" | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| ### Spacecraft Missions Schema | ||
|
|
||
| | Field | Type | Description | | ||
| |-------|------|-------------| | ||
| | `id` | number | Sequential ID | | ||
| | `name` | string | Spacecraft name | | ||
| | `mission_type` | string\|null | Mission purpose (Communication, Remote Sensing, etc.) | | ||
| | `launch_date` | string\|null | ISO 8601 date (YYYY-MM-DD) | | ||
| | `launch_site` | string\|null | Launch facility | | ||
| | `launch_vehicle` | string\|null | Rocket used | | ||
| | `orbit` | string\|null | Orbit description | | ||
| | `orbit_type` | string\|null | Classified: LEO, SSO, GEO, Lunar, Interplanetary, Failed | | ||
| | `altitude_km` | number\|null | Orbital altitude in km | | ||
| | `inclination_deg` | number\|null | Orbital inclination in degrees | | ||
| | `mass_kg` | number\|null | Lift-off mass in kg | | ||
| | `power_watts` | number\|null | Onboard power in watts | | ||
| | `mission_life` | string\|null | Designed mission lifetime | | ||
| | `status` | string | active, decommissioned, failed, or unknown | | ||
| | `payloads` | string\|null | Onboard instruments/payloads | | ||
| | `stabilization` | string\|null | Attitude control system | | ||
| | `propulsion` | string\|null | Propulsion system | | ||
|
|
||
| ## Data Normalization | ||
|
|
||
| Raw data is scraped from [isro.gov.in](https://www.isro.gov.in) and normalized using `scripts/normalize_data.py`. The pipeline: | ||
|
|
||
| - Parses 15+ date formats into ISO 8601 | ||
| - Resolves 9+ field name variants for mass (e.g., `weight`, `lift-off_mass`, `spacecraft_mass`) into `mass_kg` | ||
| - Extracts wattage from complex power strings (e.g., "15 Sq.m Solar Array generating 1360W" -> 1360) | ||
| - Classifies orbit types (LEO, SSO, GEO, Lunar, etc.) | ||
| - Infers mission status (active/decommissioned/failed) from launch date + mission life | ||
| - Normalizes country names and fixes field casing inconsistencies | ||
| - Merges fresh scraper output with existing data (idempotent) | ||
|
|
||
| ```bash | ||
| # Run the normalization pipeline | ||
| python scripts/normalize_data.py | ||
| ``` | ||
|
|
||
| ## Tech Stack | ||
|
|
||
| - **Runtime:** Node.js (Vercel Serverless Functions) | ||
| - **Data:** Static JSON, zero npm dependencies | ||
| - **Data Pipeline:** Python 3 (BeautifulSoup for scraping, custom normalization) | ||
| - **Hosting:** [Vercel](https://vercel.com) | ||
|
|
||
| ## Contributing | ||
|
|
||
| 1. Fork the repository | ||
| 2. To update data: edit files in `data/` or run the scraper and normalization pipeline | ||
| 3. To add endpoints: create a new file in `api/` (Vercel auto-routes it) | ||
| 4. Submit a pull request | ||
|
|
||
| ## License | ||
|
|
||
| MIT |
17 changes: 4 additions & 13 deletions
api/centres.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,20 +1,11 @@ | ||
| const fs = require("fs"); | ||
| const centres = require("../data/centres.json"); | ||
|
|
||
| // Load the centers data from a JSON file | ||
| let centers = require("../data/centres.json"); | ||
|
|
||
| // Export an async function to handle requests | ||
| module.exports = async (req, res) => { | ||
| try { | ||
| // Send the centers data as the response | ||
| res.send(centers); | ||
| res.setHeader("Content-Type", "application/json"); | ||
| res.send(centres); | ||
| } catch (error) { | ||
| // If there is an error, send a 500 status code and the error message and response | ||
| res.status(500); | ||
| const response = error.response || {}; | ||
| res.send({ | ||
| message: error.message, | ||
| response, | ||
| }); | ||
| res.send({ error: error.message }); | ||
| } | ||
| }; |
17 changes: 4 additions & 13 deletions
api/customer_satellites.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,20 +1,11 @@ | ||
| const fs = require("fs"); | ||
| const customerSatellites = require("../data/customer_satellites.json"); | ||
|
|
||
| // Load the customer satellite data from a JSON file | ||
| let launchers = require("../data/customer_satellites.json"); | ||
|
|
||
| // Export an async function to handle requests | ||
| module.exports = async (req, res) => { | ||
| try { | ||
| // Send the customer satellite data as the response | ||
| res.send(launchers); | ||
| res.setHeader("Content-Type", "application/json"); | ||
| res.send(customerSatellites); | ||
| } catch (error) { | ||
| // If there is an error, send a 500 status code and the error message and response | ||
| res.status(500); | ||
| const response = error.response || {}; | ||
| res.send({ | ||
| message: error.message, | ||
| response, | ||
| }); | ||
| res.send({ error: error.message }); | ||
| } | ||
| }; |
25 changes: 16 additions & 9 deletions
api/index.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,15 +1,22 @@ | ||
| // Export an async function to handle requests | ||
| const endpoints = { | ||
| spacecrafts: "/api/spacecrafts", | ||
| launchers: "/api/launchers", | ||
| customer_satellites: "/api/customer_satellites", | ||
| centres: "/api/centres", | ||
| spacecraft_missions: "/api/spacecraft_missions", | ||
| }; | ||
|
|
||
| module.exports = async (req, res) => { | ||
| try { | ||
| // Send a message as the response | ||
| res.send("<pre> ISRO API v0.1.0 </pre>"); | ||
| } catch (error) { | ||
| // If there is an error, send a 500 status code and the error message and response | ||
| res.status(500); | ||
| const response = error.response || {}; | ||
| res.setHeader("Content-Type", "application/json"); | ||
| res.send({ | ||
| message: error.message, | ||
| response, | ||
| name: "ISRO API", | ||
| version: "1.0.0", | ||
| description: "Open Source API for ISRO spacecraft, launcher, and mission data", | ||
| endpoints, | ||
| }); | ||
| } catch (error) { | ||
| res.status(500); | ||
| res.send({ error: error.message }); | ||
| } | ||
| }; |
15 changes: 3 additions & 12 deletions
api/launchers.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,20 +1,11 @@ | ||
| const fs = require("fs"); | ||
| const launchers = require("../data/launchers.json"); | ||
|
|
||
| // Load the data for the available launchers from a JSON file | ||
| let launchers = require("../data/launchers.json"); | ||
|
|
||
| // Export an async function to handle requests for the list of available launchers | ||
| module.exports = async (req, res) => { | ||
| try { | ||
| // Send the list of available launchers as the response | ||
| res.setHeader("Content-Type", "application/json"); | ||
| res.send(launchers); | ||
| } catch (error) { | ||
| // If there is an error, send a 500 status code and the error message and response | ||
| res.status(500); | ||
| const response = error.response || {}; | ||
| res.send({ | ||
| message: error.message, | ||
| response, | ||
| }); | ||
| res.send({ error: error.message }); | ||
| } | ||
| }; |
21 changes: 8 additions & 13 deletions
api/spacecraft_missions.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,16 +1,11 @@ | ||
| const fs = require("fs"); | ||
|
|
||
| let launchers = require("../data/spacecraft_missions.json"); | ||
| const spacecraftMissions = require("../data/spacecraft_missions.json"); | ||
|
|
||
| module.exports = async (req, res) => { | ||
| try { | ||
| res.send(launchers); | ||
| } catch (error) { | ||
| res.status(500); | ||
| const response = error.response || {}; | ||
| res.send({ | ||
| message: error.message, | ||
| response, | ||
| }); | ||
| } | ||
| try { | ||
| res.setHeader("Content-Type", "application/json"); | ||
| res.send(spacecraftMissions); | ||
| } catch (error) { | ||
| res.status(500); | ||
| res.send({ error: error.message }); | ||
| } | ||
| }; |
15 changes: 3 additions & 12 deletions
api/spacecrafts.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,20 +1,11 @@ | ||
| const fs = require("fs"); | ||
| const spacecrafts = require("../data/spacecrafts.json"); | ||
|
|
||
| // Load the data for the available spacecrafts from a JSON file | ||
| let spacecrafts = require("../data/spacecrafts.json"); | ||
|
|
||
| // Export an async function to handle requests for the list of available spacecrafts | ||
| module.exports = async (req, res) => { | ||
| try { | ||
| // Send the list of available spacecrafts as the response | ||
| res.setHeader("Content-Type", "application/json"); | ||
| res.send(spacecrafts); | ||
| } catch (error) { | ||
| // If there is an error, send a 500 status code and the error message and response | ||
| res.status(500); | ||
| const response = error.response || {}; | ||
| res.send({ | ||
| message: error.message, | ||
| response, | ||
| }); | ||
| res.send({ error: error.message }); | ||
| } | ||
| }; |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.