creating api key on data-gov-in

• register and login
• go to my account section(logged In) and
• just click on create API key; there you have it
• this key can be used for any datasset
• to see how to use it look at

important open data websites

• India
• USA
• All Open Data Sites or this
• Visit any site using above link.

avoid headers problem

• some websites like amazon use systems that can detect bot/automated(basically not through a web-browser, rather done programmatically) requests,
• to avoid them we can use headers parameter in out get request
• to get your user-agent visit
• or to learn more about headers go to requests docs and also look into network-inspection tab to know more about them for a particular website
• we also can use slenium/helium automation but that is resource heavy even with headless running
• Notes:
  • we also can use other user agents like for other web-browsers(safari,firefox etc) and try what sort of info we get
  • you also can trying to rotate through different user-agents when facing problem your usual user-agent; might help when there is some IP blocks etc