Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

A Python/Django project : social media data scrapping

Notifications You must be signed in to change notification settings

JerryTom121/Scrapology

Folders and files

NameName
Last commit message
Last commit date

Latest commit

History

15 Commits

Repository files navigation

Scrapology : Social Media Data Scrapping

alt text

A REST API build with Django 1.9.11 + Django REST Framework (Python 3.5.2). This API provide a set of JSON web services to scrap then store/fetch data from main social networks.

Twitter [In Progress]

  1. User Profile [Done]

  2. User Tweets [Done]

    • Download the latest 3420 tweets of a given Twitter user identified by his username (handle). Twitter only allows access to a users most recent 3240 tweets through its API. I used Tweepy which is Python wrapper for Twitter API.

    • Require that the user profile is scrapped and saved in the DB first, deletes the corresponding tweets in the DB if they exist, scrap the latest ones and save in the DB.

      URL: http://localhost:8000/twusers/Username/tweets [Method POST]

    • Retrieve the user tweets from the DB if they exist

      URL: http://localhost:8000/twusers/Username/tweets [Method GET]

Instagram [ToDo]

About

A Python/Django project : social media data scrapping

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

AltStyle によって変換されたページ (->オリジナル) /