|
| 1 | +# Dev.to Scrapper |
| 2 | +Running this Script would allow the user to scrape any number of articles from [dev.to](https://dev.to/) from any category as per the user's choice |
| 3 | + |
| 4 | +## Setup instructions |
| 5 | +In order to run this script, you need to have Python and pip installed on your system. After you're done installing Python and pip, run the following command from your terminal to install the requirements from the same folder (directory) of the project. |
| 6 | +``` |
| 7 | +pip install -r requirements.txt |
| 8 | +``` |
| 9 | +As this script uses selenium, you will need to install the chrome webdriver from [this link](https://sites.google.com/a/chromium.org/chromedriver/downloads) |
| 10 | + |
| 11 | +Once the installation is complete, you need to add the webdriver in the path in the following line: |
| 12 | +``` |
| 13 | +driver = webdriver.Chrome('<Path to chrome webdriver >') |
| 14 | +``` |
| 15 | + |
| 16 | +After satisfying all the requirements for the project, Open the terminal in the project folder and run |
| 17 | +``` |
| 18 | +python scraper.py |
| 19 | +``` |
| 20 | +or |
| 21 | +``` |
| 22 | +python3 scraper.py |
| 23 | +``` |
| 24 | +depending upon the python version. Make sure that you are running the command from the same virtual environment in which the required modules are installed. |
| 25 | + |
| 26 | +## Output |
| 27 | +The user needs to enter Category and Number of articles |
| 28 | + |
| 29 | + |
| 30 | +The scraped pdf files get saved in the folder in which the script is run |
| 31 | + |
| 32 | + |
| 33 | +## Author |
| 34 | +[Ayush Jain](https://github.com/Ayushjain2205) |
0 commit comments