License: Apache 2.0 pytest GitHub release (latest by date) Downloads Downloads GitHub all releases GitHub top language GitHub issues GitHub pull requests
- Introduction
- Pre-requisites
- Installation and Downloads
- Commands
- Docs
- Bug Tracker and Support
- License and Copyright
- Author and Thanks
Weblinks, It get all links from a given website and we can apply filters on top of it to get desired results, when you are good you can start downloading them with -d flag.
In the library we are supporting plain webpages, authentication based webpages, proxy, authentication on proxy, etc.
We also support storing the config in either local/global configuration, for best practise we suggest not to store your password, you get a prompt on runtime wher you can provide it.
A system need to support curl commands and python3
pip install weblinks
weblinks --help
usage: weblinks [-h] [-w WEB] [-s SUBSTRING] [-e EXT] [-d] [-u USERNAME]
[-p PASSWORD] [-g] [-l] [-v] [--proxy PROXY]
[--proxy-username PROXY_USERNAME]
[--proxy-password PROXY_PASSWORD] [--version]
optional arguments:
-h, --help show this help message and exit
-w WEB, --web WEB the website
-s SUBSTRING, --substring SUBSTRING
the sub-string in the links
-e EXT, --ext EXT file extention
-d, --download download links
-u USERNAME, --username USERNAME
web login username
-p PASSWORD, --password PASSWORD
web login password
-g, --global global configuration
-l, --local local configuration
-v, --verbosity
--proxy PROXY proxy address
--proxy-username PROXY_USERNAME
proxy username
--proxy-password PROXY_PASSWORD
proxy password
--version weblinks version
Weblinks usage
To see current lib. version
weblinks --version
# weblinks version: 2.0To see python file from given url
weblinks --web https://www.python.org/ftp/python/3.8.13/ --substring Python # INFO | 2022年07月23日 16:23:33,603 | run :117 | links found # Python-3.8.13.tar.xz # Python-3.8.13.tar.xz.asc # Python-3.8.13.tgz # Python-3.8.13.tgz.asc
Still wanted to filter add file extention
weblinks --web https://www.python.org/ftp/python/3.8.13/ --substring Python --ext .tgz # INFO | 2022年07月23日 16:23:33,603 | run :117 | links found # Python-3.8.13.tgz
Start download, listed links are good
weblinks --web https://www.python.org/ftp/python/3.8.13/ --substring Python --ext .tgz -d # INFO | 2022年07月23日 16:25:34,807 | run :117 | links found # Python-3.8.13.tgz # INFO | 2022年07月23日 16:25:34,807 | run :124 | start download: Python-3.8.13.tgz # INFO | 2022年07月23日 16:25:34,807 | utils :58 | downloading: Python-3.8.13.tgz # INFO | 2022年07月23日 16:25:36,849 | run :126 | completed: Python-3.8.13.tgz
For authentication
weblinks --web <url> --substring <sub> --username <kirankotari> --password <xxxxxx> --ext .tgz # Note: don't add --password, it will ask dynamically
For verbose add -v
weblinks --web <url> --substring <sub> --username <kirankotari> --ext .tgz -v
To store config add --local or --global respectively
weblinks --local --web <url> --username <kirankotari> --ext .tgz
For proxy
weblinks --proxy <ip>:<port> --web <url> --substring <sub>
For proxy with authentication
weblinks --proxy <ip>:<port> --web <url> --substring <sub> --proxy-username <proxy user> --proxy-password <proxy password>
Adding web authentication on top of proxy auth.
weblinks --proxy <ip>:<port> --web <url> --substring <sub> --proxy-username <proxy user> --proxy-password <proxy password> --username <user>
- Please report any suggestions, bug reports, or annoyances with weblinks through the Github bug tracker. If you're having problems with general python issues, consider searching for a solution on Stack Overflow.
- If you can't find a solution for your problem or need more help, you can ask a question.
- weblinks is licensed Apache 2.0 2022
Weblinks was developed by Kiran Kumar Kotari