|
1 | | -# Oxylabs’ Residential Proxies integration with Selenium |
2 | | - |
3 | | -[<img src="https://img.shields.io/static/v1?label=&message=Python&color=brightgreen" />](https://github.com/topics/python) [<img src="https://img.shields.io/static/v1?label=&message=Selenium&color=orange" />](https://github.com/topics/selenium) [<img src="https://img.shields.io/static/v1?label=&message=Web-Scraping&color=yellow" />](https://github.com/topics/web-scraping) [<img src="https://img.shields.io/static/v1?label=&message=Rotating%20Proxies&color=blueviolet" />](https://github.com/topics/rotating-proxies) |
4 | | - |
5 | | -## Requirements |
6 | | - |
7 | | -For the integration to work, you'll need to install [Selenium Wire](https://github.com/wkeeling/selenium-wire) to extend Selenium’s |
8 | | -Python bindings as implementing proxies that require authentication using default Selenium module complicates the process too much. |
9 | | - |
10 | | -You can do it using `pip` command: |
11 | | -```bash |
12 | | -pip install selenium-wire |
13 | | -``` |
14 | | -Another required package is `webdriver-manager`. It's a package that simplifies the management of binary drivers for different browsers, |
15 | | -so you don't need to manually download a new version of a web driver after each update. Visit the [official project directory](https://pypi.org/project/webdriver-manager/) |
16 | | -on pypi to find out more information. |
17 | | - |
18 | | -You can install the following using `pip` as well: |
19 | | -```bash |
20 | | -pip install webdriver-manager |
21 | | -``` |
22 | | -Required version of Python: `Python 3.5` (or higher) |
23 | | - |
24 | | -## Proxy Authentication |
25 | | - |
26 | | -For proxies to work, you'll need to specify your account credentials inside the [main.py](https://github.com/oxylabs/selenium-proxy-integration/blob/main/main.py) file. |
27 | | -```python |
28 | | -USERNAME = "your_username" |
29 | | -PASSWORD = "your_password" |
30 | | -ENDPOINT = "pr.oxylabs.io:7777" |
31 | | -``` |
32 | | -Adjust the `your_username` and `your_password` value fields with the username and password of |
33 | | -your Oxylabs account. |
34 | | - |
35 | | -## Testing Proxy Connection |
36 | | - |
37 | | -To see if the proxy is working, try visiting [ip.oxylabs.io](https://ip.oxylabs.io) <br>If everything is working correctly, it will return an IP address of a proxy that you're using. |
38 | | -```python |
39 | | -try: |
40 | | - driver.get("https://ip.oxylabs.io/") |
41 | | - return f'\nYour IP is: {re.search(r"[0-9].{2,}", driver.page_source).group()}' |
42 | | -finally: |
43 | | - driver.quit() |
44 | | -``` |
45 | | - |
46 | | -## Full Code |
47 | | -```python |
48 | | -import re |
49 | | -from typing import Optional |
50 | | - |
51 | | -from seleniumwire import webdriver |
52 | | -# A package to have a chromedriver always up-to-date. |
53 | | -from webdriver_manager.chrome import ChromeDriverManager |
54 | | - |
55 | | -USERNAME = "your_username" |
56 | | -PASSWORD = "your_password" |
57 | | -ENDPOINT = "pr.oxylabs.io:7777" |
58 | | - |
59 | | - |
60 | | -def chrome_proxy(user: str, password: str, endpoint: str): |
61 | | - wire_options = { |
62 | | - "proxy": { |
63 | | - "http": f"http://{user}:{password}@{endpoint}", |
64 | | - "https": f"http://{user}:{password}@{endpoint}", |
65 | | - } |
66 | | - } |
67 | | - |
68 | | - return wire_options |
69 | | - |
70 | | - |
71 | | -def execute_driver(): |
72 | | - options = webdriver.ChromeOptions() |
73 | | - options.headless = True |
74 | | - proxies = chrome_proxy(USERNAME, PASSWORD, ENDPOINT) |
75 | | - driver = webdriver.Chrome( |
76 | | - ChromeDriverManager().install(), options=options, seleniumwire_options=proxies |
77 | | - ) |
78 | | - try: |
79 | | - driver.get("https://ip.oxylabs.io/") |
80 | | - return f'\nYour IP is: {re.search(r"[0-9].{2,}", driver.page_source).group()}' |
81 | | - finally: |
82 | | - driver.quit() |
83 | | - |
84 | | - |
85 | | -if __name__ == "__main__": |
86 | | - print(execute_driver()) |
87 | | -``` |
88 | | -If you're having any trouble integrating proxies with Selenium and this guide didn't help you - feel free to contact Oxylabs customer support at support@oxylabs.io. |
| 1 | +# Oxylabs’ Residential Proxies integration with Selenium |
| 2 | + |
| 3 | +[<img src="https://img.shields.io/static/v1?label=&message=Python&color=brightgreen" />](https://github.com/topics/python) [<img src="https://img.shields.io/static/v1?label=&message=Selenium&color=orange" />](https://github.com/topics/selenium) [<img src="https://img.shields.io/static/v1?label=&message=Web-Scraping&color=yellow" />](https://github.com/topics/web-scraping) [<img src="https://img.shields.io/static/v1?label=&message=Rotating%20Proxies&color=blueviolet" />](https://github.com/topics/rotating-proxies) |
| 4 | + |
| 5 | +## Requirements |
| 6 | + |
| 7 | +For the integration to work, you'll need to install |
| 8 | +[Selenium Wire](https://github.com/wkeeling/selenium-wire) |
| 9 | +to extend Selenium’s Python bindings as implementing proxies |
| 10 | +that require authentication using default Selenium module |
| 11 | +complicates the process too much. |
| 12 | + |
| 13 | +You can do it using `pip` command: |
| 14 | +```bash |
| 15 | +pip install selenium-wire |
| 16 | +``` |
| 17 | + |
| 18 | +Another recommended package is `webdriver-manager`. It simplifies the management |
| 19 | +of binary drivers for different browsers, so you don't need to manually download |
| 20 | +a new version of a web driver after each update. Visit the |
| 21 | +[official project directory](https://pypi.org/project/webdriver-manager/) on pypi to |
| 22 | +find out more information. |
| 23 | + |
| 24 | +You can install the following using `pip` as well: |
| 25 | +```bash |
| 26 | +pip install webdriver-manager |
| 27 | +``` |
| 28 | + |
| 29 | +Required version of Python: `Python 3.5` (or higher) |
| 30 | + |
| 31 | +## Proxy Authentication |
| 32 | + |
| 33 | +For proxies to work, you'll need to specify your account credentials inside |
| 34 | +the [main.py](https://github.com/oxylabs/selenium-proxy-integration/blob/main/main.py) file. |
| 35 | + |
| 36 | +```python |
| 37 | +USERNAME = "your_username" |
| 38 | +PASSWORD = "your_password" |
| 39 | +ENDPOINT = "pr.oxylabs.io:7777" |
| 40 | +``` |
| 41 | + |
| 42 | +Adjust the `your_username` and `your_password` value fields with the username and password of |
| 43 | +your Oxylabs account. |
| 44 | + |
| 45 | +## Testing Proxy Connection |
| 46 | + |
| 47 | +To see if the proxy is working, try visiting [ip.oxylabs.io](https://ip.oxylabs.io) |
| 48 | +If everything is working correctly, it will return an IP address of a proxy that you're using. |
| 49 | + |
| 50 | +## Full Code |
| 51 | +```python |
| 52 | +import re |
| 53 | +from typing import Optional |
| 54 | + |
| 55 | +from seleniumwire import webdriver |
| 56 | +# A package to have a chromedriver always up-to-date. |
| 57 | +from webdriver_manager.chrome import ChromeDriverManager |
| 58 | + |
| 59 | +USERNAME = "your_username" |
| 60 | +PASSWORD = "your_password" |
| 61 | +ENDPOINT = "pr.oxylabs.io:7777" |
| 62 | + |
| 63 | + |
| 64 | +def chrome_proxy(user: str, password: str, endpoint: str) -> dict: |
| 65 | + wire_options = { |
| 66 | + "proxy": { |
| 67 | + "http": f"http://{user}:{password}@{endpoint}", |
| 68 | + "https": f"http://{user}:{password}@{endpoint}", |
| 69 | + } |
| 70 | + } |
| 71 | + |
| 72 | + return wire_options |
| 73 | + |
| 74 | + |
| 75 | +def execute_driver(): |
| 76 | + options = webdriver.ChromeOptions() |
| 77 | + options.headless = True |
| 78 | + proxies = chrome_proxy(USERNAME, PASSWORD, ENDPOINT) |
| 79 | + driver = webdriver.Chrome( |
| 80 | + ChromeDriverManager().install(), options=options, seleniumwire_options=proxies |
| 81 | + ) |
| 82 | + try: |
| 83 | + driver.get("https://ip.oxylabs.io/") |
| 84 | + return f'\nYour IP is: {re.search(r"[0-9].{2,}", driver.page_source).group()}' |
| 85 | + finally: |
| 86 | + driver.quit() |
| 87 | + |
| 88 | + |
| 89 | +if __name__ == "__main__": |
| 90 | + print(execute_driver()) |
| 91 | +``` |
| 92 | + |
| 93 | +If you're having any trouble integrating proxies with Selenium and this guide didn't help |
| 94 | +you - feel free to contact Oxylabs customer support at support@oxylabs.io. |
0 commit comments