Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

CDP Mode - Patch 26 #3441

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mdmintz merged 10 commits into master from cdp-mode-patch-26
Jan 21, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions examples/cdp_mode/ReadMe.md
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,21 @@

--------

<!-- YouTube View --><a href="https://www.youtube.com/watch?v=Mr90iQmNsKM"><img src="https://github.com/user-attachments/assets/91e7ff7b-d155-4ba9-b17b-b097825fcf42" title="SeleniumBase on YouTube" width="350" /></a>
<!-- YouTube View --><a href="https://www.youtube.com/watch?v=Mr90iQmNsKM"><img src="https://github.com/user-attachments/assets/91e7ff7b-d155-4ba9-b17b-b097825fcf42" title="SeleniumBase on YouTube" width="320" /></a>
<p>(<b><a href="https://www.youtube.com/watch?v=Mr90iQmNsKM">Watch the CDP Mode tutorial on YouTube! ▶️</a></b>)</p>

--------

<!-- YouTube View --><a href="https://www.youtube.com/watch?v=vt2zsdiNh3U"><img src="https://github.com/user-attachments/assets/82ab2715-727e-4d09-9314-b8905795dc43" title="SeleniumBase on YouTube" width="350" /></a>
<!-- YouTube View --><a href="https://www.youtube.com/watch?v=vt2zsdiNh3U"><img src="https://github.com/user-attachments/assets/82ab2715-727e-4d09-9314-b8905795dc43" title="SeleniumBase on YouTube" width="320" /></a>
<p>(<b><a href="https://www.youtube.com/watch?v=vt2zsdiNh3U">Watch "Hacking websites with CDP" on YouTube! ▶️</a></b>)</p>

--------

<!-- YouTube View --><a href="https://www.youtube.com/watch?v=gEZhTfaIxHQ"><img src="https://github.com/user-attachments/assets/656977e1-5d66-4d1c-9eec-0aaa41f6522f" title="SeleniumBase on YouTube" width="320" /></a>
<p>(<b><a href="https://www.youtube.com/watch?v=gEZhTfaIxHQ">Watch "Web-Scraping with GitHub Actions" on YouTube! ▶️</a></b>)</p>

--------

👤 <b translate="no">UC Mode</b> avoids bot-detection by first disconnecting WebDriver from the browser at strategic times, calling special <code>PyAutoGUI</code> methods to bypass CAPTCHAs (as needed), and finally reconnecting the <code>driver</code> afterwards so that WebDriver actions can be performed again. Although this approach works for bypassing simple CAPTCHAs, more flexibility is needed for bypassing bot-detection on websites with advanced protection. (That's where <b translate="no">CDP Mode</b> comes in.)

🐙 <b translate="no">CDP Mode</b> is based on <a href="https://github.com/HyperionGray/python-chrome-devtools-protocol" translate="no">python-cdp</a>, <a href="https://github.com/HyperionGray/trio-chrome-devtools-protocol" translate="no">trio-cdp</a>, and <a href="https://github.com/ultrafunkamsterdam/nodriver" translate="no">nodriver</a>. <code>trio-cdp</code> is an early implementation of <code>python-cdp</code>, and <code>nodriver</code> is a modern implementation of <code>python-cdp</code>. (Refactored <code>Python-CDP</code> code is imported from <a href="https://github.com/mdmintz/MyCDP" translate="no">MyCDP</a>.)
Expand Down
11 changes: 11 additions & 0 deletions examples/cdp_mode/raw_glassdoor.py
View file Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
from seleniumbase import SB

with SB(uc=True, test=True, ad_block=True) as sb:
url = "https://www.glassdoor.com/Reviews/index.htm"
sb.activate_cdp_mode(url)
sb.uc_gui_click_captcha()
sb.highlight('[data-test="global-nav-glassdoor-logo"]')
sb.highlight('[data-test="site-header-companies"]')
sb.highlight('[data-test="search-button"]')
sb.highlight('[data-test="sign-in-button"]')
sb.highlight('[data-test="company-search-autocomplete"]')
9 changes: 9 additions & 0 deletions examples/raw_invisible_captcha.py
View file Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
from seleniumbase import SB

with SB(uc=True, test=True, incognito=True) as sb:
url = "https://seleniumbase.io/apps/invisible_recaptcha"
sb.activate_cdp_mode(url)
sb.sleep(1)
sb.assert_element("img#captcha-success", timeout=3)
sb.set_messenger_theme(location="top_left")
sb.post_message("SeleniumBase wasn't detected", duration=3)
9 changes: 0 additions & 9 deletions examples/raw_recaptcha.py
View file Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,5 @@
from seleniumbase import SB

with SB(uc=True, test=True, incognito=True) as sb:
url = "https://seleniumbase.io/apps/recaptcha"
sb.activate_cdp_mode(url)
sb.sleep(1)
sb.uc_gui_handle_captcha() # Try with TAB + SPACEBAR
sb.assert_element("img#captcha-success", timeout=3)
sb.set_messenger_theme(location="top_left")
sb.post_message("SeleniumBase wasn't detected", duration=3)

with SB(uc=True, test=True, incognito=True) as sb:
url = "https://seleniumbase.io/apps/recaptcha"
sb.activate_cdp_mode(url)
Expand Down
2 changes: 1 addition & 1 deletion examples/uc_cdp_events.py
View file Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from rich.pretty import pprint
from seleniumbase import BaseCase
BaseCase.main(__name__, __file__, "--uc", "--uc-cdp", "-s")
BaseCase.main(__name__, __file__, "--uc", "--uc-cdp")


class CDPTests(BaseCase):
Expand Down
2 changes: 1 addition & 1 deletion examples/verify_undetected.py
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Some sites use scripts to detect Selenium, and then block you.
To evade detection, add --uc as a pytest command-line option."""
from seleniumbase import BaseCase
BaseCase.main(__name__, __file__, "--uc", "-s")
BaseCase.main(__name__, __file__, "--uc")


class UndetectedTest(BaseCase):
Expand Down
8 changes: 4 additions & 4 deletions help_docs/uc_mode.md
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,22 @@

---

<!-- YouTube View --><a href="https://www.youtube.com/watch?v=5dMFI3e85ig"><img src="http://img.youtube.com/vi/5dMFI3e85ig/0.jpg" title="SeleniumBase on YouTube" width="350" /></a>
<!-- YouTube View --><a href="https://www.youtube.com/watch?v=5dMFI3e85ig"><img src="http://img.youtube.com/vi/5dMFI3e85ig/0.jpg" title="SeleniumBase on YouTube" width="320" /></a>
<p>(<b><a href="https://www.youtube.com/watch?v=5dMFI3e85ig">Watch the 1st UC Mode tutorial on YouTube! ▶️</a></b>)</p>

----

<!-- YouTube View --><a href="https://www.youtube.com/watch?v=2pTpBtaE7SQ"><img src="http://img.youtube.com/vi/2pTpBtaE7SQ/0.jpg" title="SeleniumBase on YouTube" width="350" /></a>
<!-- YouTube View --><a href="https://www.youtube.com/watch?v=2pTpBtaE7SQ"><img src="http://img.youtube.com/vi/2pTpBtaE7SQ/0.jpg" title="SeleniumBase on YouTube" width="320" /></a>
<p>(<b><a href="https://www.youtube.com/watch?v=2pTpBtaE7SQ">Watch the 2nd UC Mode tutorial on YouTube! ▶️</a></b>)</p>

----

<!-- YouTube View --><a href="https://www.youtube.com/watch?v=-EpZlhGWo9k"><img src="http://img.youtube.com/vi/-EpZlhGWo9k/0.jpg" title="SeleniumBase on YouTube" width="350" /></a>
<!-- YouTube View --><a href="https://www.youtube.com/watch?v=-EpZlhGWo9k"><img src="http://img.youtube.com/vi/-EpZlhGWo9k/0.jpg" title="SeleniumBase on YouTube" width="320" /></a>
<p>(<b><a href="https://www.youtube.com/watch?v=-EpZlhGWo9k">Watch the 3rd UC Mode tutorial on YouTube! ▶️</a></b>)</p>

----

<!-- YouTube View --><a href="https://www.youtube.com/watch?v=Mr90iQmNsKM"><img src="http://img.youtube.com/vi/Mr90iQmNsKM/0.jpg" title="SeleniumBase on YouTube" width="350" /></a>
<!-- YouTube View --><a href="https://www.youtube.com/watch?v=Mr90iQmNsKM"><img src="http://img.youtube.com/vi/Mr90iQmNsKM/0.jpg" title="SeleniumBase on YouTube" width="320" /></a>
<p>(<b><a href="https://www.youtube.com/watch?v=Mr90iQmNsKM">Watch the 4th UC Mode tutorial on YouTube! ▶️</a></b>)</p>

----
Expand Down
4 changes: 2 additions & 2 deletions integrations/docker/ReadMe.md
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@ https://docs.docker.com/engine/install/

docker build -t seleniumbase .

If running on an Apple M1/M2 Mac, use this instead:
**(NOTE) - If running on an Apple M1/M2 Mac, use this instead:**

docker build --platform linux/amd64 -t seleniumbase .

M1/M2 Mac users should also see [StackOverflow.com/a/76586216/7058266](https://stackoverflow.com/a/76586216/7058266) to **Enable Rosetta in Docker Desktop**. (Otherwise **you will** encounter errors like this when Chrome tries to launch: `"Chrome failed to start: crashed."`)
**M1/M2 Mac users** should also see [StackOverflow.com/a/76586216/7058266](https://stackoverflow.com/a/76586216/7058266) to **Enable Rosetta in Docker Desktop**. (Otherwise **you will** encounter errors like this when Chrome tries to launch: `"Chrome failed to start: crashed."`)

#### 4. Run [the example test](https://github.com/seleniumbase/SeleniumBase/blob/master/examples/my_first_test.py) with Chrome inside your Docker: (Once the test completes after a few seconds, you'll automatically exit the Docker shell)

Expand Down
2 changes: 1 addition & 1 deletion mkdocs_build/requirements.txt
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ lxml==5.3.0
pyquery==2.0.1
readtime==3.0.0
mkdocs==1.6.1
mkdocs-material==9.5.49
mkdocs-material==9.5.50
mkdocs-exclude-search==0.6.6
mkdocs-simple-hooks==0.1.5
mkdocs-material-extensions==1.3.1
8 changes: 5 additions & 3 deletions requirements.txt
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@ attrs>=24.3.0
certifi>=2024年12月14日
exceptiongroup>=1.2.2
websockets~=13.1;python_version<"3.9"
websockets>=14.1;python_version>="3.9"
filelock>=3.16.1
websockets>=14.2;python_version>="3.9"
filelock~=3.16.1;python_version<"3.9"
filelock>=3.17.0;python_version>="3.9"
fasteners>=0.19
mycdp>=1.1.0
pynose>=1.5.3
Expand Down Expand Up @@ -41,7 +42,8 @@ trio==0.28.0;python_version>="3.9"
trio-websocket==0.11.1
wsproto==1.2.0
websocket-client==1.8.0
selenium==4.27.1
selenium==4.27.1;python_version<"3.9"
selenium==4.28.0;python_version>="3.9"
cssselect==1.2.0
sortedcontainers==2.4.0
execnet==2.1.1
Expand Down
2 changes: 1 addition & 1 deletion seleniumbase/__version__.py
View file Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
# seleniumbase package
__version__ = "4.33.15"
__version__ = "4.34.0"
Loading
Loading

AltStyle によって変換されたページ (->オリジナル) /