-
Notifications
You must be signed in to change notification settings - Fork 1.4k
-
After successfully bypassing Cloudflare, the page takes a total of 90-120 seconds to load. The contents I need shows up within the first 20 seconds. I have a large number of pages to process, in the neighborhood of 450k pages. Is there a way to access the contents before the page fully loads?
Beta Was this translation helpful? Give feedback.
All reactions
There are ways of accessing the page sooner:
--pls="none" / pls="none" --> Set pageLoadStrategy to "none": This strategy causes Selenium to return immediately after the initial HTML content is fully received by the browser.
--sjw / sjw=True --> Skip JS Waits, such as wait_for_ready_state_complete().
With those combined, you'll have access to the page quickly, but you may need to add waits on top of that if things are too fast.
Replies: 2 comments 2 replies
-
There are ways of accessing the page sooner:
--pls="none" / pls="none" --> Set pageLoadStrategy to "none": This strategy causes Selenium to return immediately after the initial HTML content is fully received by the browser.
--sjw / sjw=True --> Skip JS Waits, such as wait_for_ready_state_complete().
With those combined, you'll have access to the page quickly, but you may need to add waits on top of that if things are too fast.
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 2
-
Thanks!
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 1
-
SJW not work!
I try like this:
import seleniumbase as SB
driver = SB.Driver(browser='Chrome', headless=False, uc=True, locale_code="en", block_images=False, pls="none", sjw=True)
And get error:
Driver() got an unexpected keyword argument 'sjw'
Beta Was this translation helpful? Give feedback.
All reactions
-
Skipping JS waits ("sjw") is for the SB() format only. Not the Driver() format.
Beta Was this translation helpful? Give feedback.