Hello,

Let me elaborate on my question.

browser = Driver(uc = True, headless = True)
browser.default_get(url) # Faster, but Selenium can be detected

I have recently used this method to parse 4 000 urls, which took 53 000 seconds with 10 core-threads. Looks a lot, 14 hours per iteration. 120 seconds per URL on average.

Earlier I used:

options = ChromeOptions()
options.add_argument("--headless=new")
options.page_load_strategy = 'eager'
browser = webdriver.Chrome(options = options)
browser.set_page_load_timeout(timeout_secs=10)
browser.get(url)

from pure Selenium. Which resluts in circa 20 times less time per the same amount of URLs visited, but with less successes opening the page.

Namely: 2 665 seconds per 4 545 URLs in 10-core parallelism, so 6 seconds per URL.

However, I am flexible here in tuning the timeout and strategy, so I can get less but quicker and vice versa.

So, why the UC method is much slower, and what exact strategy and timeout does it utilizes? Could you advice on the parameter set of the UC's default_get(url) to make it faster?