Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

File descriptors leak in undetected (UC) mode #3679

felipehertzer started this conversation in General
Discussion options

Hi,

I’m seeing a steady increase in open file descriptors whenever I activate UC mode (uc=True), leading eventually to:

filedescriptor out of range in select()

When UC mode is off the FD count remains flat.

When I use uc_open_with_reconnect and force a gc.collect() after teardown, the leak stops.

Below are three minimal repros—please let me know what else I can provide.

CDP

This script repeatedly opens a browser, calls activate_cdp_mode(...), then quits. You can see the number of FDs jump by ~8–10 each time, and connections in CLOSE_WAIT remain.

import os
import time
import psutil
from seleniumbase import SB
def _fd_exists(fd: int) -> bool:
 try:
 os.fstat(fd)
 return True
 except OSError:
 return False
def inspect_fds():
 p = psutil.Process()
 print(f"Count FD (psutil): {p.num_fds()}")
 connections = p.net_connections()
 if connections:
 print("Open connections:")
 for conn in connections:
 print(f" {conn}")
if __name__ == "__main__":
 print("Start")
 inspect_fds()
 print('Browser Running')
 for i in range(1, 4):
 print(f"\nIteration {i}")
 with SB(uc=True) as browser:
 browser.activate_cdp_mode("https://example.com")
 browser.sleep(1)
 time.sleep(1)
 inspect_fds()

Output:

Start
Count FD (psutil): 3
Browser Running
Iteration 1
Count FD (psutil): 11
Open connections:
 pconn(fd=5, family=<AddressFamily.AF_INET6: 30>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::1', port=51824), raddr=addr(ip='::1', port=51681), status='CLOSE_WAIT')
 pconn(fd=6, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=51743), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')
 pconn(fd=7, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=51750), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')
Iteration 2
Count FD (psutil): 17
Open connections:
 pconn(fd=5, family=<AddressFamily.AF_INET6: 30>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::1', port=51824), raddr=addr(ip='::1', port=51681), status='CLOSE_WAIT')
 pconn(fd=6, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=51743), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')
 pconn(fd=7, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=51750), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')
 pconn(fd=9, family=<AddressFamily.AF_INET6: 30>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::1', port=51968), raddr=addr(ip='::1', port=51837), status='CLOSE_WAIT')
 pconn(fd=17, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=51890), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')
 pconn(fd=18, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=51891), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')
Iteration 3
Count FD (psutil): 26
Open connections:
 pconn(fd=5, family=<AddressFamily.AF_INET6: 30>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::1', port=51824), raddr=addr(ip='::1', port=51681), status='CLOSE_WAIT')
 pconn(fd=6, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=51743), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')
 pconn(fd=7, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=51750), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')
 pconn(fd=9, family=<AddressFamily.AF_INET6: 30>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::1', port=51968), raddr=addr(ip='::1', port=51837), status='CLOSE_WAIT')
 pconn(fd=16, family=<AddressFamily.AF_INET6: 30>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::1', port=52119), raddr=addr(ip='::1', port=51976), status='CLOSE_WAIT')
 pconn(fd=17, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=51890), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')
 pconn(fd=18, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=51891), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')
 pconn(fd=27, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=52043), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')
 pconn(fd=28, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=52044), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')

UC without GC

Here we use uc_open_with_reconnect(...) instead of CDP. The leak still occurs, growing by ~1 FD per iteration.

import os
import time
import psutil
from seleniumbase import SB
def _fd_exists(fd: int) -> bool:
 try:
 os.fstat(fd)
 return True
 except OSError:
 return False
def inspect_fds():
 p = psutil.Process()
 print(f"Count FD (psutil): {p.num_fds()}")
 connections = p.net_connections()
 if connections:
 print("Open connections:")
 for conn in connections:
 print(f" {conn}")
if __name__ == "__main__":
 print("Start")
 inspect_fds()
 print('Browser Running')
 for i in range(1, 10):
 print(f"\nIteration {i}")
 with SB(uc=True) as browser:
 browser.uc_open_with_reconnect("https://example.com", 4)
 time.sleep(1)
 inspect_fds()

Output:

Start
Count FD (psutil): 3
Browser Running
Iteration 1
Count FD (psutil): 9
Open connections:
 pconn(fd=3, family=<AddressFamily.AF_INET6: 30>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::1', port=50953), raddr=addr(ip='::1', port=50847), status='CLOSE_WAIT')
Iteration 2
Count FD (psutil): 10
Open connections:
 pconn(fd=3, family=<AddressFamily.AF_INET6: 30>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::1', port=50953), raddr=addr(ip='::1', port=50847), status='CLOSE_WAIT')
 pconn(fd=5, family=<AddressFamily.AF_INET6: 30>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::1', port=51075), raddr=addr(ip='::1', port=50976), status='CLOSE_WAIT')
Iteration 3
Count FD (psutil): 11
Open connections:
 pconn(fd=3, family=<AddressFamily.AF_INET6: 30>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::1', port=50953), raddr=addr(ip='::1', port=50847), status='CLOSE_WAIT')
 pconn(fd=5, family=<AddressFamily.AF_INET6: 30>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::1', port=51075), raddr=addr(ip='::1', port=50976), status='CLOSE_WAIT')
 pconn(fd=6, family=<AddressFamily.AF_INET6: 30>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::1', port=51192), raddr=addr(ip='::1', port=51090), status='CLOSE_WAIT')

UC using GC

Adding a manual garbage collection after each teardown appears to reclaim all sockets—FD count stays constant. But it only works for uc_open_with_reconnect.

import gc
import os
import time
import psutil
from seleniumbase import SB
def _fd_exists(fd: int) -> bool:
 try:
 os.fstat(fd)
 return True
 except OSError:
 return False
def inspect_fds():
 p = psutil.Process()
 print(f"Count FD (psutil): {p.num_fds()}")
 connections = p.net_connections()
 if connections:
 print("Open connections:")
 for conn in connections:
 print(f" {conn}")
if __name__ == "__main__":
 print("Start")
 inspect_fds()
 print('Browser Running')
 for i in range(1, 4):
 print(f"\nIteration {i}")
 with SB(uc=True) as browser:
 browser.uc_open_with_reconnect("https://example.com", 4)
 time.sleep(1)
 gc.collect()
 inspect_fds()

Output:

Start
Count FD (psutil): 3
Browser Running
Iteration 1
Count FD (psutil): 6
Open connections:
 pconn(fd=3, family=<AddressFamily.AF_INET6: 30>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::1', port=49683), raddr=addr(ip='::1', port=49619), status='CLOSE_WAIT')
Iteration 2
Count FD (psutil): 6
Open connections:
 pconn(fd=5, family=<AddressFamily.AF_INET6: 30>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::1', port=49759), raddr=addr(ip='::1', port=49690), status='CLOSE_WAIT')
Iteration 3
Count FD (psutil): 6
Open connections:
 pconn(fd=3, family=<AddressFamily.AF_INET6: 30>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='::1', port=49996), raddr=addr(ip='::1', port=49776), status='CLOSE_WAIT')

Environment:
SeleniumBase version: e.g. 4.37.2
Python version: e.g. 3.13
OS: MacOS and Linux
Chrome: 135.0.7049.85
Chromedriver: matching Chrome
You must be logged in to vote

Replies: 7 comments 14 replies

Comment options

Do you know how to find specifically the code where the file descriptors leaks occur? The specific locations would be helpful. Much of UC Mode / CDP Mode is based on undetected-chromedriver / nodriver, which had several memory leaks: https://github.com/search?q=repo%3Aultrafunkamsterdam%2Fundetected-chromedriver+%22memory+leak%22&type=issues

You must be logged in to vote
0 replies
Comment options

Hello @mdmintz,

Thank you for your response. I spent several hours debugging and have identified a number of issues:

1. Session-based requests
I replaced all calls to requests.get with a with requests.Session() as session: context manager in both init.py and cdp.py.

Original

res = requests.get("http://127.0.0.1:9222", timeout=1)
resp = self._session.get(self.server_addr + uri)

Updated

https://github.com/felipehertzer/SeleniumBase/blob/f09035dcc138018d5cb208d6d65aba55929d64b7/seleniumbase/undetected/__init__.py#L139

https://github.com/felipehertzer/SeleniumBase/blob/f09035dcc138018d5cb208d6d65aba55929d64b7/seleniumbase/undetected/cdp.py#L132

You can review all of my changes here:
master...felipehertzer:SeleniumBase:fd-leak

https://github.com/felipehertzer/SeleniumBase/blob/f09035dcc138018d5cb208d6d65aba55929d64b7/seleniumbase/undetected/__init__.py#L570

With these adjustments, the leak in uc_open_with_reconnect no longer occurs (even without invoking the garbage collector), and the number of leaked file descriptors in activate_cdp_mode is greatly reduced.

2. Remaining leak in activate_cdp_mode
Despite the improvements above, two file descriptors (FD 6 and FD 7) still remain open when invoking activate_cdp_mode. Both originate from the start and get functions in seleniumbase/undetected/cdp_driver/browser.py:

async def start(self=None) -> Browser:

It appears that each of these functions establishes its own socket connection, which is never closed. Moreover, I have confirmed that the corresponding stop method (at line 569) is never invoked, which I believe is the underlying cause of the remaining leak:

  1. Diagnostic output
    Below is the output demonstrating the two unclosed sockets and their respective call stacks:

FD=6 and FD=7 are the two socks that do not close, and this is the trace of when each is called

[CONNECT] fd=6 → ('127.0.0.1', 9222)
 /test//.venv/lib/python3.13/site-packages/seleniumbase/fixtures/base_case.py:4881 in activate_cdp_mode
 /test//.venv/lib/python3.13/site-packages/seleniumbase/core/browser_launcher.py:5426 in <lambda>
 /test//.venv/lib/python3.13/site-packages/seleniumbase/core/browser_launcher.py:559 in uc_open_with_cdp_mode
 /opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py:712 in run_until_complete
 /opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py:683 in run_forever
 /opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py:2040 in _run_once
 /opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/events.py:89 in _run
 /test//.venv/lib/python3.13/site-packages/seleniumbase/undetected/cdp_driver/cdp_util.py:318 in start
 /test//.venv/lib/python3.13/site-packages/seleniumbase/undetected/cdp_driver/browser.py:122 in create
 /test//.venv/lib/python3.13/site-packages/seleniumbase/undetected/cdp_driver/browser.py:414 in start
 /test//.venv/lib/python3.13/site-packages/seleniumbase/undetected/cdp_driver/connection.py:396 in send
 /test//.venv/lib/python3.13/site-packages/seleniumbase/undetected/cdp_driver/connection.py:267 in aopen
 /test//.venv/lib/python3.13/site-packages/websockets/asyncio/client.py:541 in __await_impl__
 /test//.venv/lib/python3.13/site-packages/websockets/asyncio/client.py:467 in create_connection
 /opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py:1141 in create_connection
 /opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py:1044 in _connect_sock
 /opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/selector_events.py:639 in sock_connect
 /opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/selector_events.py:649 in _sock_connect
 /test//tests/test.py:181 in connect
 /test//tests/test.py:191 in _print_stack

[NEW SOCKET] fd=7 family=<AddressFamily.AF_INET: 2> type=<SocketKind.SOCK_STREAM: 1>
 /test//tests/test.py:205 in <module>
 /test//.venv/lib/python3.13/site-packages/seleniumbase/fixtures/base_case.py:4881 in activate_cdp_mode
 /test//.venv/lib/python3.13/site-packages/seleniumbase/core/browser_launcher.py:5426 in <lambda>
 /test//.venv/lib/python3.13/site-packages/seleniumbase/core/browser_launcher.py:607 in uc_open_with_cdp_mode
 /opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py:712 in run_until_complete
 /opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py:683 in run_forever
 /opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py:2040 in _run_once
 /opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/events.py:89 in _run
 /test//.venv/lib/python3.13/site-packages/seleniumbase/undetected/cdp_driver/browser.py:294 in get
 /test//.venv/lib/python3.13/site-packages/seleniumbase/undetected/cdp_driver/connection.py:396 in send
 /test//.venv/lib/python3.13/site-packages/seleniumbase/undetected/cdp_driver/connection.py:267 in aopen
 /test//.venv/lib/python3.13/site-packages/websockets/asyncio/client.py:541 in __await_impl__
 /test//.venv/lib/python3.13/site-packages/websockets/asyncio/client.py:467 in create_connection
 /opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py:1141 in create_connection
 /opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py:1022 in _connect_sock
 /test//tests/test.py:175 in __init__
 /test//tests/test.py:191 in _print_stack
You must be logged in to vote
0 replies
Comment options

4.37.3 - https://github.com/seleniumbase/SeleniumBase/releases/tag/v4.37.3
(At least UC Mode without CDP Mode no longer has memory leaks. Thank you!)

CDP Mode memory improvements will have to be figured out later.
You must be logged in to vote
0 replies
Comment options

Hey @mdmintz,

Thank you for the update.

After running some additional tests I’ve traced the file‐descriptor leak to the Connection class. Although an aclose() coroutine exists, it’s never invoked. The aexit() method would normally trigger aclose(), but it only executes when Connection is used as an async context manager—which isn’t happening in the current workflow.

To prevent the leak it will need to make sure every connection is explicitly closed, either by:

  1. Refactoring usage to an async with Connection(...) as conn: pattern, or
  2. Calling await self.aclose() at the end of send() (and any other method that opens a connection).
You must be logged in to vote
0 replies
Comment options

Fixed in 4.37.4 - https://github.com/seleniumbase/SeleniumBase/releases/tag/v4.37.4

You must be logged in to vote
2 replies
Comment options

Thanks again for your help on this, @felipehertzer!
Comment options

That is great, it seems totally fixed, Thank you.
Comment options

Hey @mdmintz,

While deploying to production today I noticed that we’re still seeing a connection leak on the CDP side.

It appears to originate around

async def start(self=None) -> Browser:
From what I can tell, the flow calls start() and then get(), which opens two separate WebSocket connections. The recent patch only closes the most recently opened socket via sb_config._cdp_aclose = self.aclose, leaving the first one dangling. 
for i in range(1, 4):
 print(f"\nIteration {i}")
 with SB(
 uc=True,
 ) as browser:
 browser.activate_cdp_mode("https://bloomberg.com/news/articles/2024-12-05/barclays-promotes-116-staffers-across-bank-to-managing-director")
 browser.sleep(5)
 browser.cdp.get_page_source()
 inspect_fds()
Iteration 1
Count FD (psutil): 9
Open connections:
 pconn(fd=6, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=65386), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')
Iteration 2
Count FD (psutil): 13
Open connections:
 pconn(fd=6, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=65386), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')
 pconn(fd=19, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=49386), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')
Iteration 3
Count FD (psutil): 17
Open connections:
 pconn(fd=6, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=65386), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')
 pconn(fd=19, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=49386), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')
 pconn(fd=23, family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, laddr=addr(ip='127.0.0.1', port=49763), raddr=addr(ip='127.0.0.1', port=9222), status='CLOSE_WAIT')

/selenium/.venv/lib/python3.13/site-packages/seleniumbase/fixtures/base_case.py:4882 in activate_cdp_mode
/selenium/.venv/lib/python3.13/site-packages/seleniumbase/core/browser_launcher.py:5425 in <lambda>
/selenium/.venv/lib/python3.13/site-packages/seleniumbase/core/browser_launcher.py:558 in uc_open_with_cdp_mode
/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py:712 in run_until_complete
/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py:683 in run_forever
/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py:2040 in _run_once
/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/events.py:89 in _run
/selenium/.venv/lib/python3.13/site-packages/seleniumbase/undetected/cdp_driver/cdp_util.py:318 in start
/selenium/.venv/lib/python3.13/site-packages/seleniumbase/undetected/cdp_driver/browser.py:122 in create
/selenium/.venv/lib/python3.13/site-packages/seleniumbase/undetected/cdp_driver/browser.py:412 in start
/selenium/.venv/lib/python3.13/site-packages/seleniumbase/undetected/cdp_driver/connection.py:397 in send
/selenium/.venv/lib/python3.13/site-packages/seleniumbase/undetected/cdp_driver/connection.py:268 in aopen
/selenium/.venv/lib/python3.13/site-packages/websockets/asyncio/client.py:541 in __await_impl__
/selenium/.venv/lib/python3.13/site-packages/websockets/asyncio/client.py:467 in create_connection
/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py:1141 in create_connection
/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py:1044 in _connect_sock
/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/selector_events.py:639 in sock_connect
/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/selector_events.py:649 in _sock_connect
/selenium/tests/test.py:176 in connect
/selenium/tests/test.py:186 in _print_stack
You must be logged in to vote
3 replies
Comment options

@felipehertzer Did you run on 4.36.7? I saw no open socket connections with:

import shutil
import os
import subprocess
import psutil
from seleniumbase import SB
# from seleniumbase import config as sb_config
def get_open_fds():
 """
 Returns the number of open file descriptors for the current process.
 """
 lsof_path = shutil.which("lsof")
 if lsof_path is None:
 raise NotImplementedError("lsof is not available on this system.")
 try:
 process = subprocess.run(
 [lsof_path, "-w", "-Ff", "-p", str(os.getpid())],
 capture_output=True,
 text=True,
 check=True,
 )
 output = process.stdout
 fds = [
 line for line in output.split(os.linesep)
 if line.startswith("f") and line[1:].isdigit()
 ]
 return len(fds)
 except subprocess.CalledProcessError as e:
 raise RuntimeError(f"Error executing lsof: {e}") from e
def inspect_fds():
 p = psutil.Process()
 # print(f"Count FD (psutil): {p.num_fds()}")
 connections = p.net_connections()
 if connections:
 # print("Open connections:")
 for conn in connections:
 print(f" {conn}")
if __name__ == "__main__":
 print("Start")
 inspect_fds()
 # print('Browser Running')
 # print(get_open_fds())
 for i in range(1, 4):
 print(f"\nIteration {i}")
 with SB(uc=True) as browser:
 browser.activate_cdp_mode("https://example.com")
 # browser.uc_open_with_reconnect("https://example.com", 1)
 # print(get_open_fds())
 inspect_fds()

If you remove the following line, then the socket connections remain open:

loop.run_until_complete(sb_config._cdp_aclose())
Comment options

Yes, I am using the latest version. Interestingly, the issue only occurs when I add these lines. I not sure if it has anything to do with the time spent to render the website.

browser.sleep(5)
browser.cdp.get_page_source()
Comment options

Not sure, but just sleeping and getting the page source isn't causing any issue:

import shutil
import os
import subprocess
import psutil
from seleniumbase import SB
# from seleniumbase import config as sb_config
def get_open_fds():
 """
 Returns the number of open file descriptors for the current process.
 """
 lsof_path = shutil.which("lsof")
 if lsof_path is None:
 raise NotImplementedError("lsof is not available on this system.")
 try:
 process = subprocess.run(
 [lsof_path, "-w", "-Ff", "-p", str(os.getpid())],
 capture_output=True,
 text=True,
 check=True,
 )
 output = process.stdout
 fds = [
 line for line in output.split(os.linesep)
 if line.startswith("f") and line[1:].isdigit()
 ]
 return len(fds)
 except subprocess.CalledProcessError as e:
 raise RuntimeError(f"Error executing lsof: {e}") from e
def inspect_fds():
 p = psutil.Process()
 # print(f"Count FD (psutil): {p.num_fds()}")
 connections = p.net_connections()
 if connections:
 # print("Open connections:")
 for conn in connections:
 print(f" {conn}")
if __name__ == "__main__":
 print("Start")
 inspect_fds()
 # print('Browser Running')
 # print(get_open_fds())
 for i in range(1, 4):
 print(f"\nIteration {i}")
 with SB(uc=True) as sb:
 sb.activate_cdp_mode("https://example.com")
 sb.sleep(1)
 source = sb.get_page_source()
 # sb.uc_open_with_reconnect("https://example.com", 1)
 # print(get_open_fds())
 inspect_fds()
Comment options

Hey @mdmintz,

I apologise for the delayed response. I was only able to replicate this issue with heavy websites as Bloomberg and WSJ, because they use a lot of iframes and background services. I’ve completed my investigation and implemented the following fixes:

  1. Tab connections not closing
    Complex pages with multiple iframes and background‐sync tasks result in each iframe/background task being treated as a separate Tab instance, and thus opening its own WebSocket connection that was never closed. To address this, I:

    • Added an __aexit__ method to the Tab class.
    • Refactored all Tab() instantiations to use the async context manager pattern (async with Tab(...)), ensuring every connection opened by a tab is properly closed on exit.

  2. Browser vs. page CDP connections
    SeleniumBase’s CDP integration issues two DevTools requests—one for the browser and one for the page—but the previous patch only closed the page connection. I introduced:

    loop.run_until_complete(driver.cdp_base.connection.aclose())

    This explicitly closes the browser connection as well, eliminating the sock‐connection leak.

  3. File descriptor leak
    Even after fixing the CDP leaks, file descriptors were still accumulating. I discovered that explicitly closing the event loop prevents residual FDs from persisting. Adding:

    loop.close()

    at the end of the sequence stops the FD leaks entirely.

With these changes, both the connection and FD leaks no longer occur in my tests.

Replicate:

Add a print after this line:

if not self.websocket or self.websocket.state is State.CLOSED: 
print("aopen", self.websocket_url)

And another after this line

print("aclose", self.websocket_url)

Output:

aopen ws://127.0.0.1:9222/devtools/browser/8704fae5-0c58-4cde-9e71-3bc08db5d67c
aopen ws://127.0.0.1:9222/devtools/page/44DEB82983EC2934E27A533AF3B4B211
aclose ws://127.0.0.1:9222/devtools/page/44DEB82983EC2934E27A533AF3B4B211 -- ADDED VIA LAST PATCH

Add this print after this line

super().__init__(websocket_url, target, browser, **kwargs) 
print(f"Tab: {websocket_url}")

Output:

Tab: ws://127.0.0.1:9222/devtools/background_page/0795658581062DE1B5C7324A1CE59E1B
Tab: ws://127.0.0.1:9222/devtools/page/BE100F9470F4DFF9EAE6111D063528E9
Tab: ws://127.0.0.1:9222/devtools/iframe/D47CA9CF9CD6A1C7E4B960AA2B6C35D2
Tab: ws://127.0.0.1:9222/devtools/other/28ECCAD15EDE39A2715B6D62F47E3880
Tab: ws://127.0.0.1:9222/devtools/iframe/58B370B21BE44228935BC612331B847A
Tab: ws://127.0.0.1:9222/devtools/iframe/972656F2E27C0D14F8DC873594231CD7
Tab: ws://127.0.0.1:9222/devtools/iframe/A2104C041DE228411DF2D6C480545B35
Tab: ws://127.0.0.1:9222/devtools/iframe/E49C3A9A60EF0E9F51EFE28F0767FC29
Tab: ws://127.0.0.1:9222/devtools/iframe/139D3EFDB3A6682AF0EE518855BCE4CB
Tab: ws://127.0.0.1:9222/devtools/shared_storage_worklet/D2AADED9BDC6A64D9502D3E3CE031659
Tab: ws://127.0.0.1:9222/devtools/iframe/C62A699A188A55AED880137A4FC035AE
Tab: ws://127.0.0.1:9222/devtools/iframe/AD8821505E41C9AF384B995213AFD8B0
Tab: ws://127.0.0.1:9222/devtools/iframe/438D8D778866CC42742858F87EBDC06A
Tab: ws://127.0.0.1:9222/devtools/iframe/782456508D48CE8E79ACE870EA9DCDD5
Tab: ws://127.0.0.1:9222/devtools/service_worker/7E032A9235985AAA3B39ACB5F7D50A40
Tab: ws://127.0.0.1:9222/devtools/iframe/E65FA68B93899660A3CDB600A2A3AF1D
Tab: ws://127.0.0.1:9222/devtools/iframe/207E7ED64B68B047015D5173CC0148EE
Tab: ws://127.0.0.1:9222/devtools/iframe/C45D58802C43A3E2D15E3C5CAC76E360
Tab: ws://127.0.0.1:9222/devtools/worker/D72ADF0B8D1D5F4DC2C228F434ADCE75
Tab: ws://127.0.0.1:9222/devtools/iframe/880DF8BA73E2AE050D4B740F6935BE4E
Tab: ws://127.0.0.1:9222/devtools/iframe/C0A6CA93591AC401A7E5DFEACF4EC6EB
Tab: ws://127.0.0.1:9222/devtools/shared_storage_worklet/A14904B5C51F63FB5155F3314BBF1692
Tab: ws://127.0.0.1:9222/devtools/iframe/9CCE76ACCDD938D148314A1916FAD196
Tab: ws://127.0.0.1:9222/devtools/iframe/0CF877B5E7B8A56BEE8F4B68F6E3ECCB
Tab: ws://127.0.0.1:9222/devtools/iframe/AE28E35914D65A9BE56E9C41F0C18446
Tab: ws://127.0.0.1:9222/devtools/iframe/B38A9B8A798BC9BBC755A060265607FA
Tab: ws://127.0.0.1:9222/devtools/iframe/A6982A83029017435083711C08E9B34E

AFTER THE FIX:

aopen ws://127.0.0.1:9222/devtools/browser/ce99f11f-946b-47a1-acbb-dc3eaee1c710
aclose ws://127.0.0.1:9222/devtools/browser/ce99f11f-946b-47a1-acbb-dc3eaee1c710
aopen ws://127.0.0.1:9222/devtools/page/068CF0E77EC08B808B992D7015CBE11B
aclose ws://127.0.0.1:9222/devtools/page/068CF0E77EC08B808B992D7015CBE11B

My branch: master...felipehertzer:SeleniumBase:master

You must be logged in to vote
9 replies
Comment options

I added a few items into 4.37.8 that didn't cause issues.
(It doesn't resolve the leak, but it's less work from here.)
Comment options

Hey @mdmintz,

Thank you for your input.
I have updated the code again to ensure it checks all tabs, closes their connections, and then closes the browser connection for all drivers

master...felipehertzer:SeleniumBase:fd-leak-2

Comment options

Still seeing it with examples/cdp_mode/raw_priceline.py, and at random with other scripts.
Try from a full pull (there are merge conflicts from your branch, as it is behind).
Comment options

Hey @mdmintz,

I have fixed that issue, I have added a close connection when close the tab.

I have run the tests and only those failed:

FAILED examples/swag_labs_user_tests.py::SwagLabsTests::test_swag_labs_user_flows_1_problem_user - AssertionError: Sort Failed! Expecting "Test.allTheThings() T-Shirt" on top!
FAILED examples/test_deferred_asserts.py::DeferredAssertTests::test_deferred_asserts - Exception: 
FAILED examples/test_detect_404s.py::BrokenLinkTests::test_link_checking - AssertionError: Broken links detected:
FAILED examples/test_fail.py::FailingTests::test_find_army_of_robots_on_xkcd_desert_island - seleniumbase.common.exceptions.NoSuchElementException: Message: 
FAILED examples/test_suite.py::MyTestSuite::test_2 - seleniumbase.common.exceptions.TextNotVisibleException: 
FAILED examples/test_suite.py::MyTestSuite::test_4 - seleniumbase.common.exceptions.NoSuchElementException: Message: 
FAILED examples/test_todomvc.py::TodoMVC::test_todomvc_0_jquery - seleniumbase.common.exceptions.TextNotVisibleException: 
FAILED examples/time_limit_test.py::TimeLimitTests::test_runtime_limit_decorator - seleniumbase.common.exceptions.TimeoutException: 
FAILED examples/time_limit_test.py::TimeLimitTests::test_set_time_limit_method - seleniumbase.common.exceptions.TimeLimitExceededException: 
FAILED examples/visual_testing/test_layout_fail.py::VisualLayout_FixtureTests::test_python_home_change - seleniumbase.common.exceptions.VisualException: 
FAILED examples/visual_testing/test_layout_fail.py::VisualLayoutFailureTests::test_applitools_change - seleniumbase.common.exceptions.VisualException: 
FAILED examples/visual_testing/test_layout_fail.py::VisualLayoutFailureTests::test_xkcd_logo_change - seleniumbase.common.exceptions.VisualException: 
FAILED examples/youtube_search_test.py::YouTubeSearchTests::test_youtube_autocomplete_results - AssertionError: False is not true : Expected text "seleniumbase" not found in top results! Actual text was "selenium

master...felipehertzer:SeleniumBase:leak

Comment options

Merged in 4.37.10 - https://github.com/seleniumbase/SeleniumBase/releases/tag/v4.37.10
Thanks for your work on this!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet

AltStyle によって変換されたページ (->オリジナル) /