Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CDP Mode: Patch 42 #3603

Merged
merged 4 commits into from
Mar 12, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@

<p align="center"><a href="https://github.com/seleniumbase/SeleniumBase/"><img src="https://seleniumbase.github.io/cdn/img/super_logo_sb3.png" alt="SeleniumBase" title="SeleniumBase" width="350" /></a></p>


<p align="center" class="hero__title"><b>All-in-one Browser Automation Framework:<br />Web Crawling / Testing / Scraping / Stealth</b></p>

<p align="center"><a href="https://pypi.python.org/pypi/seleniumbase" target="_blank"><img src="https://img.shields.io/pypi/v/seleniumbase.svg?color=3399EE" alt="PyPI version" /></a> <a href="https://github.com/seleniumbase/SeleniumBase/releases" target="_blank"><img src="https://img.shields.io/github/v/release/seleniumbase/SeleniumBase.svg?color=22AAEE" alt="GitHub version" /></a> <a href="https://seleniumbase.io"><img src="https://img.shields.io/badge/docs-seleniumbase.io-11BBAA.svg" alt="SeleniumBase Docs" /></a> <a href="https://github.com/seleniumbase/SeleniumBase/actions" target="_blank"><img src="https://github.com/seleniumbase/SeleniumBase/workflows/CI%20build/badge.svg" alt="SeleniumBase GitHub Actions" /></a> <a href="https://discord.gg/EdhQTn3EyE" target="_blank"><img src="https://img.shields.io/badge/join-discord-infomational" alt="Join the SeleniumBase chat on Discord"/></a></p>
Expand Down Expand Up @@ -54,7 +53,9 @@
<br />
</p>

<p>SeleniumBase is the professional toolkit for web automation activities. Built for testing websites, bypassing CAPTCHAs, enhancing productivity, completing tasks, and scaling your business.</p>
<p align="center"><a href="https://trendshift.io/repositories/12493" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12493" alt="seleniumbase%2FSeleniumBase | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a></p>

<p>SeleniumBase is the professional toolkit for web automation. Built for testing websites, bypassing CAPTCHAs, completing tasks, and scaling your business.</p>

--------

Expand All @@ -81,7 +82,7 @@ with SB(test=True, uc=True) as sb:

> `python raw_google.py`

<a href="https://github.com/seleniumbase/SeleniumBase/blob/master/examples/raw_google.py"><img src="https://seleniumbase.github.io/cdn/gif/google_search.gif" alt="SeleniumBase Test" title="SeleniumBase Test" width="420" /></a>
<a href="https://github.com/seleniumbase/SeleniumBase/blob/master/examples/raw_google.py"><img src="https://seleniumbase.github.io/cdn/gif/google_search.gif" alt="SeleniumBase Test" title="SeleniumBase Test" width="480" /></a>

--------

Expand Down Expand Up @@ -1393,5 +1394,5 @@ pytest --reruns=1 --reruns-delay=1
<div><a href="https://seleniumbase.io"><img src="https://img.shields.io/badge/docs-seleniumbase.io-11BBAA.svg" alt="SeleniumBase Docs" /></a> <a href="https://github.com/seleniumbase/SeleniumBase/blob/master/LICENSE"><img src="https://img.shields.io/badge/license-MIT-22BBCC.svg" title="SeleniumBase" /></a></div>
<div><a href="https://github.com/seleniumbase/SeleniumBase"><img src="https://img.shields.io/badge/tested%20with-SeleniumBase-04C38E.svg" alt="Tested with SeleniumBase" /></a> <a href="https://github.com/seleniumbase/SeleniumBase/stargazers"><img src="https://img.shields.io/github/stars/seleniumbase/seleniumbase.svg?color=19A57B" title="Stargazers" /></a></div>
<div><a href="https://hellogithub.com/repository/c6be2d0f1969448697683d11a4ff915e" target="_blank"><img src="https://abroad.hellogithub.com/v1/widgets/recommend.svg?rid=c6be2d0f1969448697683d11a4ff915e&claim_uid=xcrm4p9j3d6JCO5&theme=small" alt="Featured|HelloGitHub" /></a> <a href="https://discord.gg/EdhQTn3EyE" target="_blank"><img src="https://img.shields.io/badge/join-discord-infomational" alt="Join the SeleniumBase chat on Discord"/></a> <a href="https://gitter.im/seleniumbase/SeleniumBase" target="_blank"><img src="https://img.shields.io/gitter/room/seleniumbase/SeleniumBase.svg" alt="Gitter chat"/></a></div>
<div><a href="https://pepy.tech/project/seleniumbase" target="_blank"><img src="https://static.pepy.tech/badge/seleniumbase" alt="SeleniumBase PyPI downloads" /></a> <img src="https://views.whatilearened.today/views/github/seleniumbase/SeleniumBase.svg" width="98px" height="20px" alt="Views" /></div>
<div><a href="https://pepy.tech/projects/seleniumbase?timeRange=threeMonths&category=version&includeCIDownloads=true&granularity=daily&viewType=line&versions=*" target="_blank"><img src="https://static.pepy.tech/badge/seleniumbase" alt="SeleniumBase PyPI downloads" /></a> <img src="https://views.whatilearened.today/views/github/seleniumbase/SeleniumBase.svg" width="98px" height="20px" alt="Views" /></div>
<div align="left"></div>
5 changes: 5 additions & 0 deletions examples/cdp_mode/ReadMe.md
Original file line number Diff line number Diff line change
Expand Up @@ -413,6 +413,7 @@ sb.cdp.send_keys(selector, text, timeout=None)
sb.cdp.press_keys(selector, text, timeout=None)
sb.cdp.type(selector, text, timeout=None)
sb.cdp.set_value(selector, text, timeout=None)
sb.cdp.submit(selector)
sb.cdp.evaluate(expression)
sb.cdp.js_dumps(obj_name)
sb.cdp.maximize()
Expand All @@ -438,6 +439,8 @@ sb.cdp.get_page_source()
sb.cdp.get_user_agent()
sb.cdp.get_cookie_string()
sb.cdp.get_locale_code()
sb.cdp.get_local_storage_item(key)
sb.cdp.get_session_storage_item(key)
sb.cdp.get_screen_rect()
sb.cdp.get_window_rect()
sb.cdp.get_window_size()
Expand All @@ -454,6 +457,8 @@ sb.cdp.get_element_attribute(selector, attribute)
sb.cdp.get_attribute(selector, attribute)
sb.cdp.get_element_html(selector)
sb.cdp.set_locale(locale)
sb.cdp.set_local_storage_item(key, value)
sb.cdp.set_session_storage_item(key, value)
sb.cdp.set_attributes(selector, attribute, value)
sb.cdp.gui_press_key(key)
sb.cdp.gui_press_keys(keys)
Expand Down
2 changes: 1 addition & 1 deletion mkdocs_build/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

regex>=2024.11.6
pymdown-extensions>=10.14.3
pipdeptree>=2.25.0
pipdeptree>=2.25.1
python-dateutil>=2.8.2
Markdown==3.7
click==8.1.8
Expand Down
7 changes: 4 additions & 3 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
pip>=25.0.1
packaging>=24.2
setuptools~=70.2;python_version<"3.10"
setuptools>=75.8.2;python_version>="3.10"
setuptools>=76.0.0;python_version>="3.10"
wheel>=0.45.1
attrs>=25.1.0
certifi>=2025.1.31
Expand All @@ -11,7 +11,7 @@ websockets>=15.0.1;python_version>="3.9"
filelock~=3.16.1;python_version<"3.9"
filelock>=3.17.0;python_version>="3.9"
fasteners>=0.19
mycdp>=1.1.0
mycdp>=1.1.1
pynose>=1.5.4
platformdirs>=4.3.6
typing-extensions>=4.12.2
Expand Down Expand Up @@ -44,7 +44,8 @@ wsproto==1.2.0
websocket-client==1.8.0
selenium==4.27.1;python_version<"3.9"
selenium==4.29.0;python_version>="3.9"
cssselect==1.2.0
cssselect==1.2.0;python_version<"3.9"
cssselect==1.3.0;python_version>="3.9"
sortedcontainers==2.4.0
execnet==2.1.1
iniconfig==2.0.0
Expand Down
2 changes: 1 addition & 1 deletion seleniumbase/__version__.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
# seleniumbase package
__version__ = "4.35.6"
__version__ = "4.35.7"
5 changes: 5 additions & 0 deletions seleniumbase/core/browser_launcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -662,6 +662,7 @@ def uc_open_with_cdp_mode(driver, url=None):
cdp.press_keys = CDPM.press_keys
cdp.type = CDPM.type
cdp.set_value = CDPM.set_value
cdp.submit = CDPM.submit
cdp.evaluate = CDPM.evaluate
cdp.js_dumps = CDPM.js_dumps
cdp.maximize = CDPM.maximize
Expand All @@ -670,6 +671,8 @@ def uc_open_with_cdp_mode(driver, url=None):
cdp.set_window_rect = CDPM.set_window_rect
cdp.reset_window_size = CDPM.reset_window_size
cdp.set_locale = CDPM.set_locale
cdp.set_local_storage_item = CDPM.set_local_storage_item
cdp.set_session_storage_item = CDPM.set_session_storage_item
cdp.set_attributes = CDPM.set_attributes
cdp.gui_press_key = CDPM.gui_press_key
cdp.gui_press_keys = CDPM.gui_press_keys
Expand Down Expand Up @@ -705,6 +708,8 @@ def uc_open_with_cdp_mode(driver, url=None):
cdp.get_user_agent = CDPM.get_user_agent
cdp.get_cookie_string = CDPM.get_cookie_string
cdp.get_locale_code = CDPM.get_locale_code
cdp.get_local_storage_item = CDPM.get_local_storage_item
cdp.get_session_storage_item = CDPM.get_session_storage_item
cdp.get_text = CDPM.get_text
cdp.get_title = CDPM.get_title
cdp.get_page_title = CDPM.get_title
Expand Down
34 changes: 34 additions & 0 deletions seleniumbase/core/sb_cdp.py
Original file line number Diff line number Diff line change
Expand Up @@ -955,6 +955,20 @@ def set_value(self, selector, text, timeout=None):
self.__slow_mode_pause_if_set()
self.loop.run_until_complete(self.page.sleep(0.025))

def submit(self, selector):
submit_script = (
"""elm = document.querySelector('%s');
const event = new KeyboardEvent("keydown", {
key: "Enter",
keyCode: 13,
code: "Enter",
which: 13,
bubbles: true
});
elm.dispatchEvent(event);""" % selector
)
self.loop.run_until_complete(self.page.evaluate(submit_script))

def evaluate(self, expression):
"""Run a JavaScript expression and return the result."""
expression = expression.strip()
Expand Down Expand Up @@ -1115,6 +1129,16 @@ def get_locale_code(self):
self.page.evaluate("navigator.language || navigator.languages[0]")
)

def get_local_storage_item(self, key):
js_code = """localStorage.getItem('%s');""" % key
with suppress(Exception):
return self.loop.run_until_complete(self.page.evaluate(js_code))

def get_session_storage_item(self, key):
js_code = """sessionStorage.getItem('%s');""" % key
with suppress(Exception):
return self.loop.run_until_complete(self.page.evaluate(js_code))

def get_screen_rect(self):
coordinates = self.loop.run_until_complete(
self.page.js_dumps("window.screen")
Expand Down Expand Up @@ -1302,6 +1326,16 @@ def set_locale(self, locale):
"""(Settings will take effect on the next page load)"""
self.loop.run_until_complete(self.page.set_locale(locale))

def set_local_storage_item(self, key, value):
js_code = """localStorage.setItem('%s','%s');""" % (key, value)
with suppress(Exception):
self.loop.run_until_complete(self.page.evaluate(js_code))

def set_session_storage_item(self, key, value):
js_code = """sessionStorage.setItem('%s','%s');""" % (key, value)
with suppress(Exception):
self.loop.run_until_complete(self.page.evaluate(js_code))

def set_attributes(self, selector, attribute, value):
"""This method uses JavaScript to set/update a common attribute.
All matching selectors from querySelectorAll() are used.
Expand Down
13 changes: 13 additions & 0 deletions seleniumbase/fixtures/base_case.py
Original file line number Diff line number Diff line change
Expand Up @@ -1164,6 +1164,9 @@ def submit(self, selector, by="css selector"):
"""Alternative to self.driver.find_element_by_*(SELECTOR).submit()"""
self.__check_scope()
selector, by = self.__recalculate_selector(selector, by)
if self.__is_cdp_swap_needed():
self.cdp.submit(selector)
return
element = self.wait_for_element_clickable(
selector, by=by, timeout=settings.SMALL_TIMEOUT
)
Expand Down Expand Up @@ -8800,6 +8803,9 @@ def set_local_storage_item(self, key, value):
self.__check_scope()
if not self.__is_valid_storage_url():
raise WebDriverException("Local Storage is not available here!")
if self.__is_cdp_swap_needed():
self.cdp.set_local_storage_item(key, value)
return
self.execute_script(
"window.localStorage.setItem('{}', '{}');".format(key, value)
)
Expand All @@ -8808,6 +8814,8 @@ def get_local_storage_item(self, key):
self.__check_scope()
if not self.__is_valid_storage_url():
raise WebDriverException("Local Storage is not available here!")
if self.__is_cdp_swap_needed():
return self.cdp.get_local_storage_item(key)
return self.execute_script(
"return window.localStorage.getItem('{}');".format(key)
)
Expand Down Expand Up @@ -8859,6 +8867,9 @@ def set_session_storage_item(self, key, value):
self.__check_scope()
if not self.__is_valid_storage_url():
raise WebDriverException("Session Storage is not available here!")
if self.__is_cdp_swap_needed():
self.cdp.set_session_storage_item(key, value)
return
self.execute_script(
"window.sessionStorage.setItem('{}', '{}');".format(key, value)
)
Expand All @@ -8867,6 +8878,8 @@ def get_session_storage_item(self, key):
self.__check_scope()
if not self.__is_valid_storage_url():
raise WebDriverException("Session Storage is not available here!")
if self.__is_cdp_swap_needed():
return self.cdp.get_session_storage_item(key)
return self.execute_script(
"return window.sessionStorage.getItem('{}');".format(key)
)
Expand Down
7 changes: 4 additions & 3 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@
'pip>=25.0.1',
'packaging>=24.2',
'setuptools~=70.2;python_version<"3.10"', # Newer ones had issues
'setuptools>=75.8.2;python_version>="3.10"',
'setuptools>=76.0.0;python_version>="3.10"',
'wheel>=0.45.1',
'attrs>=25.1.0',
"certifi>=2025.1.31",
Expand All @@ -160,7 +160,7 @@
'filelock~=3.16.1;python_version<"3.9"',
'filelock>=3.17.0;python_version>="3.9"',
'fasteners>=0.19',
"mycdp>=1.1.0",
"mycdp>=1.1.1",
"pynose>=1.5.4",
'platformdirs>=4.3.6',
'typing-extensions>=4.12.2',
Expand Down Expand Up @@ -193,7 +193,8 @@
'websocket-client==1.8.0',
'selenium==4.27.1;python_version<"3.9"',
'selenium==4.29.0;python_version>="3.9"',
'cssselect==1.2.0',
'cssselect==1.2.0;python_version<"3.9"',
'cssselect==1.3.0;python_version>="3.9"',
"sortedcontainers==2.4.0",
'execnet==2.1.1',
'iniconfig==2.0.0',
Expand Down