Develop Reference

Python Selenium webdriver doesn't open chrome, and if it does - it keeps refreshing it without any result

I am trying to enter web.whatsapp.com while using Selenium through Python, it opens Chrome web browser but doesn't enter the site but shows "data.;" blank page instead.
import pandas as pd
import webbrowser
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManagerz
options=Options()
options.add_experimental_option("detach",True)
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
#driver.get("'https://web.whatsapp.com/send?phone='+x+'&text='+message+''")
driver.get("'https://ynet.co.il")
#driver = webdriver.Edge()
import time
I supposed to get the browser open on the website page but instead I get the "data:," blank page opened and those errors:
File "D:\liranew\Lib\Main.py", line 12, in <module>
driver.get("'https://ynet.co.il")
File "D:\liranew\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 455, in get
self.execute(Command.GET, {"url": url})
File "D:\liranew\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 444, in execute
self.error_handler.check_response(response)
File "D:\liranew\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 249, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument
(Session info: chrome=107.0.5304.88)
Stacktrace:
Backtrace:
Ordinal0 [0x010EACD3+2075859]
Ordinal0 [0x0107EE61+1633889]
Ordinal0 [0x00F7B680+571008]
Ordinal0 [0x00F6E8FE+518398]
Ordinal0 [0x00F6D2A3+512675]
Ordinal0 [0x00F6D5AD+513453]
Ordinal0 [0x00F7D0CE+577742]
Ordinal0 [0x00FDBC7D+965757]
Ordinal0 [0x00FC731C+881436]
Ordinal0 [0x00FDB56A+963946]
Ordinal0 [0x00FC7136+880950]
Ordinal0 [0x00F9FEFD+720637]
Ordinal0 [0x00FA0F3F+724799]
GetHandleVerifier [0x0139EED2+2769538]
GetHandleVerifier [0x01390D95+2711877]
GetHandleVerifier [0x0117A03A+521194]
GetHandleVerifier [0x01178DA0+516432]
Ordinal0 [0x0108682C+1665068]
Ordinal0 [0x0108B128+1683752]
Ordinal0 [0x0108B215+1683989]
Ordinal0 [0x01096484+1729668]
BaseThreadInitThunk [0x74F86359+25]
RtlGetAppContainerNamedObjectPath [0x773C7C14+228]
RtlGetAppContainerNamedObjectPath [0x773C7BE4+180]

You have to remove the ' in front of 'https://ynet.co.il. It should be https://ynet.co.il i.e.
driver.get("https://ynet.co.il")
So simple

Try to click the "accept-cookies" button Error: "no such element: Unable to locate element"

i want to click the "accept Cookies" Button, but everytime i get the following error "Message: no such element: Unable to locate element:".
I think this is because this is a popup-window. I try to click the Button via Xpath.
How can i handle this?
Picture of the Website
My Code:
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome('./chromedriver')
link = 'https://www.ticket-onlineshop.com/ols/fckoeln/de/heim/channel/shop/index/index/event/424413'
#Open Website
driver.get(link)
#Timeout
time.sleep(5)
#Cookies
driver.find_element(By.XPATH,'//*[#id="uc-center-container"]/div[2]/div/div/div/div[2]/button[2]').click()
Error:
driver = webdriver.Chrome('./chromedriver')
Traceback (most recent call last):
File "C:\Users\Manu\PycharmProjects\SampleSale\FC.py", line 12, in <module>
driver.find_element(By.XPATH, '//*[#id="uc-center-container"]/div[2]/div/div/div/div[2]/button[2]').click()
File "C:\Users\Manu\PycharmProjects\SampleSale\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 1248, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Users\Manu\PycharmProjects\SampleSale\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 425, in execute
self.error_handler.check_response(response)
File "C:\Users\Manu\PycharmProjects\SampleSale\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id="uc-center-container"]/div[2]/div/div/div/div[2]/button[2]"}
(Session info: chrome=101.0.4951.41)
Stacktrace:
Backtrace:
Ordinal0 [0x0067B8F3+2406643]
Ordinal0 [0x0060AF31+1945393]
Ordinal0 [0x004FC748+837448]
Ordinal0 [0x005292E0+1020640]
Ordinal0 [0x0052957B+1021307]
Ordinal0 [0x00556372+1205106]
Ordinal0 [0x005442C4+1131204]
Ordinal0 [0x00554682+1197698]
Ordinal0 [0x00544096+1130646]
Ordinal0 [0x0051E636+976438]
Ordinal0 [0x0051F546+980294]
GetHandleVerifier [0x008E9612+2498066]
GetHandleVerifier [0x008DC920+2445600]
GetHandleVerifier [0x00714F2A+579370]
GetHandleVerifier [0x00713D36+574774]
Ordinal0 [0x00611C0B+1973259]
Ordinal0 [0x00616688+1992328]
Ordinal0 [0x00616775+1992565]
Ordinal0 [0x0061F8D1+2029777]
BaseThreadInitThunk [0x761D6739+25]
RtlGetFullPathName_UEx [0x77C78E7F+1215]
RtlGetFullPathName_UEx [0x77C78E4D+1165]
Process finished with exit code 1

The element is present in the shadow root, to click the button we need to use querySelector() ..... with the below code i can able to click the button
driver.execute_script('''return document.querySelector('div#usercentrics-root').shadowRoot.querySelector('button[data-testid="uc-accept-all-button"]')''').click()
it is already explained in one of the StackOverflow question Link

Try:
driver.find_element(By.XPATH,”//*[#id=‘uc-center-container’]/div[2]/div/div/div/div[2]/button[2]”).click()

When i try to scrape this website with selenium and python

When i try and scrape the website it just throws some errors
I think it may have something to do with my webdriver but idk
I am trying to get this data so i can put it in a spreadsheet to get some cool staticstics
from selenium import webdriver
url = 'https://prosettings.net/cs-go-pro-settings-gear-list/'
driver = webdriver.Chrome(executable_path="C:\WebDrivers\chromedriver.exe")
driver.get(url)
names = driver.find_elements_by_class_name(" column-player")
for name in names:
title = name.find_element_by_xpath('.//a').text
print(name)
Here are the error i get in terminal
d:\downloads\PythonScraping\Test.py:5: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
driver = webdriver.Chrome(executable_path="C:\WebDrivers\chromedriver.exe")
DevTools listening on ws://127.0.0.1:53131/devtools/browser/73ca0453-352e-47a0-a98a-fb539150d6f9
d:\downloads\PythonScraping\Test.py:8: DeprecationWarning: find_elements_by_* commands are deprecated. Please use find_elements() instead
names = driver.find_elements_by_class_name(" column-player")
Traceback (most recent call last):
File "d:\downloads\PythonScraping\Test.py", line 8, in <module>
names = driver.find_elements_by_class_name(" column-player")
File "C:\Users\terk0\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\selenium\webdriver\remote\webdriver.py", line 783, in
find_elements_by_class_name
return self.find_elements(by=By.CLASS_NAME, value=name)
File "C:\Users\terk0\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\selenium\webdriver\remote\webdriver.py", line 1279, in find_elements
return self.execute(Command.FIND_ELEMENTS, {
File "C:\Users\terk0\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\selenium\webdriver\remote\webdriver.py", line 424, in
execute
self.error_handler.check_response(response)
File "C:\Users\terk0\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\selenium\webdriver\remote\errorhandler.py", line 247,
in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: An invalid or illegal selector was specified
(Session info: chrome=98.0.4758.102)
Stacktrace:
Backtrace:
Ordinal0 [0x00EF69A3+2582947]
Ordinal0 [0x00E8A6D1+2139857]
Ordinal0 [0x00D83A98+1063576]
Ordinal0 [0x00D862B7+1073847]
Ordinal0 [0x00D8617E+1073534]
Ordinal0 [0x00D863F0+1074160]
Ordinal0 [0x00DAFCB2+1244338]
Ordinal0 [0x00DB013B+1245499]
Ordinal0 [0x00DD9F8C+1417100]
Ordinal0 [0x00DC8594+1344916]
Ordinal0 [0x00DD834A+1409866]
Ordinal0 [0x00DC8366+1344358]
Ordinal0 [0x00DA5176+1200502]
Ordinal0 [0x00DA6066+1204326]
GetHandleVerifier [0x0109BE02+1675858]
GetHandleVerifier [0x0115036C+2414524]
GetHandleVerifier [0x00F8BB01+560977]
GetHandleVerifier [0x00F8A8D3+556323]
Ordinal0 [0x00E9020E+2163214]
Ordinal0 [0x00E95078+2183288]
Ordinal0 [0x00E951C0+2183616]
Ordinal0 [0x00E9EE1C+2223644]
BaseThreadInitThunk [0x7586FA29+25]
RtlGetAppContainerNamedObjectPath [0x77957A9E+286]
RtlGetAppContainerNamedObjectPath [0x77957A6E+238]

There are 2 problems here:
Instead of
names = driver.find_elements_by_class_name(" column-player")
it should be
names = driver.find_elements_by_class_name("column-player")
(I know, there are spaces before column-player class name there, but you still should not put them inside the locator)
2) You should add a delay to access these elements only after the page have been completely loaded.
This should work better:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
url = 'https://prosettings.net/cs-go-pro-settings-gear-list/'
driver = webdriver.Chrome(executable_path="C:\WebDrivers\chromedriver.exe")
wait = WebDriverWait(driver, 20)
driver.get(url)
#wait for at least 1 element visibility
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".column-player")))
#short additional wait for all the other elements to complete loading
time.sleep(0.5)
names = driver.find_elements_by_class_name("column-player")
for name in names:
title = name.find_element_by_xpath('.//a').text
print(name)

Is CssSelector the only way I can use for shadowroot? selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: invalid locator

Is CSS_SELECTOR the only way I can use to find element inside a shadowroot element?
I am using Python 3.10.1, Selenium (4.1.0) chrome(96.0.4664.110).
Below is my code
host = browser.find_element(By.ID, 'someid')
shadow_root = browser.execute_script('return arguments[0].shadowRoot', host)
goal = shadow_root.find_element(By.XPATH, '//*/button[#aria-label="Required"]')
and the error I got as below if I don't use By.CSS_SELECTOR.
Traceback (most recent call last):
File "C:\Files\Workspace\PostInstallVerify.py", line 787, in <module>
maxlenInput = getShadowElement(maxlenid, ".rwa-input")
File "C:\Files\Workspace\PostInstallVerify.py", line 272, in getShadowElement
goal = shadow_root.find_element(By.XPATH, '//*/button[#aria-label="Required"]')
File "C:\Users\Tomas.Tang\AppData\Local\Programs\Python\Python310\lib\site-packages\selenium\webdriver\remote\shadowroot.py", line 45, in find_element
return self._execute(Command.FIND_ELEMENT_FROM_SHADOW_ROOT, {"using": using, "value": value})['value']
File "C:\Users\Tomas.Tang\AppData\Local\Programs\Python\Python310\lib\site-packages\selenium\webdriver\remote\shadowroot.py", line 64, in _execute
return self.session.execute(command, params)
File "C:\Users\Tomas.Tang\AppData\Local\Programs\Python\Python310\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 424, in execute
self.error_handler.check_response(response)
File "C:\Users\Tomas.Tang\AppData\Local\Programs\Python\Python310\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: invalid locator
(Session info: chrome=96.0.4664.110)
Stacktrace:
Backtrace:
Ordinal0 [0x00326903+2517251]
Ordinal0 [0x002BF8E1+2095329]
Ordinal0 [0x001C2848+1058888]
Ordinal0 [0x001ED5C4+1234372]
Ordinal0 [0x001ED68B+1234571]
Ordinal0 [0x001E3AE1+1194721]
Ordinal0 [0x0020650A+1336586]
Ordinal0 [0x001E3A36+1194550]
Ordinal0 [0x002065BA+1336762]
Ordinal0 [0x00215BBF+1399743]
Ordinal0 [0x0020639B+1336219]
Ordinal0 [0x001E27A7+1189799]
Ordinal0 [0x001E3609+1193481]
GetHandleVerifier [0x004B5904+1577972]
GetHandleVerifier [0x00560B97+2279047]
GetHandleVerifier [0x003B6D09+534521]
GetHandleVerifier [0x003B5DB9+530601]
Ordinal0 [0x002C4FF9+2117625]
Ordinal0 [0x002C98A8+2136232]
Ordinal0 [0x002C99E2+2136546]
Ordinal0 [0x002D3541+2176321]
BaseThreadInitThunk [0x766CFA29+25]
RtlGetAppContainerNamedObjectPath [0x773A7A9E+286]
RtlGetAppContainerNamedObjectPath [0x773A7A6E+238]
(No symbol) [0x00000000]

Simply use
.//button[#aria-label='Required']
Xpath from an element requires a . unlike others.

You can locate elements inside the shadow DOM by any regular locator: CSS_SELECTOR, XPATH, ID, CLASS_NAME etc.
Here you are trying to use an invalid XPath expression.
'//*/button[#aria-label="Required"]'
is not a valid XPath expression.
I can't know what page you are working on and what is the correct locator of that element, but this will be at least a correct XPath expression that may be correct for you:
'//button[#aria-label="Required"]'
So instead of
goal = shadow_root.find_element(By.XPATH, '//*/button[#aria-label="Required"]')
Try this:
goal = shadow_root.find_element(By.XPATH, '//button[#aria-label="Required"]')
Or maybe
goal = shadow_root.find_element(By.XPATH, './/button[#aria-label="Required"]')

No, css-selectors isn't the only way to interact with shadowroot.
Your code trial was near perfect but the XPATH expression wasn't a valid xpath expression as within the expression:
goal = shadow_root.find_element(By.XPATH, '//*/button[#aria-label="Required"]')
the /* wasn't needed as //button itself would look for a button tag within the entire DOM Tree.
So effectively you line of code will be:
goal = shadow_root.find_element(By.XPATH, '//button[#aria-label="Required"]')

What does Backtrace: Ordinal0 [0x00723AB3+2505395] in stack trace error means while using Selenium and ChromeDriver

I'm relatively new to coding and kinda unfamiliar with stacktrace errors.
I have been given this error, but im not sure which part of my code do i fix? Did the error come
up because of a NoSuchElementException, or was it due to a timeout error?
I thought that within my code I had dealt with how the program will face the nosuchelement exception. Basically, my code searches for slots that are available and will disable the back button when a slot becomes available. However, I am met with other sorts of error such as connection time out when the webpage has been idle for too long. Thus, my mission now is to filter out the different problems i am faced with so that i can deal with the issues respectively. I have added the expected conditions as the filter to differentiate the errors i am faced with
Code:
no_slot = True
while no_slot == 1:
for i in range(60):
search_button = browser.find_element(By.NAME, "btnSearch")
search_button.click()
try:
noslot_back_button = browser.find_element(By.CSS_SELECTOR, 'input[name="btnBack"]')
except NoSuchElementException:
if EC.presence_of_element_located(browser.find_element(By.XPATH, '//td[#class="title"]')):
# Proceed with booking (make sure that it is at the booking page (implementation)
slot_radio_info = browser.find_element(By.NAME, 'slot')
slot_id = slot_radio_info.get_attribute('id')
radio_button = browser.find_element(By.XPATH, '//input[#id="' + str(slot_id) + '"]')
radio_button.click()
slot_submit_button = browser.find_element(By.CSS_SELECTOR, 'input[value="Submit"]')
WebDriverWait(EC.element_to_be_clickable(slot_submit_button))
slot_submit_button.click()
double_cfm_button = browser.find_element(By.CSS_SELECTOR, 'input[value="Confirm"]')
double_cfm_button.click()
elif not EC.title_contains('BBDC: Booking For TP Driving Simulator Lesson'):
print("Connection timed out or something else happened")
else:
print("unaccounted condition")
Error received:
Traceback (most recent call last):
File "C:/Users/Jonathan/PycharmProjects/Riddle/BBDC.py", line 83, in <module>
noslot_back_button = browser.find_element(By.CSS_SELECTOR, 'input[name="btnBack"]')
File "C:\Users\Jonathan\venvs\automation\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 1238, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Users\Jonathan\venvs\automation\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 418, in execute
self.error_handler.check_response(response)
File "C:\Users\Jonathan\venvs\automation\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"input[name="btnBack"]"}
(Session info: chrome=95.0.4638.69)
Stacktrace:
Backtrace:
Ordinal0 [0x00723AB3+2505395]
Ordinal0 [0x006BAE41+2076225]
Ordinal0 [0x005C2498+1057944]
Ordinal0 [0x005ECB74+1231732]
Ordinal0 [0x00616D92+1404306]
Ordinal0 [0x00605A2A+1333802]
Ordinal0 [0x00615168+1397096]
Ordinal0 [0x006058BB+1333435]
Ordinal0 [0x005E23E4+1188836]
Ordinal0 [0x005E323F+1192511]
GetHandleVerifier [0x008ACB36+1554566]
GetHandleVerifier [0x00954A0C+2242396]
GetHandleVerifier [0x007B0E0B+523099]
GetHandleVerifier [0x007AFEB0+519168]
Ordinal0 [0x006C02FD+2097917]
Ordinal0 [0x006C4388+2114440]
Ordinal0 [0x006C44C2+2114754]
Ordinal0 [0x006CE041+2154561]
BaseThreadInitThunk [0x7739FA29+25]
RtlGetAppContainerNamedObjectPath [0x77987A9E+286]
RtlGetAppContainerNamedObjectPath [0x77987A6E+238]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Jonathan\PyCharm\plugins\python-ce\helpers\pydev\pydevd.py", line 1483, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\Users\Jonathan\PyCharm\plugins\python-ce\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/Jonathan/PycharmProjects/Riddle/BBDC.py", line 86, in <module>
if EC.presence_of_element_located(browser.find_element(By.XPATH, '//td[#class="title"]')):
File "C:\Users\Jonathan\venvs\automation\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 1238, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Users\Jonathan\venvs\automation\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 418, in execute
self.error_handler.check_response(response)
File "C:\Users\Jonathan\venvs\automation\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//td[#class="title"]"}
(Session info: chrome=95.0.4638.69)
Stacktrace:
Backtrace:
Ordinal0 [0x00723AB3+2505395]
Ordinal0 [0x006BAE41+2076225]
Ordinal0 [0x005C2498+1057944]
Ordinal0 [0x005ECB74+1231732]
Ordinal0 [0x00616D92+1404306]
Ordinal0 [0x00605A2A+1333802]
Ordinal0 [0x00615168+1397096]
Ordinal0 [0x006058BB+1333435]
Ordinal0 [0x005E23E4+1188836]
Ordinal0 [0x005E323F+1192511]
GetHandleVerifier [0x008ACB36+1554566]
GetHandleVerifier [0x00954A0C+2242396]
GetHandleVerifier [0x007B0E0B+523099]
GetHandleVerifier [0x007AFEB0+519168]
Ordinal0 [0x006C02FD+2097917]
Ordinal0 [0x006C4388+2114440]
Ordinal0 [0x006C44C2+2114754]
Ordinal0 [0x006CE041+2154561]
BaseThreadInitThunk [0x7739FA29+25]
RtlGetAppContainerNamedObjectPath [0x77987A9E+286]
RtlGetAppContainerNamedObjectPath [0x77987A6E+238]
python-BaseException
Traceback (most recent call last):
File "C:\Users\Jonathan\PyCharm\plugins\python-ce\helpers\pydev\_pydevd_bundle\pydevd_comm.py", line 292, in _on_run
r = self.sock.recv(1024)
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
Process finished with exit code 1

Selenium is unable to locate the element
noslot_back_button = browser.find_element(By.CSS_SELECTOR, 'input[name="btnBack"]')
most likely a typo or maybe you forgot to name the css selector.

This error message...
Traceback (most recent call last):
.
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"input[name="btnBack"]"}
(Session info: chrome=95.0.4638.69)
Stacktrace:
Backtrace:
Ordinal0 [0x00723AB3+2505395]
Ordinal0 [0x006BAE41+2076225]
Ordinal0 [0x005C2498+1057944]
...implies that NoSuchElementException was raised while locating the elements.
Addressing NoSuchElementException
Without the relevant HTML of the element it would be difficult to address if the Locator Strategy identifies the element uniquely within the HTML DOM.
Additionally, though you are using chrome=95.0.4638.69 but your ChromeDriver version isn't in sync with the Chrome browser version. Hence you see the Backtrace.
Solution
Ensure that:
ChromeDriver is updated to current ChromeDriver v95.0 level.