I am trying to iterate over pages having a program List and then I have to select checkbox defined in an image below in order to get the run length of a loop on-page.
But when I am trying to navigate over pages via the next button it will stop clicking on checkboxes and throws me below error.
I have tried with xpath,css selector and id method But all get failed.
DOM element SS along with checkbox highlighted
Line of Code Used-
CheckBoxclick = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#workflow_table > thead > tr > th.first.checkbox-column.checkbox > input"))).click()
Error:-
File "D:/Vipul Garg Backup/XXXX/TestingPardotExportProgramsData.py", line 74, in <module>
ProgramsPathlist = getDataForPrograms(RecordsonOnePage,totalnofpages)
File "D:/Vipul Garg Backup/XXXX/TestingPardotExportProgramsData.py", line 63, in getDataForPrograms
RecordsonOnePage = getrecordsoNpage()
File "D:/Vipul Garg Backup/XXXX/TestingPardotExportProgramsData.py", line 42, in getrecordsoNpage
CheckBoxclick = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#workflow_table > thead > tr > th.first.checkbox-column.checkbox > input"))).click()
File "C:\Users\Vipul.Garg\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "C:\Users\Vipul.Garg\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "C:\Users\Vipul.Garg\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\Vipul.Garg\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
(Session info: chrome=80.0.3987.122)
But Above Line of Code is Working Intermetantly and throwing error as above.
Any suggestions for this line of code replacement as why I am getting such inconsistency.
You are using ID of the Table and checkbox is inside that table, so you need to pass locator of the check box.
As there is no id defined for the checkbox, so using xpath to reach the locator like - //input[#type='checkbox']
Related
I'm trying to extract 1000 usernames and the team rosters for each username in a list from this SPA (single page application) sports website (requires free login): https://rumble.otmnft.com/contest/live?contest=Rumble
I've tried numerous strategies to try to avoid the common StaleElementReferenceException problem in Selenium including try / except block, WebDriverWait and time.sleep(5) among many others, but none have worked. Would be amazing if someone knows an alternative solution which could work as I've been trying to solve this by myself for quite a few days now.
Here is the relevant part of the latest attempted code which still throws StaleElementReferenceException below:
parent_dict = {}
parent_dict['data'] = []
USERNAME_XPATH = "//span[#class='hidden lg:flex font-semibold']"
ROSTER_XPATH = "//span[#class='text-white hidden lg:flex font-semibold']"
WebDriverWait(driver, 50).until(EC.presence_of_element_located((By.XPATH, "//table")))
username_elements = driver.find_elements(By.XPATH, USERNAME_XPATH)
try:
for usr_element in username_elements:
usr_element.click()
except StaleElementReferenceException:
# Find the element again and continue with the loop
WebDriverWait(driver, 50).until(EC.presence_of_element_located((By.XPATH, USERNAME_XPATH)))
username_elements = driver.find_elements(By.XPATH, USERNAME_XPATH)
time.sleep(5)
for usr_element in username_elements:
usr_element.click()
WebDriverWait(driver, 50).until(EC.presence_of_element_located((By.XPATH, ROSTER_XPATH)))
roster_elements = driver.find_elements(By.XPATH, ROSTER_XPATH)
roster = [cell.text for cell in roster_elements]
TITLE_USER_XPATH = "//div[#class='flex flex-col space-y-2']/span"
title_user = driver.find_element(By.XPATH, TITLE_USER_XPATH)
child_dict = {"username": title_user.text, "roster": roster }
print("CHILD_DICT", child_dict)
parent_dict['data'].append(child_dict)
with open('data_v2.json', 'w') as outfile:
json.dump(parent_dict, outfile)
print("PARENT_DICT", parent_dict)
return parent_dict
In the browser I can see that the script is able to click on the very first username but it fails when trying click on the second username, but reports the error for the line usr_element.click() both inside the try and except blocks
Full error trace is:
Traceback (most recent call last):
File "/Users/danpro12/Dropbox/01 FREE TIME STUFF/01 WEB DEVELOPMENT/12 DAPPER RELATED/otm_python_scraper/otm_specific/utils.py", line 23, in sub_func
usr_element.click()
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 88, in click
self._execute(Command.CLICK_ELEMENT)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 396, in _execute
return self._parent.execute(command, params)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
self.error_handler.check_response(response)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: The element reference of <span class="hidden lg:flex font-semibold"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
Stacktrace:
RemoteError#chrome://remote/content/shared/RemoteError.sys.mjs:8:8
WebDriverError#chrome://remote/content/shared/webdriver/Errors.sys.mjs:180:5
StaleElementReferenceError#chrome://remote/content/shared/webdriver/Errors.sys.mjs:461:5
element.resolveElement#chrome://remote/content/marionette/element.sys.mjs:674:11
evaluate.fromJSON#chrome://remote/content/marionette/evaluate.sys.mjs:255:31
evaluate.fromJSON#chrome://remote/content/marionette/evaluate.sys.mjs:263:29
receiveMessage#chrome://remote/content/marionette/actors/MarionetteCommandsChild.sys.mjs:74:34
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/danpro12/Dropbox/01 FREE TIME STUFF/01 WEB DEVELOPMENT/12 DAPPER RELATED/otm_python_scraper/otm_specific/rumble_client.py", line 53, in <module>
sub_func(driver)
File "/Users/danpro12/Dropbox/01 FREE TIME STUFF/01 WEB DEVELOPMENT/12 DAPPER RELATED/otm_python_scraper/otm_specific/utils.py", line 30, in sub_func
usr_element.click()
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 88, in click
self._execute(Command.CLICK_ELEMENT)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 396, in _execute
return self._parent.execute(command, params)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
self.error_handler.check_response(response)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: The element reference of <span class="hidden lg:flex font-semibold"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
Stacktrace:
RemoteError#chrome://remote/content/shared/RemoteError.sys.mjs:8:8
WebDriverError#chrome://remote/content/shared/webdriver/Errors.sys.mjs:180:5
StaleElementReferenceError#chrome://remote/content/shared/webdriver/Errors.sys.mjs:461:5
element.resolveElement#chrome://remote/content/marionette/element.sys.mjs:674:11
evaluate.fromJSON#chrome://remote/content/marionette/evaluate.sys.mjs:255:31
evaluate.fromJSON#chrome://remote/content/marionette/evaluate.sys.mjs:263:29
receiveMessage#chrome://remote/content/marionette/actors/MarionetteCommandsChild.sys.mjs:74:34
Seems that after you do usr_element.click() DOM updated and so you cannot iterate over username_elements since all elements in this list are stale.
Try this solution
username_elements_length = len(driver.find_elements(By.XPATH, USERNAME_XPATH))
for usr_element_index in range(username_elements_length):
username_element = driver.find_elements(By.XPATH, USERNAME_XPATH)[usr_element_index]
username_element.click()
WebDriverWait(driver, 5).until(EC.staleness_of(username_element))
I am running some testing on our website using selenium.
At the login page I would like to target the login button and click it.
the source code of the page looks like this:
I am trying to target the second button that has the class=OTSigninButton by using its xpath.
so here is the python code.
time.sleep(5)
element = driver.find_element_by_xpath('//*[#id="root"]/div/div[1]/div/svg[2]/path')
element.click()
but when I run the code I get the following error:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id="root"]/div/div[1]/div/svg[2]/path"}
(Session info: chrome=91.0.4472.106)
Here is the weird part. The login page has 2 buttons, their xpath is similar as its a list.
the first button is:
//*[#id="root"]/div/div[1]/div/svg[1]/path
and the second button(the one I want to target) is
//*[#id="root"]/div/div[1]/div/svg[1]/path
As both of them has the id=root, if I find_element_by_id and target root, I am able to click on the first button.
Here is where I am struggling a lot, how can I use the xpath to target the second button?
thank you so much for your time and help guys.
EDIT: Full error
Traceback (most recent call last):
File "test.py", line 17, in <module>
element = driver.find_element_by_xpath("//*[local-name()='svg' and contains(#class, 'OTSigninButton')]/title")
File "/Users/<user>/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 394, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "/Users/<user>/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "/Users/<user>/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/Users/<user>/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[local-name()='svg' and contains(#class, 'OTSigninButton')]/title"}
(Session info: chrome=91.0.4472.106)
class=OTSigninButton
is a attribute of SVG. These are special tags, you can not just write their tag name in console and locate them.
try this instead :
//*[local-name()='svg' and contains(#class, 'OTSigninButton')]
"selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable"
This is the error i get for no reason at all. i mean earlier it used to work but now it isnt.
In my code there is this website : https://www.tempinbox.xyz/
i open it using the chromedriver.
what i want to do is select (#fitschool.be) from the dropdown menu of domains.
and input some variable string which i have stored as variable "str(res)"
my code is:-
second_tab = webdriver.Chrome(options=options,)
second_tab.get("https://www.tempinbox.xyz/")
randombutton = second_tab.find_element_by_xpath("/html/body/main/div[1]/div/div[1]/div[2]/form[1]/input[2]") #for the email generation
randombutton.send_keys(str(res))
mainurl = second_tab.current_url
selectoption1=second_tab.find_element_by_id("selected-domain").click()
selectoption2=second_tab.find_element_by_css_selector(".domain-selector:nth-child(3)").click()
clickfinal= second_tab.find_element_by_xpath("//input[#value='Create']").click()
emailentry = str(res)+ "#fitschool.be"
print ("your email is " + emailentry)
but it's giving me errors....
Traceback (most recent call last):
File "C:\Users\BRS\Desktop\AccountChef\wattpad acc maker.py", line 45, in <module>
selectoption2=second_tab.find_element_by_css_selector(".domain-selector:nth-child(3)").click()
File "C:\Users\BRS\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "C:\Users\BRS\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "C:\Users\BRS\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\BRS\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
(Session info: chrome=84.0.4147.105)
pls suggest some changes and why this is happening. Thanks :)
Instead of the following two lines:
selectoption1=second_tab.find_element_by_id("selected-domain").click()
selectoption2=second_tab.find_element_by_css_selector(".domain-selector:nth-child(3)").click()
Try to set the innerHTML value with .execute_script:
element = second_tab.find_element_by_id("selected-domain")
target = "#fitschool.be"
second_tab.execute_script("arguments[0].innerHTML = '{}';".format(target), element)
So im trying to click a button in an iframe but it just errors saying that the button can't be found.
recaptcha = driver.find_element_by_tag_name('iframe')
driver.switch_to.frame(recaptcha)
submit1 = driver.find_element_by_id('recaptcha-anchor')
actions = ActionChains(driver)
actions.move_to_element(submit1)
actions.click(submit1).perform()
time.sleep(3)
driver.find_element_by_id('recaptcha-audio-button').click()
On the last line I try to get an element by id recaptcha-audio-button.
As I said above it just errors with that it can't find it.
The website that I am trying to do this with is: https://patrickhlauke.github.io/recaptcha/
Why can't I click this button?
Thanks, Ira.
EDIT
Here is the error
Traceback (most recent call last):
File "main.py", line 62, in <module>
driver.find_element_by_id('recaptcha-audio-button').click()
File "/home/ira/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 269, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "/home/ira/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 752, in find_element
'value': value})['value']
File "/home/ira/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "/home/ira/.local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: An element could not be located on the page using the given search parameters.
Element is in different iframe so you can do:
switch to main window
find all iframes and switch to second iframe
find and click your button
Code:
# go back to main window
driver.switch_to.default_content()
# find all frames and switch to second frame
all_frames = driver.find_elements_by_tag_name('iframe')
driver.switch_to.frame(all_frames[1])
driver.find_element_by_id('recaptcha-audio-button').click()
Minimal working code so everyone can copy and test it
import selenium.webdriver
import time
driver = selenium.webdriver.Firefox()
driver.get('https://patrickhlauke.github.io/recaptcha/')
recaptcha = driver.find_element_by_tag_name('iframe')
driver.switch_to.frame(recaptcha)
submit1 = driver.find_element_by_id('recaptcha-anchor')
actions = selenium.webdriver.ActionChains(driver)
actions.move_to_element(submit1)
actions.click(submit1).perform()
time.sleep(3)
# go back to main window
driver.switch_to.default_content()
# find all frames and switch to second frame
all_frames = driver.find_elements_by_tag_name('iframe')
driver.switch_to.frame(all_frames[1])
driver.find_element_by_id('recaptcha-audio-button').click()
I am trying to extract all those tags whose class name fits the regex pattern frag-0-0, frag-1-0, etc. from this enter link description here
I am trying the following code
driver = webdriver.PhantomJS()
for frg in frgs:
driver.get(URL + frg[1:])
frags=driver.find_elements_by_xpath("//*[starts-with(#id, 'frag-') and ends-with(#id, '-0')]")
for frag in frags:
for tag in frag.find_elements_by_css_selector('[class^=fragmark]'):
lst.append([tag.get_attribute('class'), tag.text])
driver.quit()
This is my traceback:
Traceback (most recent call last):
File "/home/ubuntu/workspace/vroniplag/vroni.py", line 116, in
op('Aaf')
File "/home/ubuntu/workspace/vroniplag/vroni.py", line 101, in op
plags=getplags(cd)
File "/home/ubuntu/workspace/vroniplag/vroni.py", line 92, in getplags
frags=driver.find_elements_by_xpath("//[starts-with(#id, 'frag-') and ends-with(#id, '-0')]")
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py",
line 305, in find_elements_by_xpath
return self.find_elements(by=By.XPATH, value=xpath)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py",
line 778, in find_elements
'value': value})['value']
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py",
line 236, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py",
line 192, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSelectorException: Message: Error Message => 'Unable to locate an element with the xpath
expression //[starts-with(#id, 'frag-') and ends-with(#id, '-0')]
because of the following error:
Error: INVALID_EXPRESSION_ERR: DOM XPath Exception 51'
caused by Request => {"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"139","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:45340","User-Agent":"Python-urllib/2.7"},"httpVersion":"1.1","method":"POST","post":"{\"using\":
\"xpath\", \"sessionId\": \"0dbc6ad0-4352-11e6-8cb8-4faebd646180\",
\"value\": \"//*[starts-with(#id, 'frag-') and ends-with(#id,
'-0')]\"}","url":"/elements","urlParsed":{"anchor":"","query":"","file":"elements","directory":"/","path":"/elements","relative":"/elements","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/elements","queryKey":{},"chunks":["elements"]},"urlOriginal":"/session/0dbc6ad0-4352-11e6-8cb8-4faebd646180/elements"}
Screenshot: available via screen
What am I doing wrong?
You can try to replace
"//*[starts-with(#id, 'frag-') and ends-with(#id, '-0')]"
with
"//*[starts-with(#id, 'frag-') and contains(#id, '-0')]"
as Selenium doesn't support ends-with option