I am trying to click a button but I am having an error when finding the element. In general I am trying to refresh a page until the link to join the waiting list goes live. When I run the script the webpage pulls up but does not refresh or click the join waitlist button when it is live.
I've tried finding element by class, link text, xpath, each time I get an error saying the element wasn't found.
code
while not joinList:
try:
addToCartBtn = addButton = driver.find_element_by_class_name("disabled")
print("Button isnt ready yet.")
time.sleep(2)
driver.refresh()
except:
addToCartBtn = addButton = driver.find_element_by_class_name("signup-btn")
print("Join")
addToCartBtn.click()
joinList = True
Where the element for the disabled join waitlist button is:
button _ngcontent-yur-c149="" disabled="" class="signup-btn ng-tns-c149-6 ng-star-inserted" style="background-color: rgb(0, 59, 153);"> Waitlist is currently closed </button
and the element for the enabled waitlist is:
button _ngcontent-yur-c149="" class="signup-btn ng-tns-c149-4 ng-star-inserted" tabindex="0" style="background-color: rgb(0, 59, 153);"> Join waitlist </button
The error I get is:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Python\Pycharm\MCattempt2\Main test folder\Test 1.py", line 20, in <module>
addToCartBtn = addButton = driver.find_element_by_class_name("signup-btn")
File "C:\Python\Pycharm\MCattempt2\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 564, in find_element_by_class_name
return self.find_element(by=By.CLASS_NAME, value=name)
File "C:\Python\Pycharm\MCattempt2\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Python\Pycharm\MCattempt2\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Python\Pycharm\MCattempt2\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".signup-btn"}
(Session info: chrome=88.0.4324.190)
Any advice on what I am doing wrong?
Related
I'm trying to extract 1000 usernames and the team rosters for each username in a list from this SPA (single page application) sports website (requires free login): https://rumble.otmnft.com/contest/live?contest=Rumble
I've tried numerous strategies to try to avoid the common StaleElementReferenceException problem in Selenium including try / except block, WebDriverWait and time.sleep(5) among many others, but none have worked. Would be amazing if someone knows an alternative solution which could work as I've been trying to solve this by myself for quite a few days now.
Here is the relevant part of the latest attempted code which still throws StaleElementReferenceException below:
parent_dict = {}
parent_dict['data'] = []
USERNAME_XPATH = "//span[#class='hidden lg:flex font-semibold']"
ROSTER_XPATH = "//span[#class='text-white hidden lg:flex font-semibold']"
WebDriverWait(driver, 50).until(EC.presence_of_element_located((By.XPATH, "//table")))
username_elements = driver.find_elements(By.XPATH, USERNAME_XPATH)
try:
for usr_element in username_elements:
usr_element.click()
except StaleElementReferenceException:
# Find the element again and continue with the loop
WebDriverWait(driver, 50).until(EC.presence_of_element_located((By.XPATH, USERNAME_XPATH)))
username_elements = driver.find_elements(By.XPATH, USERNAME_XPATH)
time.sleep(5)
for usr_element in username_elements:
usr_element.click()
WebDriverWait(driver, 50).until(EC.presence_of_element_located((By.XPATH, ROSTER_XPATH)))
roster_elements = driver.find_elements(By.XPATH, ROSTER_XPATH)
roster = [cell.text for cell in roster_elements]
TITLE_USER_XPATH = "//div[#class='flex flex-col space-y-2']/span"
title_user = driver.find_element(By.XPATH, TITLE_USER_XPATH)
child_dict = {"username": title_user.text, "roster": roster }
print("CHILD_DICT", child_dict)
parent_dict['data'].append(child_dict)
with open('data_v2.json', 'w') as outfile:
json.dump(parent_dict, outfile)
print("PARENT_DICT", parent_dict)
return parent_dict
In the browser I can see that the script is able to click on the very first username but it fails when trying click on the second username, but reports the error for the line usr_element.click() both inside the try and except blocks
Full error trace is:
Traceback (most recent call last):
File "/Users/danpro12/Dropbox/01 FREE TIME STUFF/01 WEB DEVELOPMENT/12 DAPPER RELATED/otm_python_scraper/otm_specific/utils.py", line 23, in sub_func
usr_element.click()
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 88, in click
self._execute(Command.CLICK_ELEMENT)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 396, in _execute
return self._parent.execute(command, params)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
self.error_handler.check_response(response)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: The element reference of <span class="hidden lg:flex font-semibold"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
Stacktrace:
RemoteError#chrome://remote/content/shared/RemoteError.sys.mjs:8:8
WebDriverError#chrome://remote/content/shared/webdriver/Errors.sys.mjs:180:5
StaleElementReferenceError#chrome://remote/content/shared/webdriver/Errors.sys.mjs:461:5
element.resolveElement#chrome://remote/content/marionette/element.sys.mjs:674:11
evaluate.fromJSON#chrome://remote/content/marionette/evaluate.sys.mjs:255:31
evaluate.fromJSON#chrome://remote/content/marionette/evaluate.sys.mjs:263:29
receiveMessage#chrome://remote/content/marionette/actors/MarionetteCommandsChild.sys.mjs:74:34
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/danpro12/Dropbox/01 FREE TIME STUFF/01 WEB DEVELOPMENT/12 DAPPER RELATED/otm_python_scraper/otm_specific/rumble_client.py", line 53, in <module>
sub_func(driver)
File "/Users/danpro12/Dropbox/01 FREE TIME STUFF/01 WEB DEVELOPMENT/12 DAPPER RELATED/otm_python_scraper/otm_specific/utils.py", line 30, in sub_func
usr_element.click()
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 88, in click
self._execute(Command.CLICK_ELEMENT)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 396, in _execute
return self._parent.execute(command, params)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
self.error_handler.check_response(response)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: The element reference of <span class="hidden lg:flex font-semibold"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
Stacktrace:
RemoteError#chrome://remote/content/shared/RemoteError.sys.mjs:8:8
WebDriverError#chrome://remote/content/shared/webdriver/Errors.sys.mjs:180:5
StaleElementReferenceError#chrome://remote/content/shared/webdriver/Errors.sys.mjs:461:5
element.resolveElement#chrome://remote/content/marionette/element.sys.mjs:674:11
evaluate.fromJSON#chrome://remote/content/marionette/evaluate.sys.mjs:255:31
evaluate.fromJSON#chrome://remote/content/marionette/evaluate.sys.mjs:263:29
receiveMessage#chrome://remote/content/marionette/actors/MarionetteCommandsChild.sys.mjs:74:34
Seems that after you do usr_element.click() DOM updated and so you cannot iterate over username_elements since all elements in this list are stale.
Try this solution
username_elements_length = len(driver.find_elements(By.XPATH, USERNAME_XPATH))
for usr_element_index in range(username_elements_length):
username_element = driver.find_elements(By.XPATH, USERNAME_XPATH)[usr_element_index]
username_element.click()
WebDriverWait(driver, 5).until(EC.staleness_of(username_element))
I am trying to return false if an element exist, but I don't want to end the program if the element doesn't exist.
My problem is that when Selenium doesn't find the element, it closes all but I don't want to close it, I want to run the rest of the code.
Is it possible?
My Python code:
if driver.find_element_by_xpath("//a[#class='btn btn-primary im_edit_forward_btn disabled']") != 0 :
print("Break here")
return False
print("Running the rest")
encaminhar = driver.find_element_by_xpath("//a[#class='btn btn-primary im_edit_forward_btn']")
encaminhar.click()
Output:
Traceback (most recent call last):
File "webscraping.py", line 94, in <module>
if forwardMessages():
File "webscraping.py", line 57, in forwardMessages
if driver.find_element_by_xpath("//a[#class='btn btn-primary im_edit_forward_btn disabled']") != 0 :
File "C:\Users\gusta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\selenium\webdriver\remote\webdriver.py", line 394, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "C:\Users\gusta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\selenium\webdriver\remote\webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Users\gusta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\gusta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//a[#class='btn btn-primary im_edit_forward_btn disabled']"}
You can catch the exception.
from selenium.common.exception import NoSuchElementException
try:
driver.find_element_by_xpath("//a[#class='btn btn-primary im_edit_forward_btn disabled']")
except NoSuchElementException:
pass
I am trying to iterate over pages having a program List and then I have to select checkbox defined in an image below in order to get the run length of a loop on-page.
But when I am trying to navigate over pages via the next button it will stop clicking on checkboxes and throws me below error.
I have tried with xpath,css selector and id method But all get failed.
DOM element SS along with checkbox highlighted
Line of Code Used-
CheckBoxclick = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#workflow_table > thead > tr > th.first.checkbox-column.checkbox > input"))).click()
Error:-
File "D:/Vipul Garg Backup/XXXX/TestingPardotExportProgramsData.py", line 74, in <module>
ProgramsPathlist = getDataForPrograms(RecordsonOnePage,totalnofpages)
File "D:/Vipul Garg Backup/XXXX/TestingPardotExportProgramsData.py", line 63, in getDataForPrograms
RecordsonOnePage = getrecordsoNpage()
File "D:/Vipul Garg Backup/XXXX/TestingPardotExportProgramsData.py", line 42, in getrecordsoNpage
CheckBoxclick = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#workflow_table > thead > tr > th.first.checkbox-column.checkbox > input"))).click()
File "C:\Users\Vipul.Garg\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "C:\Users\Vipul.Garg\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "C:\Users\Vipul.Garg\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\Vipul.Garg\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
(Session info: chrome=80.0.3987.122)
But Above Line of Code is Working Intermetantly and throwing error as above.
Any suggestions for this line of code replacement as why I am getting such inconsistency.
You are using ID of the Table and checkbox is inside that table, so you need to pass locator of the check box.
As there is no id defined for the checkbox, so using xpath to reach the locator like - //input[#type='checkbox']
I need to get text from emails and save for send in webwhatsapp, but the text is in a iframe and returns a error when I try to get him
Thats the html code:
<div id="mailContent" class="mail-content" style="height: 724px;">
<div id="phishingLinkScanContent"></div>
<iframe id="bodyMailViewer"
class="body-mail-viewer" style="height: 719px;"
cd_frame_id_="18a0312a828e12092646c9a6b1ca58c4">
#document
<html><head></head>
<body>MetaTrader email test -> cilabassa#terra.com.br<br><div id="ozoneAttachments"></div></body>
</html>
</iframe>
</div>
Thats my code:
driver.switch_to.frame(driver.find_element_by_tag_name("iframe"))
texto = driver.find_element_by_id('bodyMailViewer').text
Thats the error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 360, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
'value': value})['value']
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="bodyMailViewer"]"}
(Session info: chrome=75.0.3770.142)
First switch to the iframe and then locate the body tag under iframe and finally get the text from body tag. Try the below code.
driver.switch_to.frame(driver.find_element_by_tag_name("iframe"))
textInMail = driver.find_element_by_tag_name('body').text
The folowing is a snip of some crowdfire.com HTML code, where im trying to upload a file in the input class =...
<div id="ember1089" class="ember-view">
<div id="ember1090" class="action__addImage pointer ember-view tooltipstered">
<div data-ember-action="1091">
<img class="iconImg iconImg--camera" src="/publish/images/icon-camera-b432ac4c5b369d4616baf097b951d9b4.png"/>
<span>Add an image</span>
</div>
<input class="js-file-input action__fileInput" type="file" data-ember-action="1092"/>
</div>
Now below is what i have so far. I have created driver instance, the code before logs in the website without a problem and locates all other xpaths
image = driver.find_element_by_xpath('.//*[#id=\'ember1089\']/input')
time.sleep(2)
print 'found element'
image.send_keys('C:\Users\Brian\Desktop\Empire_fort\Bots\GetPICtures\Empire.jpg)
print 'uploading'
time.sleep(5)
the following is the error i get, and i really don't understand what is the problem,On the website there is an "add an image" link. if clicked it brings up window's explorer from where the user can select a file and upload it.
waiting
about toupload
found element
Traceback (most recent call last):
File "C:\Users\Brian\Desktop\Empire_fort\Bots\GetPICtures\nowupload.py", line 55, in <module>
image.send_keys(r'C:\Users\Brian\Desktop\Empire_fort\Bots\GetPICtures\Empire.jpg')
File "C:\Python2.7.11\lib\site-packages\selenium\webdriver\remote\webelement.py", line 321, in send_keys
self._execute(Command.SEND_KEYS_TO_ELEMENT, {'value': keys_to_typing(value)})
File "C:\Python2.7.11\lib\site-packages\selenium\webdriver\remote\webelement.py", line 456, in _execute
return self._parent.execute(command, params)
File "C:\Python2.7.11\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "C:\Python2.7.11\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 194, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotVisibleException: Message: Element is not currently visible and so may not be interacted with
Stacktrace:
at fxdriver.preconditions.visible (file:///c:/users/brian/appdata/local/temp/tmpxjigan/extensions/fxdriver#googlecode.com/components/command-processor.js:10092)
at DelayedCommand.prototype.checkPreconditions_ (file:///c:/users/brian/appdata/local/temp/tmpxjigan/extensions/fxdriver#googlecode.com/components/command-processor.js:12644)
at DelayedCommand.prototype.executeInternal_/h (file:///c:/users/brian/appdata/local/temp/tmpxjigan/extensions/fxdriver#googlecode.com/components/command-processor.js:12661)
at DelayedCommand.prototype.executeInternal_ (file:///c:/users/brian/appdata/local/temp/tmpxjigan/extensions/fxdriver#googlecode.com/components/command-processor.js:12666)
at DelayedCommand.prototype.execute/< (file:///c:/users/brian/appdata/local/temp/tmpxjigan/extensions/fxdriver#googlecode.com/components/command-processor.js:12608)
Thanks!
Ps Update, It works with with a webdriver.Chrome() instance but not with Firefox or phantomjs
I suspect you need to make the input visible first:
image = driver.find_element_by_css_selector('#ember1089 input')
driver.execute_script("arguments[0].style.display = 'block';", image);
image.send_keys('C:\Users\Brian\Desktop\Empire_fort\Bots\GetPICtures\Empire.jpg')