"selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable"
This is the error i get for no reason at all. i mean earlier it used to work but now it isnt.
In my code there is this website : https://www.tempinbox.xyz/
i open it using the chromedriver.
what i want to do is select (#fitschool.be) from the dropdown menu of domains.
and input some variable string which i have stored as variable "str(res)"
my code is:-
second_tab = webdriver.Chrome(options=options,)
second_tab.get("https://www.tempinbox.xyz/")
randombutton = second_tab.find_element_by_xpath("/html/body/main/div[1]/div/div[1]/div[2]/form[1]/input[2]") #for the email generation
randombutton.send_keys(str(res))
mainurl = second_tab.current_url
selectoption1=second_tab.find_element_by_id("selected-domain").click()
selectoption2=second_tab.find_element_by_css_selector(".domain-selector:nth-child(3)").click()
clickfinal= second_tab.find_element_by_xpath("//input[#value='Create']").click()
emailentry = str(res)+ "#fitschool.be"
print ("your email is " + emailentry)
but it's giving me errors....
Traceback (most recent call last):
File "C:\Users\BRS\Desktop\AccountChef\wattpad acc maker.py", line 45, in <module>
selectoption2=second_tab.find_element_by_css_selector(".domain-selector:nth-child(3)").click()
File "C:\Users\BRS\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "C:\Users\BRS\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "C:\Users\BRS\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\BRS\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
(Session info: chrome=84.0.4147.105)
pls suggest some changes and why this is happening. Thanks :)
Instead of the following two lines:
selectoption1=second_tab.find_element_by_id("selected-domain").click()
selectoption2=second_tab.find_element_by_css_selector(".domain-selector:nth-child(3)").click()
Try to set the innerHTML value with .execute_script:
element = second_tab.find_element_by_id("selected-domain")
target = "#fitschool.be"
second_tab.execute_script("arguments[0].innerHTML = '{}';".format(target), element)
Related
I'm trying to extract 1000 usernames and the team rosters for each username in a list from this SPA (single page application) sports website (requires free login): https://rumble.otmnft.com/contest/live?contest=Rumble
I've tried numerous strategies to try to avoid the common StaleElementReferenceException problem in Selenium including try / except block, WebDriverWait and time.sleep(5) among many others, but none have worked. Would be amazing if someone knows an alternative solution which could work as I've been trying to solve this by myself for quite a few days now.
Here is the relevant part of the latest attempted code which still throws StaleElementReferenceException below:
parent_dict = {}
parent_dict['data'] = []
USERNAME_XPATH = "//span[#class='hidden lg:flex font-semibold']"
ROSTER_XPATH = "//span[#class='text-white hidden lg:flex font-semibold']"
WebDriverWait(driver, 50).until(EC.presence_of_element_located((By.XPATH, "//table")))
username_elements = driver.find_elements(By.XPATH, USERNAME_XPATH)
try:
for usr_element in username_elements:
usr_element.click()
except StaleElementReferenceException:
# Find the element again and continue with the loop
WebDriverWait(driver, 50).until(EC.presence_of_element_located((By.XPATH, USERNAME_XPATH)))
username_elements = driver.find_elements(By.XPATH, USERNAME_XPATH)
time.sleep(5)
for usr_element in username_elements:
usr_element.click()
WebDriverWait(driver, 50).until(EC.presence_of_element_located((By.XPATH, ROSTER_XPATH)))
roster_elements = driver.find_elements(By.XPATH, ROSTER_XPATH)
roster = [cell.text for cell in roster_elements]
TITLE_USER_XPATH = "//div[#class='flex flex-col space-y-2']/span"
title_user = driver.find_element(By.XPATH, TITLE_USER_XPATH)
child_dict = {"username": title_user.text, "roster": roster }
print("CHILD_DICT", child_dict)
parent_dict['data'].append(child_dict)
with open('data_v2.json', 'w') as outfile:
json.dump(parent_dict, outfile)
print("PARENT_DICT", parent_dict)
return parent_dict
In the browser I can see that the script is able to click on the very first username but it fails when trying click on the second username, but reports the error for the line usr_element.click() both inside the try and except blocks
Full error trace is:
Traceback (most recent call last):
File "/Users/danpro12/Dropbox/01 FREE TIME STUFF/01 WEB DEVELOPMENT/12 DAPPER RELATED/otm_python_scraper/otm_specific/utils.py", line 23, in sub_func
usr_element.click()
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 88, in click
self._execute(Command.CLICK_ELEMENT)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 396, in _execute
return self._parent.execute(command, params)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
self.error_handler.check_response(response)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: The element reference of <span class="hidden lg:flex font-semibold"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
Stacktrace:
RemoteError#chrome://remote/content/shared/RemoteError.sys.mjs:8:8
WebDriverError#chrome://remote/content/shared/webdriver/Errors.sys.mjs:180:5
StaleElementReferenceError#chrome://remote/content/shared/webdriver/Errors.sys.mjs:461:5
element.resolveElement#chrome://remote/content/marionette/element.sys.mjs:674:11
evaluate.fromJSON#chrome://remote/content/marionette/evaluate.sys.mjs:255:31
evaluate.fromJSON#chrome://remote/content/marionette/evaluate.sys.mjs:263:29
receiveMessage#chrome://remote/content/marionette/actors/MarionetteCommandsChild.sys.mjs:74:34
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/danpro12/Dropbox/01 FREE TIME STUFF/01 WEB DEVELOPMENT/12 DAPPER RELATED/otm_python_scraper/otm_specific/rumble_client.py", line 53, in <module>
sub_func(driver)
File "/Users/danpro12/Dropbox/01 FREE TIME STUFF/01 WEB DEVELOPMENT/12 DAPPER RELATED/otm_python_scraper/otm_specific/utils.py", line 30, in sub_func
usr_element.click()
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 88, in click
self._execute(Command.CLICK_ELEMENT)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 396, in _execute
return self._parent.execute(command, params)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
self.error_handler.check_response(response)
File "/Users/danpro12/.virtualenvs/otm/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: The element reference of <span class="hidden lg:flex font-semibold"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
Stacktrace:
RemoteError#chrome://remote/content/shared/RemoteError.sys.mjs:8:8
WebDriverError#chrome://remote/content/shared/webdriver/Errors.sys.mjs:180:5
StaleElementReferenceError#chrome://remote/content/shared/webdriver/Errors.sys.mjs:461:5
element.resolveElement#chrome://remote/content/marionette/element.sys.mjs:674:11
evaluate.fromJSON#chrome://remote/content/marionette/evaluate.sys.mjs:255:31
evaluate.fromJSON#chrome://remote/content/marionette/evaluate.sys.mjs:263:29
receiveMessage#chrome://remote/content/marionette/actors/MarionetteCommandsChild.sys.mjs:74:34
Seems that after you do usr_element.click() DOM updated and so you cannot iterate over username_elements since all elements in this list are stale.
Try this solution
username_elements_length = len(driver.find_elements(By.XPATH, USERNAME_XPATH))
for usr_element_index in range(username_elements_length):
username_element = driver.find_elements(By.XPATH, USERNAME_XPATH)[usr_element_index]
username_element.click()
WebDriverWait(driver, 5).until(EC.staleness_of(username_element))
Here is the code which I use to open the new tab using Selenium Python,
self.driver.execute_script("window.open(arguments[0])", self.listing_url)
self.driver.switch_to.window(self.driver.window_handles[1])
time.sleep(10)
and it works perfectly. But I am planning to use WebDriverWait instead of time.sleep
self.driver.execute_script("window.open(arguments[0])", self.listing_url)
self.driver.switch_to.window(self.driver.window_handles[1])
WebDriverWait(self.driver, 20).until(EC.presence_of_element_located((By.XPATH, '//div[#class="honbun"]')))
and when I am using it, it's throwing an error. Here is the error log:
Traceback (most recent call last):
File "D:\tender_scraper\engine\Spider.py", line 308, in parse_index_page
index_handle = self.load_listing_page(listing, i, index_listing)
File "D:\tender_scraper\spiders\gunmapref_listing_spider.py", line 298, in load_listing_page
WebDriverWait(self.driver, 20).until(EC.presence_of_element_located((By.XPATH, '//div[#class="honbun"]')))
File "D:\scraper\lib\site-packages\selenium\webdriver\support\wait.py", line 78, in until
value = method(self._driver)
File "D:\scraper\lib\site-packages\selenium\webdriver\support\expected_conditions.py", line 64, in _predicate
return driver.find_element(*locator)
File "D:\scraper\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 1251, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "D:\scraper\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 430, in execute
self.error_handler.check_response(response)
File "D:\scraper\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: missing or invalid columnNumber
(Session info: headless chrome=104.0.5112.48)
Any idea what's wrong with it & how to solve it? Previously, I used chrome 103 and changed it to 104 but still got the same error. It seems the error only occurs when changing into the new tab.
I followed this tutorial for Selenium with Python:
So I did the same as the tutorial:
from selenium import webdriver
driver=webdriver.Chrome()
driver.get('https://www.youtube.com/')
searchbox = driver.find_element_by_xpath('//*[#id="search-input"]')
searchbox.send_keys('cooking')
But I get this error:
Traceback (most recent call last):
File "C:/ANGUS/DATA_ANALYSIS/WEBSCRAPING/TextXXX.py", line 7, in <module>
searchbox.send_keys('cooking')
File "C:\Users\Angus\PycharmProjects\Test\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 477, in send_keys
self._execute(Command.SEND_KEYS_TO_ELEMENT,
File "C:\Users\Angus\PycharmProjects\Test\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "C:\Users\Angus\PycharmProjects\Test\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\Angus\PycharmProjects\Test\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
(Session info: chrome=81.0.4044.138)
How can I fix this?
Another way to do that in order not to use the search is by inserting the query itself.
Code:
your_query = "cooking"
driver.get("https://www.youtube.com/results?search_query=" + str(your_query))
time.sleep(5)
In this way, it is like you are going immediately in the output that you want after searching a specific query of yours. So, let's say that instead of typing cooking, you can .get() to the URL you want by adding your query into this URL.
This is the code that I am using trying to pass a variable into the xpath.
def findcase(year):
select = Select(driver.find_element_by_id('SheetContentPlaceHolder_coaCaseSearch_ddlCaseYear'))
select.select_by_value(str(year))
Getting the following error message:
Traceback (most recent call last):
File "/Users/jackboland/Library/Mobile Documents/com~apple~CloudDocs/Python/Selenium/8thDistBriefScraping_FUNCTIONVersion.py", line 27, in <module>
findcase('2013','100604')
File "/Users/jackboland/Library/Mobile Documents/com~apple~CloudDocs/Python/Selenium/8thDistBriefScraping_FUNCTIONVersion.py", line 18, in findcase
select = Select(driver.find_element_by_id('SheetContentPlaceHolder_coaCaseSearch_ddlCaseYear'))
File "/anaconda2/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 353, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "/anaconda2/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 957, in find_element
'value': value})['value']
File "/anaconda2/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 314, in execute
self.error_handler.check_response(response)
File "/anaconda2/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"id","selector":"SheetContentPlaceHolder_coaCaseSearch_ddlCaseYear"}
(Session info: chrome=66.0.3359.181)
(Driver info: chromedriver=2.38.552518 (183d19265345f54ce39cbb94cf81ba5f15905011),platform=Mac OS X 10.13.4 x86_64)
I have tried the string in quotes (double and single) I have tried inserting the variable with a str(variable). Nothing doin' Cannot get the function to permit the passing in of a variable.
This is the relevant HTML. But, note that when I pass in an actual number, a year, the above code works perfectly. It only fails when trying to pass in a variable for that number.
//*[#id="SheetContentPlaceHolder_coaCaseSearch_ddlCaseYear"]
I figured this out. The code that works is below:
year = 2014
yearstr = str(year)
casenum = 100604
casenumstr = str(casenum)
select = Select(driver.find_element_by_id('SheetContentPlaceHolder_coaCaseSearch_ddlCaseYear'))
select.select_by_value(yearstr)
#inserts the case number in that box below the year
driver.find_element_by_id('SheetContentPlaceHolder_coaCaseSearch_txtCaseNum').send_keys(casenumstr)
I'm new to python coding. And I'm trying to play around with python and Selenium in chrome. And I have this code where I'm trying to move a webpage up and down. But it isn't working.
body_elem = browser.find_element_by_tag_name('body')
for __ in range(3):
body_elem.send_keys(Keys.END)
sleep(2)
body_elem.send_keys(Keys.HOME)
sleep(2)
All I get is this long error message:
Traceback (most recent call last):
File "selenium_test.py", line 34, in <module>
body_elem.send_keys(Keys.HOME)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/selenium/webdriver/remote/webelement.py", line 479, in send_keys
'value': keys_to_typing(value)})
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/selenium/webdriver/remote/webelement.py", line 628, in _execute
return self._parent.execute(command, params)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 312, in execute
self.error_handler.check_response(response)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
Anybody got an idea of whats is wrong?
This is most likely because of the fact that the DOM tree or the body element signature itself changed after the first "down" move. One straightforward way to approach the problem would be to "re-find" the element constantly:
for _ in range(3):
browser.find_element_by_tag_name('body').send_keys(Keys.END)
sleep(2)
browser.find_element_by_tag_name('body').send_keys(Keys.HOME)
sleep(2)