I use Python Selenium for scraping a website,
but my crawler stopped because of a exception:
StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
How can i continue to crawl even if the element is not attached?
UPDATE
i change my code to:
try:
libelle1 = prod.find_element_by_css_selector('.em11')
libelle1produit = libelle1.text # libelle1 OK
libelle1produit = libelle1produit.decode('utf-8', 'strict')
except StaleElementReferenceException:
pass
but i have this exception
NoSuchElementException: Message: no such element
i also tried this one:
try:
libelle1 = prod.find_element_by_css_selector('.em11')
libelle1produit = libelle1.text # libelle1 OK
libelle1produit = libelle1produit.decode('utf-8', 'strict')
except :
pass
Put a try-except block around the piece of code that produced that error.
To be more specific about what John Gordon is talking about. Handle the StaleElementReferenceException common selenium exception and ignore it:
from selenium.common.exceptions import StaleElementReferenceException
try:
element.click()
except StaleElementReferenceException: # ignore this error
pass # TODO: consider logging the exception
It looks like the browser rendering engine or Javascript engine is using the element and it is blocking other external operations on this element. You can attempt to access it after some time. If it is not accessible for longer duration, an exception can be thrown. Some good examples are given here.
Related
I am trying to open a page and click on download button. It works fine for the pages that have download element but for the pages which doesn't have that element it raises error
Code:
for i in data["allurl"]:
driver.get('{0}'.format(i))
if(driver.find_element_by_id('ContentPlaceHolder1_grdFileUpload_lnkDownload_0')):
button_element = driver.find_element_by_id('ContentPlaceHolder1_grdFileUpload_lnkDownload_0')
button_element.click()
else:
pass
It should pass instead of raising the error but when I run this it says:
NoSuchElementException: Message: no such element: Unable to locate
element:
{"method":"id","selector":"ContentPlaceHolder1_grdFileUpload_lnkDownload_0"}
How do I solve this?
driver.find_element_by_id() doesn't return True or False as your if-statement expects. Either change your if-statement, or use a try/except statement.
from selenium.common.exceptions import NoSuchElementException
for i in data["allurl"]:
driver.get('{0}'.format(i))
try:
button_element = driver.find_element_by_id('ContentPlaceHolder1_grdFileUpload_lnkDownload_0')
button_element.click()
except NoSuchElementException:
pass
Check the length count of the web element.If it is more than 0 then element available and click otherwise it will go to else condition.
for i in data["allurl"]:
driver.get('{0}'.format(i))
if len(driver.find_elements_by_id('ContentPlaceHolder1_grdFileUpload_lnkDownload_0'))>0:
button_element = driver.find_element_by_id('ContentPlaceHolder1_grdFileUpload_lnkDownload_0')
button_element.click()
else:
pass
from selenium.common.exceptions import NoSuchElementException
try:
button_element = driver.find_element_by_id('ContentPlaceHolder1_grdFileUpload_lnkDownload_0')
except NoSuchElementException:
pass
else:
button_element.click()
Note that even if it worked as you expected, it's inefficient because you perform search for the element twice.
EDIT: included the import statement for the exception
UPDATE: as a side note, assuming elements in data["allurl"] are url (i.e. strings) there is no need for string formatting. driver.get(i) would do. And i is poor choice for variable name - better use something more meaningful....
The following script follows a page in Instagram:
browser = webdriver.Chrome('./chromedriver')
# GO INSTAGRAM PAGE FOR LOGIN
browser.get('https://www.instagram.com/accounts/login/?hl=it')
sleep(2)
# ID AND PASSWORD
elem = browser.find_element_by_name("username").send_keys('test')
elem = browser.find_element_by_name("password").send_keys('passw')
# CLICK BUTTON AND OPEN INSTAGRAM
sleep(5)
good_elem = browser.find_element_by_xpath('//*[#id="react-root"]/section/main/div/article/div/div[1]/div/form/span/button').click()
sleep(5)
browser.get("https://www.instagram.com")
# GO TO PAGE FOR FOLLOW
browser.get("https://www.instagram.com/iam.ai4/")
sleep(28)
segui = browser.find_element_by_class_name('BY3EC').click()
If an element with class BY3EC isn't found I want the script to keep working.
When an element is not found it throws NoSuchElementException, so you can use try/except to avoid that, for example:
from selenium.common.exceptions import NoSuchElementException
try:
segui = browser.find_element_by_class_name('BY3EC').click()
except NoSuchElementException:
print('Element BY3EC not found') # or do something else here
You can take a look at selenium exceptions to get an idea of what each one of them is for.
surround it with try catches, than you can build a happy path and handle failures as well, so your test case will always work
Best practice is to not use Exceptions to control flow. Exceptions should be exceptional... rare and unexpected. The simple way to do this is to get a collection using the locator and then see if the collection is empty. If it is, you know the element doesn't exist.
In the example below we search the page for the element you wanted and check to see that the collection contains an element, if it does... click it.
segui = browser.find_elements_by_class_name('BY3EC')
if segui:
segui[0].click()
I have a page where the number of elements to loop through is not easily known. As such I’ve made the job loop through Href “a” (364 matches) and when it cannot find the Next button, I am wanting the job to come to an end.
I have tried:
try:
element = driver.find_element_by_xpath('//span[text()="Next Page"]')
except NoSuchElementException:
pass
#except IOError:
#pass
#except OSError:
#pass
As well as other variations and indenting.
My full code is here (it does not allow me to post it here fully due to character limit)
Below code should allow you to close browser and WebDriver session once you reach last page
from selenium.common.exceptions import NoSuchElementException
try:
element = driver.find_element_by_xpath('//span[text()="Next Page"]')
except NoSuchElementException:
driver.quit()
I'm trying the use a try/except statement to findout if an element exists in the WebDrive or not, if so then run a specific code line,
try:
WebDriver.find_element_by_css_selector('div[class="..."')
except NoSuchElement:
ActionToRunInCaseNoSuchElementTrue
else:
ActionToRunInCaseNoSuchElementFalse
but running this code gives an error:
NameError: name 'NoSuchElement' is not defined
how should the Exception be defined?
Is there any shorter/easier way to check if an element does exist in a web page and run a command if so and another if not?
To be able to use required exception you have to import it first with correct name (NoSuchElement -> NoSuchElementException):
from selenium.common.exceptions import NoSuchElementException
try:
WebDriver.find_element_by_css_selector('div[class="..."')
except NoSuchElementException:
ActionToRunInCaseNoSuchElementTrue
Instead of using try except you can use find_elements and check if the returned list has any elements
elements = WebDriver.find_elements_by_css_selector('div[class="..."')
if not elements:
ActionToRunInCaseNoSuchElementTrue
else:
ActionToRunInCaseNoSuchElementFalse
# the element is in elements[0]
I am attempting to validate that text is present on a page. Validating an element by ID is simple enough, buy trying to do it with text isn't working right. And, I can not locate the correct attribute for By to validate text on a webpage.
Example that works for ID using By attribute
self.assertTrue(self.is_element_present(By.ID, "FOO"))
Example I am trying to use (doesn't work) for text using By attribute
self.assertTrue(self.is_element_present(By.TEXT, "BAR"))
I've tried these as well, with *error (below)
self.assertTrue(self.is_text_present("FOO"))
and
self.assertTrue(self.driver.is_text_present("FOO"))
*error: AttributeError: 'WebDriver' object has no attribute 'is_element_present'
I have the same issue when trying to validate By.Image as well.
From what I have seen, is_element_present is generated by a Firefox extension (Selenium IDE) and looks like:
def is_element_present(self, how, what):
try: self.driver.find_element(by=how, value=what)
except NoSuchElementException: return False
return True
"By" is imported from selenium.webdriver.common:
from selenium.webdriver.common.by import By
from selenium.common.exceptions import NoSuchElementException
There are several "By" constants to address each API find_element_by_* so, for example:
self.assertTrue(self.is_element_present(By.LINK_TEXT, "My link"))
verifies that a link exists and, if it doesn't, avoids an exception raised by selenium, thus allowing a proper unittest behaviour.
First of all, it's discouraged to do so, it's better to change your testing logic than finding text in page.
Here's how you create you own is_text_present method though, if you really want to use it:
def is_text_present(self, text):
try:
body = self.driver.find_element_by_tag_name("body") # find body tag element
except NoSuchElementException, e:
return False
return text in body.text # check if the text is in body's text
For images, the logic is you pass the locator into it. (I don't think is_element_present exists in WebDriver API though, not sure how you got By.ID working, let's assume it's working.)
self.assertTrue(self.is_element_present(By.ID, "the id of your image"))
# alternatively, there are others like CSS_SELECTOR, XPATH, etc.
# self.assertTrue(self.is_element_present(By.CSS_SELECTOR, "the css selector of your image"))
I like wrapping the whole thing into a custom assertion
from selenium.common.exceptions import NoSuchElementException
def assertElementIsPresentByXPath(self, xpath, msg=None):
try:
self.browser.find_element_by_xpath(xpath)
self.assertTrue(True, msg)
except NoSuchElementException:
self.assertTrue(False, msg)
def test_element_10_should_exists(self):
self.browser.get('url/to/test')
self.assertElementIsPresentByXPath('//a[#id=el_10]')