Develop Reference

python script to open a page and click download

I am trying to open a page and click on download button. It works fine for the pages that have download element but for the pages which doesn't have that element it raises error
Code:
for i in data["allurl"]:
driver.get('{0}'.format(i))
if(driver.find_element_by_id('ContentPlaceHolder1_grdFileUpload_lnkDownload_0')):
button_element = driver.find_element_by_id('ContentPlaceHolder1_grdFileUpload_lnkDownload_0')
button_element.click()
else:
pass
It should pass instead of raising the error but when I run this it says:
NoSuchElementException: Message: no such element: Unable to locate
element:
{"method":"id","selector":"ContentPlaceHolder1_grdFileUpload_lnkDownload_0"}
How do I solve this?

driver.find_element_by_id() doesn't return True or False as your if-statement expects. Either change your if-statement, or use a try/except statement.
from selenium.common.exceptions import NoSuchElementException
for i in data["allurl"]:
driver.get('{0}'.format(i))
try:
button_element = driver.find_element_by_id('ContentPlaceHolder1_grdFileUpload_lnkDownload_0')
button_element.click()
except NoSuchElementException:
pass

Check the length count of the web element.If it is more than 0 then element available and click otherwise it will go to else condition.
for i in data["allurl"]:
driver.get('{0}'.format(i))
if len(driver.find_elements_by_id('ContentPlaceHolder1_grdFileUpload_lnkDownload_0'))>0:
button_element = driver.find_element_by_id('ContentPlaceHolder1_grdFileUpload_lnkDownload_0')
button_element.click()
else:
pass

from selenium.common.exceptions import NoSuchElementException
try:
button_element = driver.find_element_by_id('ContentPlaceHolder1_grdFileUpload_lnkDownload_0')
except NoSuchElementException:
pass
else:
button_element.click()
Note that even if it worked as you expected, it's inefficient because you perform search for the element twice.
EDIT: included the import statement for the exception
UPDATE: as a side note, assuming elements in data["allurl"] are url (i.e. strings) there is no need for string formatting. driver.get(i) would do. And i is poor choice for variable name - better use something more meaningful....

How can I check if an element exists on a page using Selenium XPath?

I'm writing a script in to do some webscraping on my Firebase for a few select users. After accessing the events page for a user, I want to check for the condition that no events have been logged by that user first.
For this, I am using Selenium and Python. Using XPath seems to work fine for locating links and navigation in all other parts of the script, except for accessing elements in a table. At first, I thought I might have been using the wrong XPath expression, so I copied the path directly from Chrome's inspection window, but still no luck.
As an alternative, I have tried to copy the page source and pass it into Beautiful Soup, and then parse it there to check for the element. No luck there either.
Here's some of the code, and some of the HTML I'm trying to parse. Where am I going wrong?
# Using WebDriver - always triggers an exception
def check_if_user_has_any_data():
try:
time.sleep(10)
element = WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH, '//*[#id="event-table"]/div/div/div[2]/mobile-table/md-whiteframe/div[1]/ga-no-data-table/div')))
print(type(element))
if element == True:
print("Found empty state by copying XPath expression directly. It is a bit risky, but it seems to have worked")
else:
print("didn’t find empty state")
except:
print("could not find the empty state element", EC)
# Using Beautiful Soup
def check_if_user_has_any_data#2():
time.sleep(10)
html = driver.execute_script("return document.documentElement.outerHTML")
soup = BeautifulSoup(html, 'html.parser')
print(soup.text[:500])
print(len(soup.findAll('div', {"class": "table-row-no-data ng-scope"})))
HTML
<div class="table-row-no-data ng-scope" ng-if="::config" ng-class="{overlay: config.isBuilderOpen()}">
<div class="no-data-content layout-align-center-center layout-row" layout="row" layout-align="center center">
<!-- ... -->
</div>
The first version triggers the exception and is expected to evaluate 'element' as True. Actual, the element is not found.
The second version prints the first 500 characters (correctly, as far as I can tell), but it returns '0'. It is expected to return '1' after inspecting the page source.

Use the following code:
elements = driver.find_elements_by_xpath("//*[#id='event-table']/div/div/div[2]/mobile-table/md-whiteframe/div[1]/ga-no-data-table/div")
size = len(elements)
if len(elements) > 0:
# Element is present. Do your action
else:
# Element is not present. Do alternative action
Note: find_elements will not generate or throw any exception

Here is the method that generally I use.
Imports
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.by import By
Method
def is_element_present(self, how, what):
try:
self.driver.find_element(by=how, value=what)
except NoSuchElementException as e:
return False
return True

Some things load dynamically. It is better to just set a timeout on a wait exception.

If you're using Python and Selenium, you can use this:
try:
driver.find_element_by_xpath("<Full XPath expression>") # Test the element if exist
# <Other code>
except:
# <Run these if element doesn't exist>

I've solved it. The page had a bunch of different iframe elements, and I didn't know that one had to switch between frames in Selenium to access those elements.
There was nothing wrong with the initial code, or the suggested solutions which also worked fine when I tested them.
Here's the code I used to test it:
# Time for the page to load
time.sleep(20)
# Find all iframes
iframes = driver.find_elements_by_tag_name("iframe")
# From inspecting page source, it looks like the index for the relevant iframe is [0]
x = len(iframes)
print("Found ", x, " iFrames") # Should return 5
driver.switch_to.frame(iframes[0])
print("switched to frame [0]")
if WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH, '//*[#class="no-data-title ng-binding"]'))):
print("Found it in this frame!")

Check the length of the element you are retrieving with an if statement,
Example:
element = ('https://www.example.com').
if len(element) > 1:
# Do something.

Continue when element is not found in selenium python

The following script follows a page in Instagram:
browser = webdriver.Chrome('./chromedriver')
# GO INSTAGRAM PAGE FOR LOGIN
browser.get('https://www.instagram.com/accounts/login/?hl=it')
sleep(2)
# ID AND PASSWORD
elem = browser.find_element_by_name("username").send_keys('test')
elem = browser.find_element_by_name("password").send_keys('passw')
# CLICK BUTTON AND OPEN INSTAGRAM
sleep(5)
good_elem = browser.find_element_by_xpath('//*[#id="react-root"]/section/main/div/article/div/div[1]/div/form/span/button').click()
sleep(5)
browser.get("https://www.instagram.com")
# GO TO PAGE FOR FOLLOW
browser.get("https://www.instagram.com/iam.ai4/")
sleep(28)
segui = browser.find_element_by_class_name('BY3EC').click()
If an element with class BY3EC isn't found I want the script to keep working.

When an element is not found it throws NoSuchElementException, so you can use try/except to avoid that, for example:
from selenium.common.exceptions import NoSuchElementException
try:
segui = browser.find_element_by_class_name('BY3EC').click()
except NoSuchElementException:
print('Element BY3EC not found') # or do something else here
You can take a look at selenium exceptions to get an idea of what each one of them is for.

surround it with try catches, than you can build a happy path and handle failures as well, so your test case will always work

Best practice is to not use Exceptions to control flow. Exceptions should be exceptional... rare and unexpected. The simple way to do this is to get a collection using the locator and then see if the collection is empty. If it is, you know the element doesn't exist.
In the example below we search the page for the element you wanted and check to see that the collection contains an element, if it does... click it.
segui = browser.find_elements_by_class_name('BY3EC')
if segui:
segui[0].click()

Selenium Python- IndexError: list index out of range on iterating through a table

I am getting IndexError: list index out of range on iterating through html table but I am not sure what is causing it. Below is the function I wrote to iterate through the table.In each iteration it performs a click (one frame of webpage) and downloads files(which is in another frame) . The table has 20 rows. It works fine when the web page loads properly. But once the webpage hangs (ie frame where download happens), my code enters the timeout exception and gets to the top of the code . After that, i get the out of index error on line "employeeList[j].click()" .On debugging, I found out that previous line employeeList returns an empty list . Can someone please explain what is causing this issue.
def candidate():
for j in range(0,20):
driver.implicitly_wait(50)
employeeList=driver.find_elements_by_xpath("//td[7]/div/div[1]/a")
employeeList[j].click()
driver.switch_to_default_content()
driver.implicitly_wait(50)
driver.switch_to.frame("detail")
wait = WebDriverWait(driver,2)
try:
resume = wait.until(EC.presence_of_element_located((By.XPATH,"//div[#id='menubar']/div[1]/div/ul/li[2]/a/span")))
driver.implicitly_wait(50)
resume.click()
download = wait.until(EC.presence_of_element_located((By.XPATH,'//a[#title="Download Resume"]')))
driver.implicitly_wait(50)
download.click()
driver.implicitly_wait(50)
driver.switch_to.frame("RTFVIEWER_MS")
msword = wait.until(EC.presence_of_element_located((By.XPATH,"//div[#id='pagecontainer'>>]/div/a[2]/ul/li[2]")))
driver.implicitly_wait(50)
msword.click()
print(j)
except TimeoutException as ex1:
print("Exception has been thrown"+str(ex1))
print(j)
continue
driver.switch_to_default_content()
driver.switch_to.frame(0)

driver.find_elements_by_xpath("//td[7]/div/div[1]/a") fails after a TimeoutException because some lines get skipped due to how you handle the exception.
The last line of your for-loop sets the driver back to the original frame driver.switch_to.frame(0) where the xpath worked. When you have an exception you are doing a continue in the except-clause and skipping the final lines of the for-loop. Either don't continue or do the needed cleanup in your except-clause.

How can I ignore the exception in Selenium?

I use Python Selenium for scraping a website,
but my crawler stopped because of a exception:
StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
How can i continue to crawl even if the element is not attached?
UPDATE
i change my code to:
try:
libelle1 = prod.find_element_by_css_selector('.em11')
libelle1produit = libelle1.text # libelle1 OK
libelle1produit = libelle1produit.decode('utf-8', 'strict')
except StaleElementReferenceException:
pass
but i have this exception
NoSuchElementException: Message: no such element
i also tried this one:
try:
libelle1 = prod.find_element_by_css_selector('.em11')
libelle1produit = libelle1.text # libelle1 OK
libelle1produit = libelle1produit.decode('utf-8', 'strict')
except :
pass

Put a try-except block around the piece of code that produced that error.

To be more specific about what John Gordon is talking about. Handle the StaleElementReferenceException common selenium exception and ignore it:
from selenium.common.exceptions import StaleElementReferenceException
try:
element.click()
except StaleElementReferenceException: # ignore this error
pass # TODO: consider logging the exception

It looks like the browser rendering engine or Javascript engine is using the element and it is blocking other external operations on this element. You can attempt to access it after some time. If it is not accessible for longer duration, an exception can be thrown. Some good examples are given here.