Develop Reference

Click multiple elements at the same time selenium Python

I am trying to click multiple elements at the same time without any delay in between.
For example, instead of 1 then 2, it should be 1 and 2.
this is the 1st element that I want to click:
WebDriverWait(driver, 1).until(EC.element_to_be_clickable(
(By.XPATH,
"//div[contains(#class, 'item-button')]//div[contains(#class, 'button-game')]"))).click()
this is the 2nd elements that I want to click:
(run the first line then second line)
WebDriverWait(driver, 1).until(
EC.frame_to_be_available_and_switch_to_it((By.XPATH, "/html/body/div[4]/div[4]/iframe")))
WebDriverWait(driver, 1.4).until(EC.element_to_be_clickable(
(By.XPATH, "//*[#id='rc-imageselect']/div[3]/div[2]/div[1]/div[1]/div[4]"))).click()
Basically, Click 1st element and 2nd elements' first line then second line.
I have tried this, but did not work:
from threading import Thread
def func1():
WebDriverWait(driver, 1).until(EC.element_to_be_clickable(
(By.XPATH,
"//div[contains(#class, 'item-button')]//div[contains(#class, 'button-game')]"))).click()
def func2():
WebDriverWait(driver, 1).until(
EC.frame_to_be_available_and_switch_to_it((By.XPATH, "/html/body/div[4]/div[4]/iframe")))
WebDriverWait(driver, 1.4).until(EC.element_to_be_clickable(
(By.XPATH, "//*[#id='rc-imageselect']/div[3]/div[2]/div[1]/div[1]/div[4]"))).click()
if __name__ == '__main__':
Thread(target = func1).start()
Thread(target = func2).start()
Use case: I am trying to automate a website and I need to be fast. Sometimes, element1 is not showing, the website shows element2 instead and vice versa. If I did not check element1 and element2 at the same time, I will be late. The code above starts function1 before function2 and not at the same time. Thank you

You can make it simultaneous if you use jQuery:
click_script = """jQuery('%s').click();""" % css_selector
driver.execute_script(click_script)
That's going to click all elements that match that selector at the same time, assuming you can find a common selector between all the elements that you want to click. You may also need to escape quotes before feeding that selector into the execute_script. And you may need to load jQuery if it wasn't already loaded. If there's no common selector between the elements that you want to click simultaneously, then you can use javascript to set a common attribute between them.
You can also try it with JS execution, which might be fast enough:
script = (
"""var simulateClick = function (elem) {
var evt = new MouseEvent('click', {
bubbles: true,
cancelable: true,
view: window
});
var canceled = !elem.dispatchEvent(evt);
};
var $elements = document.querySelectorAll('%s');
var index = 0, length = $elements.length;
for(; index < length; index++){
simulateClick($elements[index]);}"""
% css_selector
)
driver.execute_script(script)
As before, you'll need to escape quotes and special characters first.
Using import re; re.escape(STRING) can be used for that.
All of this will be made easier if you use the Selenium Python framework: SeleniumBase, which has build-in methods for simultaneous clicking:
self.js_click_all(selector)
self.jquery_click_all(selector)
And each of those above will automatically escape quotes of selectors before running driver.execute_script(SCRIPT), and will also load jQuery if it wasn't already loaded on the current page. If the elements above didn't already have a common selector, you can use self.set_attribute(selector, attribute, value) in order to create a common one before running one of the simultaneous click methods.

Simultaneous clicking is possible with selenium ActionChains class.
Reference:
https://github.com/SeleniumHQ/selenium/blob/64447d4b03f6986337d1ca8d8b6476653570bcc1/py/selenium/webdriver/common/actions/pointer_input.py#L24
And here is code example, in which 2 clicks will be performed on 2 different elements at the same time:
from selenium.webdriver.common.actions.mouse_button import MouseButton
from selenium.webdriver.common.action_chains import ActionChains
b1 = driver.find_element(By.ID, 'SomeButtonId')
b2 = driver.find_element(By.ID, 'btnHintId')
location1 = b1.rect
location2 = b2.rect
actions = ActionChains(driver)
actions.w3c_actions.devices = []
new_input = actions.w3c_actions.add_pointer_input('touch', 'finger1')
new_input.create_pointer_move(x=location1['x'] + 1, y=location1['y'] + 2)
new_input.create_pointer_down(MouseButton.LEFT)
new_input.create_pointer_up(MouseButton.LEFT)
new_input2 = actions.w3c_actions.add_pointer_input('touch', 'finger2')
new_input2.create_pointer_move(x=location2['x'] + 1, y=location2['y'] + 2)
new_input2.create_pointer_down(MouseButton.LEFT)
new_input2.create_pointer_up(MouseButton.LEFT)
actions.perform()

You can use ActionChains to perform multiple actions almost immediately.
actions = ActionChains(driver)
actions.move_to_element(element1).click()
actions.move_to_element(element2).click()
actions.perform()
Depending on the use case, you could also use click_and_hold which is also available using ActionChains.

Can You Use Element ID to Check if the Element is Clickable

Does this line of code work for testing if an element can be clicked.
WebDriverWait(driver, .5).until(expected_conditions.element_to_be_clickable((By.ID, ButtonElm)))
if not what would be the correct way to check if an element is clickable, by using its Element ID

# Set higher wait time
wait=WebDriverWait(driver,10)
try:
element = wait.until(expected_conditions.element_to_be_clickable((By.ID, ButtonElm)))
# Or check Selenium exceptions for clickable, but you could receive either
# not found or not clickable errors here.
except Exception as e:
do stuff
pass

How can I check if an element exists on a page using Selenium XPath?

I'm writing a script in to do some webscraping on my Firebase for a few select users. After accessing the events page for a user, I want to check for the condition that no events have been logged by that user first.
For this, I am using Selenium and Python. Using XPath seems to work fine for locating links and navigation in all other parts of the script, except for accessing elements in a table. At first, I thought I might have been using the wrong XPath expression, so I copied the path directly from Chrome's inspection window, but still no luck.
As an alternative, I have tried to copy the page source and pass it into Beautiful Soup, and then parse it there to check for the element. No luck there either.
Here's some of the code, and some of the HTML I'm trying to parse. Where am I going wrong?
# Using WebDriver - always triggers an exception
def check_if_user_has_any_data():
try:
time.sleep(10)
element = WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH, '//*[#id="event-table"]/div/div/div[2]/mobile-table/md-whiteframe/div[1]/ga-no-data-table/div')))
print(type(element))
if element == True:
print("Found empty state by copying XPath expression directly. It is a bit risky, but it seems to have worked")
else:
print("didn’t find empty state")
except:
print("could not find the empty state element", EC)
# Using Beautiful Soup
def check_if_user_has_any_data#2():
time.sleep(10)
html = driver.execute_script("return document.documentElement.outerHTML")
soup = BeautifulSoup(html, 'html.parser')
print(soup.text[:500])
print(len(soup.findAll('div', {"class": "table-row-no-data ng-scope"})))
HTML
<div class="table-row-no-data ng-scope" ng-if="::config" ng-class="{overlay: config.isBuilderOpen()}">
<div class="no-data-content layout-align-center-center layout-row" layout="row" layout-align="center center">
<!-- ... -->
</div>
The first version triggers the exception and is expected to evaluate 'element' as True. Actual, the element is not found.
The second version prints the first 500 characters (correctly, as far as I can tell), but it returns '0'. It is expected to return '1' after inspecting the page source.

Use the following code:
elements = driver.find_elements_by_xpath("//*[#id='event-table']/div/div/div[2]/mobile-table/md-whiteframe/div[1]/ga-no-data-table/div")
size = len(elements)
if len(elements) > 0:
# Element is present. Do your action
else:
# Element is not present. Do alternative action
Note: find_elements will not generate or throw any exception

Here is the method that generally I use.
Imports
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.by import By
Method
def is_element_present(self, how, what):
try:
self.driver.find_element(by=how, value=what)
except NoSuchElementException as e:
return False
return True

Some things load dynamically. It is better to just set a timeout on a wait exception.

If you're using Python and Selenium, you can use this:
try:
driver.find_element_by_xpath("<Full XPath expression>") # Test the element if exist
# <Other code>
except:
# <Run these if element doesn't exist>

I've solved it. The page had a bunch of different iframe elements, and I didn't know that one had to switch between frames in Selenium to access those elements.
There was nothing wrong with the initial code, or the suggested solutions which also worked fine when I tested them.
Here's the code I used to test it:
# Time for the page to load
time.sleep(20)
# Find all iframes
iframes = driver.find_elements_by_tag_name("iframe")
# From inspecting page source, it looks like the index for the relevant iframe is [0]
x = len(iframes)
print("Found ", x, " iFrames") # Should return 5
driver.switch_to.frame(iframes[0])
print("switched to frame [0]")
if WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH, '//*[#class="no-data-title ng-binding"]'))):
print("Found it in this frame!")

Check the length of the element you are retrieving with an if statement,
Example:
element = ('https://www.example.com').
if len(element) > 1:
# Do something.

Selenium (python) explicit wait timeout error

I'm trying to automate a process that utilizes a webserver for various computational tasks. There are multiple tasks on multiple pages, so I'm using explicit waits. This works for everything except one particular task, which takes 5-7 minutes to complete (much longer than anything else).
Whenever I try the following:
def next5():
try:
myElem5 = WebDriverWait(driver, 600).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#fmdsetup > table > tbody > tr:nth-child(2) > td > input[type="radio"]')))
next5 = driver.find_element_by_class_name('nav_entry')
next5.click()
except TimeoutException:
print("Timed out waiting for page to load (next5)")
I receive the TimeoutException print statement. I've tried longer wait times, but it always times out. The weird thing is that it throws back the TimeoutException before 10 minutes has passed.
Also, the script works perfectly if I simply use a sleep function:
def next5():
time.sleep(600)
next5 = driver.find_element_by_class_name('nav_entry')
next5.click()
As I said before, I have multiple instances of next<#>() functions that work perfectly, and only differ in the wait time. What's different about this situation, and how can I make it work?

Since:
def next5():
try:
myElem5 = WebDriverWait(driver, 600).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#fmdsetup > table > tbody > tr:nth-child(2) > td > input[type="radio"]')))
next5 = driver.find_element_by_class_name('nav_entry')
next5.click()
except TimeoutException:
print("Timed out waiting for page to load (next5)")
and
def next5():
time.sleep(600)
next5 = driver.find_element_by_class_name('nav_entry')
next5.click()
are different and the first one does not work and the second one works, I can say that the problem is with myElem5 element. It is not clickable at all, so it makes no sense to add wait time, or this element doesn't exists in the DOM. What exactly is in your case, I cannot say. Try to look exactly, if element is in the DOM and it is clickable. Try to do the same manually and than debug. I think you will find a problem.
EDIT: according your feedback you do like this:
def next5(counter=1):
try:
if counter == 5:
WebDriverWait(driver, 10).until(EC.url_contains(("the new url, where elem 5 exists")))
myElem5 = WebDriverWait(driver, 10).until(EC.element_to_be_clickable(
(By.CSS_SELECTOR, '#fmdsetup > table > tbody > tr:nth-child(2) > td > input[type="radio"]')))
next5 = driver.find_element_by_class_name('nav_entry')
next5.click()
counter += 1
except TimeoutException:
print("Timed out waiting for page to load (next5)")
PS make sure that this element is not in iframe/frame if yes, firstly you have to switch on it and only then this element will be ready to interact with it. You can use this approach:
WebDriverWait(driver, 600).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//xpath/to/frame")))
# do your stuff
driver.switch_to.default_content()

Scraper: Try skips code in while loop (Python)

I am working on my first scraper and ran into an issue. My scraper accesses a website and saves links from the each result page. Now, I only want it to go through 10 pages. The problem comes when the search results has less than 10 pages. I tried using a while loop along with a try statement, but it does not seem to work. After the scraper goes through the first page of results, it does not return any links on the successive pages; however, it does not give me an error and stops once it reaches 10 pages or the exception.
Here is a snippet of my code:
links = []
page = 1
while(page <= 10):
try:
# Get information from the propertyInfo class
properties = WebDriverWait(driver, 10).until(lambda driver: driver.find_elements_by_xpath('//div[#class = "propertyInfo item"]'))
# For each listing
for p in properties:
# Find all elements with a tags
tmp_link = p.find_elements_by_xpath('.//a')
# Get the link from the second element to avoid error
links.append(tmp_link[1].get_attribute('href'))
page += 1
WebDriverWait(driver, 10).until(lambda driver: driver.find_element_by_xpath('//*[#id="paginador_siguiente"]/a').click())
except ElementNotVisibleException:
break
I really appreciate any pointers on how to fix this issue.

You are explicitely catching ElementNotVisibleException exception and stopping on it. This way you won't see any error message. The error is probably in this line:
WebDriverWait(driver, 10).until(lambda driver:
driver.find_element_by_xpath('//*[#id="paginador_siguiente"]/a').click())
I assume lambda here should be a test, which is run until succeeded. So it shouldn't make any action like click. I actually believe that you don't need to wait here at all, page should be already fully loaded so you can just click on the link:
driver.find_element_by_xpath('//*[#id="paginador_siguiente"]/a').click()
This will either pass to next page (and WebDriverWait at the start of the loop will wait for it) or raise exception if no next link is found.
Also, you better minimize try ... except scope, this way you won't capture something unintentionally. E.g. here you only want to surround next link finding code not the whole loop body:
# ...
while(page <= 10):
# Scrape this page
properties = WebDriverWait(driver, 10).until(...)
for p in properties:
# ...
page += 1
# Try to pass to next page
try:
driver.find_element_by_xpath('//*[#id="paginador_siguiente"]/a').click()
except ElementNotVisibleException:
# Break if no next link is found
break