Does this line of code work for testing if an element can be clicked.
WebDriverWait(driver, .5).until(expected_conditions.element_to_be_clickable((By.ID, ButtonElm)))
if not what would be the correct way to check if an element is clickable, by using its Element ID
# Set higher wait time
wait=WebDriverWait(driver,10)
try:
element = wait.until(expected_conditions.element_to_be_clickable((By.ID, ButtonElm)))
# Or check Selenium exceptions for clickable, but you could receive either
# not found or not clickable errors here.
except Exception as e:
do stuff
pass
Related
I need to find and store the location of some elements so the bot can click on those elements even the if page changes. I have read online that for a single element storing the location of that element in a variable can help however I could not find a way to store locations of multiple elements in python. Here is my code
comment_button = driver.find_elements_by_css_selector("svg[aria-label='Comment']")
for element in comment_button:
comment_location = element.location
sleep(2)
for element in comment_location:
element.click()
this code gives out this error:
line 44, in <module>
element.click()
AttributeError: 'str' object has no attribute 'click'
Is there a way to do this so when the page refreshes the script can store the locations and move on to the next location to execute element.click without any errors?
I have tried implementing ActionChains into my code
comment_button = driver.find_elements_by_css_selector("svg[aria-label='Comment']")
for element in comment_button:
ac = ActionChains(driver)
element.click()
ac.move_to_element(element).move_by_offset(0, 0).click().perform()
sleep(2)
comment_button = driver.find_element_by_css_selector("svg[aria-label='Comment']")
comment_button.click()
sleep(2)
comment_box = driver.find_element_by_css_selector("textarea[aria-label='Add a comment…']")
comment_box.click()
comment_box = driver.find_element_by_css_selector("textarea[aria-label='Add a comment…']")
comment_box.send_keys("xxxx")
post_button = driver.find_element_by_xpath("//button[#type='submit']")
post_button.click()
sleep(2)
driver.back()
scroll()
However this method gives out the same error saying that the page was refreshed and the object can not be found.
selenium.common.exceptions.StaleElementReferenceException: Message: The element reference of <svg class="_8-yf5 "> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
Edited:
Assuming No. of such elements are not changing after refresh of page, you can use below code
commentbtns = driver.find_elements_by_css_selector("svg[aria-label='Comment']")
for n in range(1, len(commentbtns)+1):
Path = "(//*[name()='svg'])["+str(n)+"]"
time.sleep(2)
driver.find_element_by_xpath(Path).click()
You can use more sophisticated ways to wait for element to load properly. However for simplicity purpose i have used time.sleep.
Trying to break a bigger problem I have into smaller chunks
main question
I am currently inputting a boxer's name into an autocomplete box, selecting the first option that comes up (boxer's name) then clicking view more until I get a list of all the boxer's fights and the view more button stops appearing.
I am then trying to create a list of onclick hrefs I would like to click then iteratively clicking on each and getting the html from each page/fight. I would ideally want to extract the text in particular.
This is the code I have written:
page_link = 'http://beta.compuboxdata.com/fighter'
chromedriver = 'C:\\Users\\User\\Downloads\\chromedriver'
cdriver = webdriver.Chrome(chromedriver)
cdriver.maximize_window()
cdriver.get(page_link)
wait = WebDriverWait(cdriver,20)
wait.until(EC.visibility_of_element_located((By.ID,'s2id_autogen1'))).send_keys('Deontay Wilder')
wait.until(EC.visibility_of_element_located((By.CLASS_NAME, 'select2-result-label'))).click()
while True:
try:
element = wait.until(EC.visibility_of_element_located((By.CLASS_NAME, 'view_more'))).click()
except TimeoutException:
break
# fighters = cdriver.find_elements_by_xpath("//div[#class='row row-bottom-margin-5']/div[2]")
links = [x.get_attribute('onclick') for x in wait.until(EC.visibility_of_element_located((By.XPATH, "//*[contains(#onclick, 'get_fight_report')]/a")))]
htmls = []
for link in links:
cdriver.get(link)
htmls.append(cddriver.page_source)
Running this however gives me the error message:
ElementClickInterceptedException Traceback (most recent call last)
<ipython-input-229-1ee2547c0362> in <module>
10 while True:
11 try:
---> 12 element = wait.until(EC.visibility_of_element_located((By.CLASS_NAME, 'view_more'))).click()
13 except TimeoutException:
14 break
ElementClickInterceptedException: Message: element click intercepted: Element <a class="view_more" href="javascript:void(0);" onclick="_search('0')"></a> is not clickable at point (274, 774). Other element would receive the click: <div class="col-xs-12 col-sm-12 col-md-12 col-lg-12">...</div>
(Session info: chrome=78.0.3904.108)
UPDATE
I have tried looking at a few answers with similar error messages and tried this
while True:
try:
element = cdriver.find_element_by_class_name('view_more')
webdriver.ActionChains(cdriver).move_to_element(element).click(element).perform()
# element = wait.until(EC.visibility_of_element_located((By.CLASS_NAME, 'view_more'))).click()
except TimeoutException:
break
links = [x.get_attribute('onclick') for x in wait.until(EC.visibility_of_element_located((By.XPATH, "//*[contains(#onclick, 'get_fight_report')]/a")))]
htmls = []
for link in links:
cdriver.get(link)
htmls.append(cddriver.page_source)
but this seems to create some sort of infinite loop at the ActionChains point. Seems to be constantly waiting for the view more href to appear
click function should already move the window so the element is in the viewable window. So you don't need that action chain (I think...) but the original error shows some other element OVER the view more button.
You may need to remove (or hide) this element from the DOM, or if it's a html window, "dismiss" it. So pinpointing this covering element is key and then deciding on a strategy to uncover the view more button.
Your site http://beta.compuboxdata.com/fighter doesn't seem to be working at the time so I can't dig in further.
I am using the following code to wait on all 4 elements to be loaded before proceeding with the screen scrape; however, the code is not waiting on all 4 nor is throwing a timeout error -- it just proceeds and I get an error on elements that haven't yet been loaded.
What am I missing to get Selenium to wait until all four elements are present before proceeding?
CSSSelector1_toWaitOn = "#div1 table tbody tr td"
CSSSelector2_toWaitOn = "#div2 table tbody tr:nth-child(5) td"
CSSSelector3_toWaitOn = "#div3 table tbody tr:nth-child(5) td"
CSSSelector4_toWaitOn = "#div4 table tbody tr td"
browser.get(url)
browser_delay = 15 # seconds
try:
WebDriverWait(browser, browser_delay).until(expected_conditions and (
expected_conditions.presence_of_element_located((By.CSS_SELECTOR, CSSSelector1_toWaitOn)) and
expected_conditions.presence_of_element_located((By.CSS_SELECTOR, CSSSelector2_toWaitOn)) and
expected_conditions.presence_of_element_located((By.CSS_SELECTOR, CSSSelector3_toWaitOn)) and
expected_conditions.presence_of_element_located((By.CSS_SELECTOR, CSSSelector4_toWaitOn))))
except TimeoutException:
print("Selenium timeout")```
WebDriverWait.until expects callable object. This is an actual snippet from its source:
while True:
try:
value = method(self._driver)
if value:
return value
All expected_contiditions are callable objects. So in this case you need to compose them, something like following should work.
class composed_expected_conditions:
def __init__(self, expected_conditions):
self.expected_conditions = expected_conditions
def __call__(self, driver):
for expected_condition in self.expected_conditions:
if not expected_condition(driver):
return False
return True
And pass it to the until
conditions = [
expected_conditions.presence_of_element_located((By.CSS_SELECTOR, CSSSelector1_toWaitOn)),
expected_conditions.presence_of_element_located((By.CSS_SELECTOR, CSSSelector2_toWaitOn)),
expected_conditions.presence_of_element_located((By.CSS_SELECTOR, CSSSelector3_toWaitOn)),
expected_conditions.presence_of_element_located((By.CSS_SELECTOR, CSSSelector4_toWaitOn)),
]
WebDriverWait(browser, browser_delay).until(composed_expected_conditions(conditions))
The method presence_of_element_located(locator) only checks that the element is present in the DOM. It does not mean that the element can be interacted with. Furthermore, the search process finds all the elements for given locator and returns first one.
Please check that the element is valid, available, and specific. In case there are multiple elements in the list, make sure your locator is specific enough to find the single element.
Rather than trying to combine them all into a single wait, you can have a separate wait for each.
...
try:
wait = WebDriverWait(browser, browser_delay)
wait.until(expected_conditions.visibility_of_element_located((By.CSS_SELECTOR, CSSSelector1_toWaitOn))
wait.until(expected_conditions.visibility_of_element_located((By.CSS_SELECTOR, CSSSelector2_toWaitOn))
wait.until(expected_conditions.visibility_of_element_located((By.CSS_SELECTOR, CSSSelector3_toWaitOn))
wait.until(expected_conditions.visibility_of_element_located((By.CSS_SELECTOR, CSSSelector4_toWaitOn))))
except TimeoutException:
print("Selenium timeout")
Just be aware, there are 3 levels of interaction within Selenium for elements:
present - the element is in the DOM. If you attempt to click on or get text from, etc. a present (but not visble) element, an ElementNotInteractable exception will be thrown.
visible - the element is in the DOM and visible (e.g. not invisible, display: none, etc.)
clickable - the element is visible and enabled. For most cases, this is just... is it visible? The special cases would be elements like an INPUT button that is marked as disabled. An element that is styled as disabled (greyed out) using CSS, is not considered disabled.
I'm writing a script in to do some webscraping on my Firebase for a few select users. After accessing the events page for a user, I want to check for the condition that no events have been logged by that user first.
For this, I am using Selenium and Python. Using XPath seems to work fine for locating links and navigation in all other parts of the script, except for accessing elements in a table. At first, I thought I might have been using the wrong XPath expression, so I copied the path directly from Chrome's inspection window, but still no luck.
As an alternative, I have tried to copy the page source and pass it into Beautiful Soup, and then parse it there to check for the element. No luck there either.
Here's some of the code, and some of the HTML I'm trying to parse. Where am I going wrong?
# Using WebDriver - always triggers an exception
def check_if_user_has_any_data():
try:
time.sleep(10)
element = WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH, '//*[#id="event-table"]/div/div/div[2]/mobile-table/md-whiteframe/div[1]/ga-no-data-table/div')))
print(type(element))
if element == True:
print("Found empty state by copying XPath expression directly. It is a bit risky, but it seems to have worked")
else:
print("didn’t find empty state")
except:
print("could not find the empty state element", EC)
# Using Beautiful Soup
def check_if_user_has_any_data#2():
time.sleep(10)
html = driver.execute_script("return document.documentElement.outerHTML")
soup = BeautifulSoup(html, 'html.parser')
print(soup.text[:500])
print(len(soup.findAll('div', {"class": "table-row-no-data ng-scope"})))
HTML
<div class="table-row-no-data ng-scope" ng-if="::config" ng-class="{overlay: config.isBuilderOpen()}">
<div class="no-data-content layout-align-center-center layout-row" layout="row" layout-align="center center">
<!-- ... -->
</div>
The first version triggers the exception and is expected to evaluate 'element' as True. Actual, the element is not found.
The second version prints the first 500 characters (correctly, as far as I can tell), but it returns '0'. It is expected to return '1' after inspecting the page source.
Use the following code:
elements = driver.find_elements_by_xpath("//*[#id='event-table']/div/div/div[2]/mobile-table/md-whiteframe/div[1]/ga-no-data-table/div")
size = len(elements)
if len(elements) > 0:
# Element is present. Do your action
else:
# Element is not present. Do alternative action
Note: find_elements will not generate or throw any exception
Here is the method that generally I use.
Imports
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.by import By
Method
def is_element_present(self, how, what):
try:
self.driver.find_element(by=how, value=what)
except NoSuchElementException as e:
return False
return True
Some things load dynamically. It is better to just set a timeout on a wait exception.
If you're using Python and Selenium, you can use this:
try:
driver.find_element_by_xpath("<Full XPath expression>") # Test the element if exist
# <Other code>
except:
# <Run these if element doesn't exist>
I've solved it. The page had a bunch of different iframe elements, and I didn't know that one had to switch between frames in Selenium to access those elements.
There was nothing wrong with the initial code, or the suggested solutions which also worked fine when I tested them.
Here's the code I used to test it:
# Time for the page to load
time.sleep(20)
# Find all iframes
iframes = driver.find_elements_by_tag_name("iframe")
# From inspecting page source, it looks like the index for the relevant iframe is [0]
x = len(iframes)
print("Found ", x, " iFrames") # Should return 5
driver.switch_to.frame(iframes[0])
print("switched to frame [0]")
if WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH, '//*[#class="no-data-title ng-binding"]'))):
print("Found it in this frame!")
Check the length of the element you are retrieving with an if statement,
Example:
element = ('https://www.example.com').
if len(element) > 1:
# Do something.
The following script follows a page in Instagram:
browser = webdriver.Chrome('./chromedriver')
# GO INSTAGRAM PAGE FOR LOGIN
browser.get('https://www.instagram.com/accounts/login/?hl=it')
sleep(2)
# ID AND PASSWORD
elem = browser.find_element_by_name("username").send_keys('test')
elem = browser.find_element_by_name("password").send_keys('passw')
# CLICK BUTTON AND OPEN INSTAGRAM
sleep(5)
good_elem = browser.find_element_by_xpath('//*[#id="react-root"]/section/main/div/article/div/div[1]/div/form/span/button').click()
sleep(5)
browser.get("https://www.instagram.com")
# GO TO PAGE FOR FOLLOW
browser.get("https://www.instagram.com/iam.ai4/")
sleep(28)
segui = browser.find_element_by_class_name('BY3EC').click()
If an element with class BY3EC isn't found I want the script to keep working.
When an element is not found it throws NoSuchElementException, so you can use try/except to avoid that, for example:
from selenium.common.exceptions import NoSuchElementException
try:
segui = browser.find_element_by_class_name('BY3EC').click()
except NoSuchElementException:
print('Element BY3EC not found') # or do something else here
You can take a look at selenium exceptions to get an idea of what each one of them is for.
surround it with try catches, than you can build a happy path and handle failures as well, so your test case will always work
Best practice is to not use Exceptions to control flow. Exceptions should be exceptional... rare and unexpected. The simple way to do this is to get a collection using the locator and then see if the collection is empty. If it is, you know the element doesn't exist.
In the example below we search the page for the element you wanted and check to see that the collection contains an element, if it does... click it.
segui = browser.find_elements_by_class_name('BY3EC')
if segui:
segui[0].click()