I have been tasked with writing a parser to click a button on a website and I am having issues to click only one of the buttons. The following code works on every button except one.
Here's the html:
http://pastebin.com/6dLF5ru8
here's the source html:
http://pastebin.com/XhsedGLb
python code:
driver = webdriver.Firefox()
...
el = driver.find_element_by_id("-spel-nba")
actions.move_to_element(el)
actions.sleep(.1)
actions.click()
actions.perform()
I am getting this error.
ElementNotVisibleException: Message: Element is not currently visible and so may not be interacted with
as per Saifur I just tried waits with the same element not visible exception:
wait = WebDriverWait(driver, 10)
wait.until(EC.presence_of_element_located((By.XPATH, "//input[contains(#id,'spsel')][#value='nba']"))).click()
If you look at the page source, you'll understand that almost all of theSELECT, DIV elements are faked and created from JavaScript, that is why webdriver cannot SEE them.
There's a workaround though, by using ActionChains to open your developer console, and inject an artificial CLICK on the desired element, which in fact, is the Label triggering the NBA data loading... here's a working example:
from selenium import webdriver
from selenium.webdriver.common import action_chains, keys
import time
driver = webdriver.Firefox()
driver.get('Your URL here...')
assert 'NBA' in driver.page_source
action = action_chains.ActionChains(driver)
# open up the developer console, mine on MAC, yours may be diff key combo
action.send_keys(keys.Keys.COMMAND+keys.Keys.ALT+'i')
action.perform()
time.sleep(3)
# this below ENTER is to rid of the above "i"
action.send_keys(keys.Keys.ENTER)
# inject the JavaScript...
action.send_keys("document.querySelectorAll('label.boxed')[1].click()"+keys.Keys.ENTER)
action.perform()
Alternatively to replace all the ActionChains commands, you can simply run execute_script like this:
driver.execute_script("document.querySelectorAll('label.boxed')[1].click()")
There you go, at least on my local file anyway... Hope this helps!
What worked for me was to find the element just before the problematic one (that is, just before it in terms of tab order), then call Tab on that element.
from selenium.webdriver.common.keys import Keys
elem = br.find_element_by_name("username")
elem.send_keys(Keys.TAB) # tab over to not-visible element
After doing that, I was able to send actions to the element.
The actual solution of this thread did not work for me.
however,
this one did :
element = WebDriverWait(driver, 3).until(EC.visibility_of_element_located((By.XPATH, xpaths['your_xpath_path'])))
the trick is to use :
EC.visibility_of_element_located
the WebDriverWait
WebDriverWait
from this import :
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
I suggest you use xpath with explicit wait
//input[contains(#id,'spsel')][#value='nba']
if "Element is not currently visible" then make it VISIBLE
f.e.
>>> before is hidden top is outside of page
<input type="file" style="position: absolute;top:-999999" name="file_u">
>>> after move top on in page area
DRIVER.execute_script("document.getElementByName('file_u').style.top = 0;")
time.sleep(1); # give some time to render
DRIVER.find_element_by_name("file_u").send_keys("/tmp/img.png")
Instead of get_element_by_id() you can try elem = browser.find_element_by_css_selector('#elemId') (go to that webpage and the element, right click it and Copy CSS Selector, or something like that.) This is what i did and it works. You also try find_element_by_link_text(text), find_element_by_partial_link_text(text), find_element_by_tag_name(tagName_case_insensitive_here), find_element_by_name(name) etc. Something will work. After the id the CSS Selector is your best bet.
I ended up using #twasbrillig's solution, but instead of finding the previous element and sending a TAB keypress, I find the desired element, send a TAB keypress with that element, and then a SHIFT + TAB keypress to the driver:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
el = driver.find_element_by_id("-spel-nba")
el.send_keys(Keys.TAB)
webdriver.ActionChains(driver).key_down(Keys.SHIFT).send_keys(Keys.TAB).key_up(Keys.SHIFT)
I tried using the other methods but in the end found that the simplest way was to just try and click the button, and catch the error. This allows me to perform other actions based on if it worked (True) or didn't (False).
def click_button(html_object):
try:
html_object.click()
except:
return False #most likely because it is NotVisible object and can be ignored
return True
...
...
click_button(actions)
The way I solved this in python was:
try:
# the element you want to scroll to
element = driver.find_element_by_id("some_id")
ActionChains(driver).move_to_element(element).perform()
element.send_keys(Keys.TAB).key_up(Keys.SHIFT)
#element.click()
except Exception as e:
print(e)
Related
I'm trying to write a script to automate some tasks with Selenium and Python, and every time I try to click on a button
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
driver = webdriver.Chrome(options=options)
xpath = "/html/body/app-root/app-prime/div/mat-sidenav-container//app-detail-component/main//div/span/button[#aria-label='Prenota']"
# Wait for the element to be visible, always true
WebDriverWait(driver, 5).until(expected_conditions.presence_of_element_located((By.XPATH, xpath)))
# Try to click on element, get an error
driver.find_element(By.XPATH, xpath).click()
I get the following error
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
I know for sure that the element gets located correctly and has to match, but it should act only on the first one.
I tried:
Trying to click child tags, such as others divs and span
Waiting for it to be clickable
Waiting with an implicit wait
None of those activities were successful
Edit: Apparently the issue olly exist on my machine
It could be not clickable for a number of reasons. You might want to check if there is some element on the page layered on top so that that element is not interactable at that time, e.g some popup/iframe etc. There could be some other element that will receive click at that time.
You could try an actions click - something like this
myElement = driver.find_element_by_xpath("myXpath")
webdriver.ActionChains(driver).move_to_element(myElement).click(myElement).perform()
One of this should work.
IJavaScriptExecutor executor = (IJavaScriptExecutor)WebDriver.Driver;
executor.ExecuteScript("arguments[0].click();", webElement);
Actions actions = new Actions(WebDriver.Driver);
actions.MoveToElement(webElement).Click().Perform();
Note - these are c sharp code. try to do the same in java.
I'm trying to make a small program that looks at a web page at a hidden button (uses hide in the class) and waits for it to be clickable before clicking it. The code is below. I'm wondering if the WebDriverWait and element_to_be_clickable functions will already by refreshing things or if I would have to manually refresh the page.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
from selenium.common.exceptions import WebDriverException
driver = webdriver.Firefox()
driver.get(<URL>)
print("beginning 120s wait")
time.sleep(120)
print("finished 120s wait")
try:
element = WebDriverWait(driver, 1000).until(
EC.element_to_be_clickable((By.CLASS_NAME, "btn add"))
)
print("It went through")
element.click()
driver.execute_script("alert('It went through!');")
finally:
driver.execute_script("alert('Did it work?');")
First of all, I am not really sure if just searching by the class name minus the "hide" part will actually find the correct element, but the larger issue is that I do not know if the button will only be visible after refreshing the page. If I need to refresh, then it gets annoying because most sites throw up additional captchas for both Firefox or Chrome when they figure out a bot is accessing the site. (That's why I have the initial sleep: so that I can finish any captcha manually first)
So, do I need to have a refresh in my code, or will it be fine without it? If I do need it, how do I implement it? Do I just add it like:
try:
element = WebDriverWait(driver, 1000).until(
drive.refresh()
EC.element_to_be_clickable((By.CLASS_NAME, "btn add"))
)
And sorry if this has been answered elsewhere, I searched a bunch, but I have not quite found the answer on this site.
First, you shouldn't use sleep the WebDriverWait with the correct EC will do the trick.
As for the EC.element_to_be_clickable this is the code behind the function:
def element_to_be_clickable(locator):
""" An Expectation for checking an element is visible and enabled such that
you can click it."""
def _predicate(driver):
element = visibility_of_element_located(locator)(driver)
if element and element.is_enabled():
return element
else:
return False
return _predicate
As you can see the EC.element_to_be_clickable function does not refresh the browser.
If you insist you need the refresh the correct way to implement it will be:
try:
element = WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.CLASS_NAME, "btn add"))
except (NoSuchElementException, StaleElementReferenceException):
driver.refresh()
element = WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.CLASS_NAME, "btn add"))
I don't think the refresh will help with the hidden element...
I am having a terribly hard time referencing to a certain "next page" button on a website that I am trying to scrape links from [https://www.sreality.cz/adresar?strana=2]. If you scroll down you can see a red right arrow button that you can click to go to the next page and so the website load new dynamic content. Every approach seems to report the same exact error and I don't know how am I supposed to point to the element without running into it.
This is the code that I currently have :
from selenium import webdriver
chromedriver_path = "/home/user/Dokumenty/iCloud/RealityScraper/chromedriver"
driver = webdriver.Chrome(chromedriver_path)
print("WebDriver Successfully Initialized")
driver.get("https://www.sreality.cz/adresar?strana=2")
links = driver.find_elements_by_css_selector("h2.title a")
nextPage = driver.find_element_by_css_selector("li.paging-item a.btn-paging-pn.icof.icon-arr-right.paging-next")
for link in links:
print(link.get_attribute("href"))
nextPage.click()
The "nextPage" variable is holding a supposed value to be clicked on once the "links" variable search finishes scraping all the links from the company titles. However when I run this code I get an error :
selenium.common.exceptions.StaleElementReferenceException: Message:
stale element reference: element is not attached to the page document
I have been searching for various fixes online but none of them seemed to resolve the issue. I think that the issue at this point is not caused by the element not loading quickly enough but rather Selenium having trouble finding the element because of wrong reference.
Because of this I have tried using XPath to accurately point to the actual element and so I changed the "nextPage" variable to :
nextPage = driver.find_element_by_xpath("""/html/body/div[2]/div[1]/div[2]/div[2]/div[4]/div/div/div/div[2]/div/div[2]/ul[1]/li[12]/a""")
Which returns exactly the same error as stated above. I have been trying to find a solution to this for hours now and I can't understand where the issue lies. I would be grateful if anyone could explain to me what am I doing wrong. Thanks to anyone.
If you want to get all the ng-href tags from every page. Or you could look into their api.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from time import sleep
driver.get("https://www.sreality.cz/adresar?strana=2")
wait = WebDriverWait(driver, 10)
while True:
try:
links = wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "h2.title > a")))
#print(len(links))
for link in links:
print(link.get_attribute("ng-href"))
nextPage = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.btn-paging-pn.icof.icon-arr-right.paging-next")))
nextPage.click()
time.sleep(10)
except Exception as e:
print(e)
break
First of all never use the absolute xpath it will breakdown easily, Use the relative xpath.
Secondly, i think the error you are getting is because after clicking "Next" button for the first time it loads a new page. Which has a different DOM structure and that's why you are not able to find that element.
You can try searching for the element after every new page load (after clicking "Next" button everytime.)
// imports
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver import ActionChains
from selenium.webdriver.common.by import By
// initialize
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 20)
action = ActionChains(driver)
// Try to use the below code and see if it works.
Next_btn = wait.until(EC.presence_of_element_located((By.XPATH, '(//li[#class="paging-item"])[2]')))
action.move_to_element(Next_btn).click().perform()
Trying got automate a task via Selenium python i have the issue where the for each section does work only the first time, after that does not see the second variable. Also tried to add delays so the webpage would be fully loaded but the same issue.
I tested different scenarios that i found in the internet also so manual tests i did, but looks like the second div is not recognizable also the rest of the divs
for server in browser.find_elements_by_xpath("//*[starts-with(#id,'server-list-')]"):
#try:
print("Server Section----")
time.sleep(5)
#Print server name
print(server.text)
#clicn on button inside the server
server.click()
#back into the server listing
browser.back()
Basically the automation need to enter every server ( div starting with id server-list- ) click on it, after entering on that section click another button and than back to the main page.
A stale element reference exception is thrown because of below reason
element has been deleted entirely.
element is no longer attached to the DOM.
Please check your element is still present on UI with the same element XPath which you are using while interacting with it
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
driver.get("http://somedomain/url_that_delays_loading")
try:
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.XPATH, "//*[starts-with(#id,'server-list-')]"))
)
for i in range(len(element)):
element[i].click()
driver.back()
element = wait.until(EC.presence_of_element_located((By.XPATH, "//*[starts-with(#id,'server-list-')]")))
finally:
driver.quit()
With limited information provided, you can try code below:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
browser = webdriver.Chrome()
wait = WebDriverWait(browser, 10)
servers = wait.until(EC.visibility_of_all_elements_located((By.XPATH, "//*[starts-with(#id,'server-list-')]")))
servers_count = len(servers)
for i in range(servers_count):
print(servers[i].text)
servers[i].click()
browser.back()
servers = wait.until(EC.visibility_of_all_elements_located((By.XPATH, "//*[starts-with(#id,'server-list-')]")))
You capture the server list that no more exists after you have drilled into the particular list item. So when you get back to the list that old one does not exist any more hence all the items (including that one you expect to move next to) are stale.
You need to rework your logic so that you get the list of servers each time you get back from the server details and store somethere the flag that would let your script know which items you have already visited.
How do I use Python to simply find a link containing text/id/etc and then click that element?
My imports:
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.keys import Keys
Code to go to my URL:
browser = webdriver.Firefox() # Get local session of firefox
browser.get(myURL) # Load page
#right here I want to look for element containing "Click Me" in the text
#and then click it
Look at method list of WebDriver class - http://selenium.googlecode.com/svn/trunk/docs/api/py/webdriver_remote/selenium.webdriver.remote.webdriver.html
For example, to find element by its ID and click it, the code will look like this
element = browser.find_element_by_id("your_element_id")
element.click()
Hi there are a couple of ways to find a link (or any element):
element = driver.find_element_by_id("element-id")
element = driver.find_element_by_name("element-name")
element = driver.find_element_by_xpath("//input[#id='element-id']")
element = driver.find_element_by_link_text("link-text")
element = driver.find_element_by_class_name("class-name")
I think the best option for you is find_element_by_link_text since it's a link.
Once you saved the element in a variable, you call the click function: element.click() or element.send_keys(Keys.RETURN)
Take a look to the selenium-python documentation, there are a couple of examples there.
You just use this code
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
driver = webdriver.Firefox()
driver.get("https://play.spotify.com/")# here change your link
wait=WebDriverWait(driver,250)
# it will wait for 250 seconds an element to come into view, you can change the #value
submit=wait.until(EC.presence_of_element_located((By.LINK_TEXT, 'Click Me')))
submit.click()
here wait is best fit for javascript enabled website let suppose if you load the page in webdriver but due to javascript some element loads after some second then it best suite to use wait element there.
for link containing text
browser = webdriver.firefox()
browser.find_element_by_link_text(link_text).click()
You can also do it by xpath:
Browser.find_element_by_xpath("//a[#href='you_link']").click()
Finding Element by Link Text
driver.find_element_by_link_text("")
Finding Element by Name
driver.find_element_by_name("")
Finding Element By ID
driver.find_element_by_id("")
Try SST - it is a simple yet very good test framework for python.
Install it first: http://testutils.org/sst/index.html
Then:
Imports:
from sst.actions import *
First define a variable - element - that you're after
element = assert_element(text="some words and more words")
I used this: http://testutils.org/sst/actions.html#assert-element
Then click that element:
click_element('element')
And that's it.
First start one instance of browser. Then you can use any of the following methods to get element or link. After locating the element you can use element.click() or element.send_keys(Keys.RETURN) or any other object of selenium webdriver
browser = webdriver.Firefox()
Selenium provides the following methods to locate elements in a page:
To find individual elements. These methods will return individual element.
browser.find_element_by_id(id)
browser.find_element_by_name(name)
browser.find_element_by_xpath(xpath)
browser.find_element_by_link_text(link_text)
browser.find_element_by_partial_link_text(partial_link_text)
browser.find_element_by_tag_name(tag_name)
browser.find_element_by_class_name(class_name)
browser.find_element_by_css_selector(css_selector)
To find multiple elements (these methods will return a list). Later you can iterate through the list or with elementlist[n] you can locate individual element from list.
browser.find_elements_by_name(name)
browser.find_elements_by_xpath(xpath)
browser.find_elements_by_link_text(link_text)
browser.find_elements_by_partial_link_text(partial_link_text)
browser.find_elements_by_tag_name(tag_name)
browser.find_elements_by_class_name(class_name)
browser.find_elements_by_css_selector(css_selector)
browser.find_element_by_xpath("")
browser.find_element_by_id("")
browser.find_element_by_name("")
browser.find_element_by_class_name("")
Inside the ("") you have to put the respective xpath, id, name, class_name, etc...