I am quite new to python selenium and I am trying to click on a button which has the following html structure:
<div class="b_div">
<div class="button c_button s_button" onclick="submitForm('mTF')">
<input class="very_small" type="button"></input>
<div class="s_image"></div>
<span>
Search
</span>
</div>
<div class="button c_button s_button" onclick="submitForm('rMTF')" style="margin-bottom: 30px;">
<input class="v_small" type="button"></input>
<span>
Reset
</span>
</div>
</div>
I would like to be able to click both the Search and Reset buttons above (obviously individually).
I have tried a couple of things, for example:
driver.find_element_by_css_selector('.button .c_button .s_button').click()
or,
driver.find_element_by_name('s_image').click()
or,
driver.find_element_by_class_name('s_image').click()
but, I seem to always end up with NoSuchElementException, for example:
selenium.common.exceptions.NoSuchElementException: Message: u'Unable to locate element: {"method":"name","selector":"s_image"}' ;
I am wondering if I can somehow use the onclick attributes of the HTML to make selenium click?
Any thoughts which can point me in the right direction would be great.
Thanks.
Remove space between classes in css selector:
driver.find_element_by_css_selector('.button .c_button .s_button').click()
# ^ ^
=>
driver.find_element_by_css_selector('.button.c_button.s_button').click()
try this:
download firefox, add the plugin "firebug" and "firepath"; after install them go to your webpage, start firebug and find the xpath of the element, it unique in the page so you can't make any mistake.
See picture:
browser.find_element_by_xpath('just copy and paste the Xpath').click()
For python, use the
from selenium.webdriver import ActionChains
and
ActionChains(browser).click(element).perform()
open a website https://adviserinfo.sec.gov/compilation and click on button to download the file and even i want to close the pop up if it comes using python selenium
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
from selenium.webdriver.chrome.options import Options
#For Mac - If you use windows change the chromedriver location
chrome_path = '/usr/local/bin/chromedriver'
driver = webdriver.Chrome(chrome_path)
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--disable-popup-blocking")
driver.maximize_window()
driver.get("https://adviserinfo.sec.gov/compilation")
# driver.get("https://adviserinfo.sec.gov/")
# tabName = driver.find_element_by_link_text("Investment Adviser Data")
# tabName.click()
time.sleep(3)
# report1 = driver.find_element_by_xpath("//div[#class='compilation-container ng-scope layout-column flex']//div[1]//div[1]//div[1]//div[2]//button[1]")
report1 = driver.find_element_by_xpath("//button[#analytics-label='IAPD - SEC Investment Adviser Report (GZIP)']")
# print(report1)
report1.click()
time.sleep(5)
driver.close()
I had the same problem using Phantomjs as browser, so I solved in the following way:
driver.find_element_by_css_selector('div.button.c_button.s_button').click()
Essentially I have added the name of the DIV tag into the quote.
The following debugging process helped me solve a similar issue.
with open("output_init.txt", "w") as text_file:
text_file.write(driver.page_source.encode('ascii','ignore'))
xpath1 = "the xpath of the link you want to click on"
destination_page_link = driver.find_element_by_xpath(xpath1)
destination_page_link.click()
with open("output_dest.txt", "w") as text_file:
text_file.write(driver.page_source.encode('ascii','ignore'))
You should then have two textfiles with the initial page you were on ('output_init.txt') and the page you were forwarded to after clicking the button ('output_dest.txt'). If they're the same, then yup, your code did not work. If they aren't, then your code worked, but you have another issue.
The issue for me seemed to be that the necessary javascript that transformed the content to produce my hook was not yet executed.
Your options as I see it:
Have the driver execute the javascript and then call your find
element code. Look for more detailed answers on this on
stackoverflow, as I didn't follow this approach.
Just find a comparable hook on the 'output_dest.txt' that will produce the same result, which is what I did.
Try waiting a bit before clicking anything:
xpath2 = "your xpath that you are going to click on"
WebDriverWait(driver, timeout=5).until(lambda x:
x.find_element_by_xpath(xpath2))
The xpath approach isn't necessarily better, I just prefer it, you can also use your selector approach.
I had the same problem and with Firefox, I got button element with the following steps:
right click button of interest and select "Inspect Accessibility Properties"
this opens the inspector. Right click the highlighted line and click "Print to JSON"
this opens a new tab. Look for nodeCssSelector and copy the value
This allowed me to accept cookies of the website Yahoo by using.
url = "https://yahoo.com"
driver = Firefox(executable_path="geckodriver.exe")
driver.get(url)
driver.find_element_by_css_selector("button.btn:nth-child(5)").click()
I tested this further and it allowed me to accept individual cookies with ease. Simply repeat the mentioned steps from before to get the button names.
url = "https://yahoo.com"
driver = Firefox(executable_path="geckodriver.exe")
driver.get(url)
driver.find_element_by_css_selector("a.btn").click()
driver.find_element_by_css_selector(".firstPartyAds > div:nth-child(2) > label:nth-child(1)").click()
driver.find_element_by_css_selector(".preciseGeolocation > div:nth-child(2) > label:nth-child(1)").click()
driver.find_element_by_css_selector("button.btn").click()
Another method is to
right click button of interest and select "Inspect"
right click the highlighted line and click "Copy -> CSS Selector" or whatever you need (there are multiple options, including XPath)
However, I think the second method may include whitespaces depending on what you copy, so you might need to manually remove (some of) them. The first method seems to be more foolproof, but I don't know if/how it works on other browsers than Firefox. The second method should work for all browsers.
Use This code To Click On Button
# finding the button using ID
button = driver.find_element_by_id(ID)
# clicking on the button
button.click()
e = driver.find_element(By.XPATH, 's_image').click()
sometime it does not work!
you can try:
e = driver.find_element(By.XPATH, 's_image') driver.execute_script("arguments[0].click();", e)
Related
I'm trying to access a homepage, make login, click the login button and click a button (in the second page) using python/selenium.
The login button I wrote using Xpath and it's working well. The browser opens, writes user and login and click the login button.
Unfortunately, in the next page I'm not able to click on the button that I need to. It didn't work using Xpath and I can't understand why. The html is completely different from the first button, the button name is 'Reservar' and it is inside a class named <app-menu__menu>, written as:
<a href="/Services" id="advanced" class="element ">
<span>Reservar</span>
</a>
The xpath I got and tried:
xpath = "//*[#id="advanced"]"
Then I tried a second verion (it was gotten as the second line code xpath):
xpath = "//*[#id="advanced"]/span"
When I first tried to used them, I got an error. Then I change the "" to ' ' and the error was gone. But the program can't locate the button.
I'm using google-chrome, ubuntu, python3 and selenium package:
driver.find_element_by_xpath(xpath).click()
Thanks for any help.
You probably missing a delay / wait.
After clicking the submit / login button on the login page it takes some time to make the internal page loaded.
The recommended way to do that is to use Expected Conditions explicit waits, something like this:
wait.until(EC.visibility_of_element_located((By.XPATH, "//*[#id='advanced']"))).click()
To use the wait object here you will need to import the following inports:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
And to initialize the wait object with
wait = WebDriverWait(driver, 20)
You will have to validate your locator is unique. As well as to validate the element you are trying to access is not inside iframe and not in a new window / tab etc.
I'm trying to use selenium to extract the text on a button but python is only returning None
I got the button using
button = browser.find_element_by_xpath("/html/body/div[3]/main/div[3]/div[3]/div/div/div/div/div/button")
text = button.get_attribute('text')
(Button works perfectly fine when using .click() )
This is the button code:
<button class=" btn btn-primary stepbuttonnew" onclick="if (!window.__cfRLUnblockHandlers) return false; ok.performClick();gtag('config', 'UA-113527404-1', {'page_path': '/smth'});">
Text i want
</button>
It should return "Text I want"
Any help is appreciated
:)
Can you post the page?
I would suggest to try :
text = driver.find_element_by_xpath('XPATH').text
print(text)
Maybe test find_element_by_class_name if xpath won't work.
The text property of your button element will return the text.
button = browser.find_element_by_xpath("/html/body/div[3]/main/div[3]/div[3]/div/div/div/div/div/button")
buttonText = button.text
If the usual selenium does not work then I believe that JavaScript executor will be able to help you to retrieve the desired value. If you do not have prior knowledge on it then I recommend you to check that out.
`using innerText atttribute? many times plain text reads null in Angular/React ui applicatios, you should be able to get it using innerText attribute i.e , text = button.get_attribute('innertext')
For scraping text from button in Python, you can coordinate ProxyCrawl API(https://pypi.org/project/proxycrawl/) that is sufficiently adaptable to blend with your current Python codebase. APIs are in effect nowadays and is utilized by information engineers these days as they give more proficient, safe, productive information scraping and are totally versatile as indicated by the dynamism of various pages. Also, we can fetch the button text by CSS Selector, XPath and Class name. Other than this, you can likewise utilize the accompanying code to scrape information utilizing Python Selenium.
Source Code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select
import time
driver = webdriver.Chrome()
wait = WebDriverWait(driver,30)
driver.get("https://www.decoin.io/en/")
driver.maximize_window()
text = wait.until(EC.visibility_of_element_located((By.XPATH,'/html/body/section[1]/div/div[2]/a[1]/button'))).text
print(text)
On this website I try to set some filters to collect data but I can't access to the table using a click event with selenium in my python script.
I noticed that I need to change the style from :
div id="filtersWrapper" class="displayNone " style="display: none;"
to
div id="filtersWrapper" class="displayNone " style="display: block;"
I think that I should use driver.execute_script(), but I have no clue how to do it
I would greatly appreciate some help with this. Thank you!
You can change an attribute on an element using javascript through selenium
element = driver.find_element_by_id('filtersWrapper')
driver.execute_script("arguments[0].setAttribute('attributeToChange', 'new value')", element)
or you can try clicking the element with javascript
driver.execute_script("arguments[0].click();", element)
I have checked the DOM Tree of the webpage. Somehow I was unable to locate any element as:
<div id="filtersWrapper" class="displayNone " style="display: none;">
However the following element exists:
<div id="filtersWrapper" class="displayNone ">
<div id="filtersArrowIndicator" class="arrowIndicator"></div>
.
<div id="economicCalendarSearchPopupResults" class="eventSearchPopupResults economicCalendarSearchPopupResults text_align_lang_base_1 dirSearchResults calendarFilterSearchResults displayNone">
</div>
</div>
Not sure if that was your desired element. A bit of more information about your usecase would have helped us to debug the issue in a better way. However, To set the display property of style attribute as block for the element you can use:
driver.execute_script("document.getElementById('filtersWrapper').style.display='block';");
You can use driver.execute_script() to accomplish this. This is how I change the style attribute in my own code:
div_to_change = driver.find_element_by_id("filtersWrapper")
driver.execute_script("arguments[0].style.display = 'block';", div_to_change)
I had a look at the website you are automating, and you might not need to use JSE at all to do this - there's a reason the div you are trying to click has style = "display: none" - it is not meant to be clicked in this context. Working around that with Javascript might not produce your intended results. This code snippet has been updated with your requirements to set a Time filter in the Economic Calendar section:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver.get("https://www.investing.com/economic-calendar/")
driver.find_element_by_id("economicCurrentTime").click()
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.ID, "filterStateAnchor"))).click()
checkbox_for_bull3 = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//*[#id='importance2']")))
driver.execute_script("arguments[0].scrollIntoView(true);", checkbox_for_bull3)
checkbox_for_bull3.click()
checkbox_for_time = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//fieldset[label[#for='timeFiltertimeOnly']]/input")))
checkbox_for_time.click()
I modified your code snippet to fix a few issues -- when navigating to the economic-calendar page, you were clicking the 'Filters' field twice which caused an issue trying to click checkbox_for_bull3. I also added a scrollIntoView() Javascript call.
I ran this on my local machine and the code executed end to end successfully.
I am currently working on a project which fills a form automatically. And the next button appears when the form is filled, that's why it gives me an error.
I have tried:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH,"//input[#type='button' and #class='button']")))
Next = driver.find_element_by_xpath("//input[#type='button' and #class='button']")
Next.click()
HTML:
<span class="btn">
<input type="button" value="Next" class="button" payoneer="Button" data-controltovalidate="PersonalDetails" data-onfieldsvalidation="ToggleNextButton" data-onclick="UpdateServerWithCurrentSection();" id="PersonalDetailsButton">
</input>
<div class="clearfix"></div>
</span>
ERROR:
selenium.common.exceptions.ElementClickInterceptedException: Message:
element click intercepted: Element is not clickable at point (203, 530).
Other element would receive the click: ... (Session info: chrome=76.0.3809.132)
If the path of the xpath is right, maybe you can try this method to solve this problem. Replace the old code with the following code:
button = driver.find_element_by_xpath("xpath")
driver.execute_script("arguments[0].click();", button)
I solved this problem before, but to be honestly, I don't know the reason.
This error message...
selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element is not clickable at point (203, 530). Other element would receive the click: ... (Session info: chrome=76.0.3809.132)
...implies that the click() on the desired element was intercepted by some other element and the desired element wasn't clickable.
There are a couple of things which you need to consider as follows:
While using Selenium for automation using time.sleep(secs) without any specific condition to achieve defeats the purpose of automation and should be avoided at any cost. As per the documentation:
time.sleep(secs) suspends the execution of the current thread for the given number of seconds. The argument may be a floating point number to indicate a more precise sleep time. The actual suspension time may be less than that requested because any caught signal will terminate the sleep() following execution of that signal’s catching routine. Also, the suspension time may be longer than requested by an arbitrary amount because of the scheduling of other activity in the system.
You can find a detailed discussion in How to sleep webdriver in python for milliseconds
As WebDriverWait returns the WebElement you can invoke the click() method directly.
Solution
To click on the button with value as Next you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.button#PersonalDetailsButton[data-controltovalidate='PersonalDetails']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#class='button' and #id='PersonalDetailsButton'][#data-controltovalidate='PersonalDetails']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
i faced similar issues, the .click() always returns a Not clickable exception. the
driver.execute_script('arguments[0].click()', button)
does the magic. You can also use it to execute any other js script this way
script = 'your JavaScript goes here'
element = driver.find_element_by_*('your element identifier goes here')
driver.execute_script(script, element)
I looked at the exact element that was causing it and it was a banner about consent/cookies. So at first, I made sure it clicked "OK" on the consent banner and then I clicked the other button that I needed. Hope it helps someone.
It look's like there are some other elements which are having the same xpath try changing the xpath something like this
Next = driver.find_element_by_xpath("//input[#id='PersonalDetailsButton']");
Next.Click();
or
Next = driver.find_element_by_xpath(//input[#value='Next' and #id='PersonalDetailsButton']);
Next.Click();
Try first xpath if that doesn't work go with the second one . If that also doesn't work try using sikuli. I am pretty sure that first xpath will work
I faced a similar issue and I observed something that might help to understand the root cause of the issue. In my case, I was able to click at an element being in PC view mode of the website but failed to do so in mobile view (in which I needed my script to run). I found out that in mobile view, ordering of elements (li in my case) changed in view while they remained same in the html document. That's why I was not able to click on it without actually scrolling to it first. It might also explain why this works: -
driver.execute_script("arguments[0].click();", button)
I don't have enough rep to comment but the common reason for this error might be Selenium locates the element from DOM on screen and locate the x-y coordinates (300, 650) then clicks on them but if some changes takes place on screen in between the click duration, for example google ads or some pop-up then it's unable to click on it resulting in this exception
I'm just guessing if anyone has a proper explanation to pls share
I had the same problem too. But my problem was not with the element. The button was activated with href. I changed the code from
<a class="services-button" href="desired url">
To
<a class="services-button" onclick="location.href='{% url "desired url" %}'";">
This solution worked for me :
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
driver = webdriver.Firefox(executable_path="")
driver.get("https://UrlToOpen")
action = ActionChains(driver)
firstLevelMenu = driver.find_element_by_id("menu")
firstLevelMenu.click()
source : http://allselenium.info/mouse-over-actions-using-python-selenium-webdriver/
"selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element is not clickable ... "
This exception occurs when element is not found on a web page (When the element we are looking for is at bottom part of the page which is not loaded yet)
So we you can scroll the page using javascript and load complete page and
from selenium.webdriver.common.by import By
from selenium import webdriver
url = "YOUR URL"
SCROLL_PAUSE_TIME = 0.5
driver = webdriver.Chrome()
driver.maximize_window()
driver.get(url)
def scroll_page():
i = 0
while i < 5:
# Scroll down to 500 pixel
driver.execute_script("window.scrollBy(0, 500)", "")
# Wait to load page
time.sleep(SCROLL_PAUSE_TIME)
# Will scroll only for 4 increments of 500px
i += 1
You could try:
driver.execute_script("arguments[0].click();", button)
This solution solved my problems when I faced similar issues.
I have been tasked with writing a parser to click a button on a website and I am having issues to click only one of the buttons. The following code works on every button except one.
Here's the html:
http://pastebin.com/6dLF5ru8
here's the source html:
http://pastebin.com/XhsedGLb
python code:
driver = webdriver.Firefox()
...
el = driver.find_element_by_id("-spel-nba")
actions.move_to_element(el)
actions.sleep(.1)
actions.click()
actions.perform()
I am getting this error.
ElementNotVisibleException: Message: Element is not currently visible and so may not be interacted with
as per Saifur I just tried waits with the same element not visible exception:
wait = WebDriverWait(driver, 10)
wait.until(EC.presence_of_element_located((By.XPATH, "//input[contains(#id,'spsel')][#value='nba']"))).click()
If you look at the page source, you'll understand that almost all of theSELECT, DIV elements are faked and created from JavaScript, that is why webdriver cannot SEE them.
There's a workaround though, by using ActionChains to open your developer console, and inject an artificial CLICK on the desired element, which in fact, is the Label triggering the NBA data loading... here's a working example:
from selenium import webdriver
from selenium.webdriver.common import action_chains, keys
import time
driver = webdriver.Firefox()
driver.get('Your URL here...')
assert 'NBA' in driver.page_source
action = action_chains.ActionChains(driver)
# open up the developer console, mine on MAC, yours may be diff key combo
action.send_keys(keys.Keys.COMMAND+keys.Keys.ALT+'i')
action.perform()
time.sleep(3)
# this below ENTER is to rid of the above "i"
action.send_keys(keys.Keys.ENTER)
# inject the JavaScript...
action.send_keys("document.querySelectorAll('label.boxed')[1].click()"+keys.Keys.ENTER)
action.perform()
Alternatively to replace all the ActionChains commands, you can simply run execute_script like this:
driver.execute_script("document.querySelectorAll('label.boxed')[1].click()")
There you go, at least on my local file anyway... Hope this helps!
What worked for me was to find the element just before the problematic one (that is, just before it in terms of tab order), then call Tab on that element.
from selenium.webdriver.common.keys import Keys
elem = br.find_element_by_name("username")
elem.send_keys(Keys.TAB) # tab over to not-visible element
After doing that, I was able to send actions to the element.
The actual solution of this thread did not work for me.
however,
this one did :
element = WebDriverWait(driver, 3).until(EC.visibility_of_element_located((By.XPATH, xpaths['your_xpath_path'])))
the trick is to use :
EC.visibility_of_element_located
the WebDriverWait
WebDriverWait
from this import :
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
I suggest you use xpath with explicit wait
//input[contains(#id,'spsel')][#value='nba']
if "Element is not currently visible" then make it VISIBLE
f.e.
>>> before is hidden top is outside of page
<input type="file" style="position: absolute;top:-999999" name="file_u">
>>> after move top on in page area
DRIVER.execute_script("document.getElementByName('file_u').style.top = 0;")
time.sleep(1); # give some time to render
DRIVER.find_element_by_name("file_u").send_keys("/tmp/img.png")
Instead of get_element_by_id() you can try elem = browser.find_element_by_css_selector('#elemId') (go to that webpage and the element, right click it and Copy CSS Selector, or something like that.) This is what i did and it works. You also try find_element_by_link_text(text), find_element_by_partial_link_text(text), find_element_by_tag_name(tagName_case_insensitive_here), find_element_by_name(name) etc. Something will work. After the id the CSS Selector is your best bet.
I ended up using #twasbrillig's solution, but instead of finding the previous element and sending a TAB keypress, I find the desired element, send a TAB keypress with that element, and then a SHIFT + TAB keypress to the driver:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
el = driver.find_element_by_id("-spel-nba")
el.send_keys(Keys.TAB)
webdriver.ActionChains(driver).key_down(Keys.SHIFT).send_keys(Keys.TAB).key_up(Keys.SHIFT)
I tried using the other methods but in the end found that the simplest way was to just try and click the button, and catch the error. This allows me to perform other actions based on if it worked (True) or didn't (False).
def click_button(html_object):
try:
html_object.click()
except:
return False #most likely because it is NotVisible object and can be ignored
return True
...
...
click_button(actions)
The way I solved this in python was:
try:
# the element you want to scroll to
element = driver.find_element_by_id("some_id")
ActionChains(driver).move_to_element(element).perform()
element.send_keys(Keys.TAB).key_up(Keys.SHIFT)
#element.click()
except Exception as e:
print(e)