I am trying to understand Python in general as I just switched over from using VBA. I interested in the possible ways you could approach this single issue. I already went around it by just going to the link directly, but I need to understand and apply here.
from selenium import webdriver
chromedriver = r'C:\Users\dd\Desktop\chromedriver.exe'
browser = webdriver.Chrome(chromedriver)
url = 'https://www.fake.com/'
browser.get(url)
browser.find_element_by_id('txtLoginUserName').send_keys("Hello")
browser.find_element_by_id('txtLoginPassword').send_keys("There")
browser.find_element_by_id('btnLogin').click()
At this point, I am trying to navigate to a particular button/link.
Here is the info from the page/element
T-Mobile
Here are some of the things I tried:
for elem in browser.find_elements_by_xpath("//*[contains(text(), 'T-Mobile')]"):
elem.click
browser.execute_script("InitiateCallBack(187, True, T-Mobile, https://www.fake.com/, TMobile)")
I also attempted to look for tags and use css selector all of which I deleted out of frustration!
Specific questions
How do I utilize the innertext,"T-Mobile", to click the button?
How would I execute the onclick event?
I've tried to read the following links, but still have not succeeded incoming up with a different way. Part of it is probably because I don't understand the specific syntax yet. This is just some of the things I looked at. I spent about 3 hours trying various things before I came here!
selenium python onclick() gives StaleElementReferenceException
http://selenium-python.readthedocs.io/locating-elements.html
Python: Selenium to simulate onclick
https://stackoverflow.com/questions/43531654/simulate-a-onclick-with-selenium-https://stackoverflow.com/questions/45360707/python-selenium-using-onclick
Running javascript in Selenium using Python
How do I utilize the innertext,"T-Mobile", to click the button?
find_elements_by_link_text would be appropriate for this case.
elements = driver.find_elements_by_link_text('T-Mobile')
for elem in elements:
elem.click()
There's also a by_partial_link_text locator as well if you don't have the full exact text.
How would I execute the onclick event?
The simplest way would be to simply call .click() on the element as shown above and the event should, naturally, execute at that time.
Alternatively, you can retrieve the onclick attribute and use driver.execute_script to run the js.
for elem in elements:
script = elem.get_attribute('onlcick')
driver.execute_script(script)
Edit:
note that in your code you did element.click -- this does nothing. element.click() (note the parens) calls the click method.
is there a way to utilize browser.execute_script() for the onclick event
execute_script can fire the equivalent event, but there may be more listeners that you miss by doing this. Using the element click method is the most sound. There may very well be many implementation details of the site that may hinder your automation efforts, but those possibilities are endless. Without seeing the actual context, it's hard to say.
You can use JS methods to click an element or otherwise interact with the page, but you may miss certain event listeners that occur when using the site 'normally'; you want to emulate, more or less, the normal use as closely as possible.
As per the HTML you have shared it's pretty clear the website uses JavaScript. So to click() on the link with text as T-Mobile you have to induce WebDriverWait with expected_conditions clause as element_to_be_clickable and your can use the following code block :
WebDriverWait(driver, 20).until(expected_conditions.element_to_be_clickable((By.XPATH, "//a[contains(.,'T-Mobile')]"))).click()
you can use it
<div class="button c_button s_button" onclick="submitForm('rMTF')" style="margin-bottom: 30px;">
<input class="v_small" type="button"></input>
<span>
Reset
</span>
Related
I am pretty new to web-scraping...
For example here is the part of my code:
labels = driver.find_elements(By.CLASS_NAME, 'form__item-checkbox-label.placeFinder-search__checkbox-label')
checkboxes = driver. find_elements(By.CLASS_NAME, 'form__item-checkbox-input.placeFinder-search__checkbox-input')
boxes = zip(labels,checkboxes)
time.sleep(3)
for label,checkbox in boxes:
if checkbox.is_selected():
label.click()
Here is another example:
driver.get(product_link)
time.sleep(3)
button = driver.find_element(By.XPATH, '//*[#id="tab-panel__tab--product-pos-search"]/h2')
time.sleep(3)
button.click()
And I am scraping through let's say hundreds of products. 90% of the time it works fina, but occasionally giver errors like couldn't locate the element or something is not clickable etc. But all these products pages are built the same. Moreover, if I just re-run code on the product that resulted in the error, mosr of the time from the 2nd or 3rd time I will be able to scrape the data and will not get the error back.
Why does it happen? Code stays the same, web page stays the same.. What is causing an error when it happens? The only thing that comes to my mind the Internet connection sometimes gets behind the code and the program is unable to see the elenebts it is looking for... But as you can see I have added time.sleep() but it does not always help...
How can this be avoided? It is really annoying to be forced to stay in front of the monitor all the day just to supervise and re-run the code.... I mean I guess I could just add the scrape fubction inside the try: except: else: block but I am still wondering why does the same code will sometimes work and sometimes return the error on the same page?
In short Selenium deals with three distinct states of a WebElement.
presence
visibile
interactable / clickable
Ideally, to click on any clickable element you need to induce WebDriverWait for the element_to_be_clickable() as follows:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//*[#id="tab-panel__tab--product-pos-search"]/h2"))).click()
Similarly you can also create a list of desired elements waiting for their visibility and click on them one by one waiting for each of them to be clickable as follows:
checkboxes = WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.CLASS_NAME, "form__item-checkbox-input.placeFinder-search__checkbox-input")))
for checkbox in checkboxes:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((checkbox))).click()
Welcome to the "dirty" side of Web automation. We called it "Flaky" tests. In other word they are "fragile". And the major disadvantage of Selenium Webdriver.
There could be several reasons of flaky situation:
Network instability: Since all commands sent over network: client ->
(selenium grid: in case need) -> browser driver -> actual browser. Any connection issue may cause reason to failed.
CSS animations: Since it executes commands directly, if you have some animative transitions, it may cause to fail
Ajax similar requests or dynamic element changing. If you have such "load more" or displaying after some actions, It may not dedect or still overlapping
And, last comment is sleep is not good idea to use, actually it is againts to best practices. Instead of, use Expected Conditions to ensure elements are visible and ready
Apologies if this question was answered before, I want to click on an area in a browser with plain text using Selenium Webdriver in python
The code I'm using is:
element_plainText = driver.find_elements(By.XPATH, '//*[contains(#class, "WgFkxc")]')
element_plainText.click()
However this is returning "ElementNotInteractableException". Can anyone help me out with this?
Selenium is trying to be helpful here, by telling you why it won't click on the element; ElementNotInteractableException means it thinks that what you're trying to click on isn't clickable.
This usually happens because either:
The element isn't actually visible, or is disabled
Another element is "overlapping" the element, possibly invisibly
You're clicking something Selenium thinks won't do anything, like plain text
There's two things I'd try to get around this. Firstly, Actions. Selenium has an Action API you can use to cause specific UI events to occur. I'd suggest finding the co-ordinates of the text, then making Selenium click those co-ordinates instead of telling it to click the element. Read more about that API here.
Secondly, try clicking it with Javascript, using a Javascript Executor. That can often give you the same outcome as using Selenium directly, without it being so "helpful".
I am using Selenium WebDriver to do automated testing of a website. I have been successful in clicking through numerous menus and links to a point.
At one point the website I am working with generates links that look like this:
<U onclick="HourglassSubmitItem(document.all('PageName').value, '00000001Responsibility Code')">Responsibility Code</U>
I am trying to use the .click functionality of the webdriver to click this link with no success.
Using this:
page.find_element_by_xpath("//u[contains(text(),'Responsibility Code')]")
successfully finds the U tag above. but when I add .click() to the end of this xpath, the click is not performed. But it also does not generate an error. So, my question is can Selenium be used to simulate clicks on an HTML tag that is NOT an anchor () tag? If so, how?
I will also say that I do not have control over the page I am working with, so changing the to is not possible.
I would appreciate any guidance the Community could provide.
Thank You for you help,
Chris
Sometimes using JavaScript could solve the "clicking" issue:
element = page.find_element_by_xpath("//u[contains(text(),'Responsibility Code')]")
page.execute_script('arguments[0].click();', element)
You can prefer JavaScript in this case.
WebElment element = page.find_element_by_xpath("//u[contains(text(),'Responsibility Code')]")
JavaScriptExecutor executor = (JavaScriptExecutor)driver;
executor.ExecuteScript("arguments[0].click();", element);
I am trying to click a button to show more comments on a discusson thread on http://disqus.com/
The HTML looks like following:
<div class="load-more" data-role="more" style="">
Show more
</div>
I have tried using the xPath from the button like following:
driver = webdriver.Chrome("/usr/lib/chromium-browser/chromedriver")
driver.get(url)
driver.find_element_by_xpath('''//*[#id="posts"]/div[3]/a''').click()
But I get the NoSuchElement exception.
What is the proper way of clicking a button with that type of HTML?
UPDATE: Worked when I switched to a specific iFrame:
driver.switch_to.frame('myID')
Then loaded by class name:
element = driver.find_element_by_class_name('load-more')
element.click()
NOTE: That the click() did not work when it was performed on the same line like driver.find_element_by_class_name('load-more').click()
Xpath should be a last resort for finding elements. For your case, I would try
driver.find_element_by_class_name("load-more__button"));
Using btn load-more__button won't work as compound classNames aren't valid.
If that still doesn't work, Id recommend using css selectors before Xpath.
However, if none of these options work, and your Xpath is still not working, you should use the firebug add on for Firefox to ensure the Xpath you are using is correct.
Here's some of my other answers that may help you get started with Firebug:
https://stackoverflow.com/a/38980488/3537915
https://stackoverflow.com/a/38723782/3537915
https://stackoverflow.com/a/38744577/3537915
I am trying to click this radio button using selenium and python.
<input type="radio" name="tweet_target_1" value="website" class="tweet-website-button radio-selection-validate serialize-me newline-before field-order-15">
I have
website = driver.find_element(name="tweet_target_1")
website.click()
but it's not allowing me to click it. How can I click using a combo of name, value or class, value etc.?
Is there a good source of info about how to use selenium? Because most of what I've found is on java and I'm using python.
EDIT: using XPATH
I tried
website = driver.find_elements(By.XPATH, "//form[#id='dmca_form' and #class='twitter-form custom-form']/div[20][#class='list-container']/div[1][#class='list-item']/div[7][#class='clearfix inf-tweet init-hide']/div[#class='input']/ul[#class='options']/li[2]/label/input[#class='tweet-website-button radio-selection-validate serialize-me newline-before field-order-15']/")
website.click()
I keep getting
AttributeError: 'list' object has no attribute 'click'
I know this comes a little too late perhaps, but I joined just recently.
Tip: Use, Firebug and with it Firepath. Locate the radio button and find out the xpath for the element in question.
website = driver.find_element_by_xpath(".//**")
website.click()
OR
website = driver.find_element_by_xpath(".//**").click()
This should work all the time you try. Also, just using from selenium import webdriver
should make the click() function work correctly.
I'm not sure where you found the documentation that said you could call find_element like that, but you should either be doing driver.find_element_by_name("tweet_target_1") or driver.find_element(By.NAME, "tweet_target_1") (having first imported By of course). Also, Selenium Java code is pretty easily convertible to Python code; it follows a few pretty simple transformation rules, and if you still have questions, all the code for the library itself will also be on your machine to look at.