I'm trying to use selenium to extract the text on a button but python is only returning None
I got the button using
button = browser.find_element_by_xpath("/html/body/div[3]/main/div[3]/div[3]/div/div/div/div/div/button")
text = button.get_attribute('text')
(Button works perfectly fine when using .click() )
This is the button code:
<button class=" btn btn-primary stepbuttonnew" onclick="if (!window.__cfRLUnblockHandlers) return false; ok.performClick();gtag('config', 'UA-113527404-1', {'page_path': '/smth'});">
Text i want
</button>
It should return "Text I want"
Any help is appreciated
:)
Can you post the page?
I would suggest to try :
text = driver.find_element_by_xpath('XPATH').text
print(text)
Maybe test find_element_by_class_name if xpath won't work.
The text property of your button element will return the text.
button = browser.find_element_by_xpath("/html/body/div[3]/main/div[3]/div[3]/div/div/div/div/div/button")
buttonText = button.text
If the usual selenium does not work then I believe that JavaScript executor will be able to help you to retrieve the desired value. If you do not have prior knowledge on it then I recommend you to check that out.
`using innerText atttribute? many times plain text reads null in Angular/React ui applicatios, you should be able to get it using innerText attribute i.e , text = button.get_attribute('innertext')
For scraping text from button in Python, you can coordinate ProxyCrawl API(https://pypi.org/project/proxycrawl/) that is sufficiently adaptable to blend with your current Python codebase. APIs are in effect nowadays and is utilized by information engineers these days as they give more proficient, safe, productive information scraping and are totally versatile as indicated by the dynamism of various pages. Also, we can fetch the button text by CSS Selector, XPath and Class name. Other than this, you can likewise utilize the accompanying code to scrape information utilizing Python Selenium.
Source Code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select
import time
driver = webdriver.Chrome()
wait = WebDriverWait(driver,30)
driver.get("https://www.decoin.io/en/")
driver.maximize_window()
text = wait.until(EC.visibility_of_element_located((By.XPATH,'/html/body/section[1]/div/div[2]/a[1]/button'))).text
print(text)
Related
I need help to automate to click the search button in this webpage. The code works so far, until I reach the search button.
Below are the elements for this button. The value named Search is unique for this button.
<input type="button" value="Search" onclick="submitfilter();">
Below is the code:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait, Select
from selenium.common.exceptions import TimeoutException, NoSuchElementException
from selenium.webdriver.common.by import By
driver = webdriver.Chrome(executable_path='C:/chromedriver.exe')
driver.implicitly_wait(10)
url = "http://fake.com"
driver.get(url)
driver.maximize_window()
ABC = driver.find_element(By.XPATH("//input[#value="Search"]"))
ABC.click()
There is a syntactical error on that line, you're using double quotes for the Xpath and so for the value inside it, which makes the code treat search as a variable.
Change the line to:
ABC = driver.find_element(By.XPATH('//input[#value="Search"]'))
ABC = driver.find_element(By.XPATH,"//input[#value='Search']")
The proper way to write this would be like so. Not calling the By.xpath with the string of the xpat.
I am new to selenium and I try to navigate on a page and to click on a button to get to the next page with selenium web driver.
This is my python code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
WebDriverWait(driver, 30).until(EC.presence_of_element_located((By.XPATH, '//*[#id="page"]/div/div/div[5]/div/div/div/div[3]/div/span[2]')))
driver.find_element(By.XPATH, '//*[#id="page"]/div/div/div[5]/div/div/div/div[3]/div/span[2]').send_keys(Keys.RETURN)
print('Element found')
In addition to send_keys(Keys.RETURN) I also tried send_keys(Keys.ENTER) and click(). In all cases the statement "Element found" is printed to the console, but nothing happens on the webpage.
Here is the HTML bit of the button:
<span class="ACME-src-ACME-ui-Pagination--arrowRight">
<i class="flaticon-right-arrow"></I>
</span>
Every help is highly appreciated.
Wait!! Looks like The Xpath you are using is incorrect. Spans are not clickable objects. Or like they are clickable but do not do anything, which is exactly happening here. May be the "i" tag is something which should take click(However I cannot see any object in Dom taking click and performing a job)
First provide me with the proper DOM.
You can further try two more ways of doing it:-
button = find_element(By.XPATH, '//*[#id="page"]/div/div/div[5]/div/div/div/div[3]/div/span[2]/i')
driver.execute_script("arguments[0].click();", button)
OR
button = find_element(By.XPATH, '//*[#id="page"]/div/div/div[5]/div/div/div/div[3]/div/span[2]/i')
ActionChains(driver).move_to_element(button).click().perform()
<a href="/?redirectURL=%2F&returnURL=https%3A%2F%2Fpartner.yanolja.com%2Fauth%2Flogin&serviceType=PC" aria-current="page" class="v-btn--active v-btn v-btn--block v-btn--depressed v-btn--router theme--light v-size--large primary" data-v-db891762="" data-v-3d48c340=""><span class="v-btn__content">
login
</span></a>
I want to click this href button using python-selenium.
First, I tried to using find_element_by_xpath(). But, I saw a error message that is no such element: Unable to locate element. Then, I used link_text, partial_link_text, and so on.
I think that message said that 'It can't find login-button' right?
How can I specify the login button?
It is my first time to using selenium, and My html knowledge is also not enough.
What should I study first?
+)
url : https://account.yanolja.bz/
I want to login and get login session this URL.
from urllib.request import urlopen
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
html = urlopen('https://account.yanolja.bz/')
id = 'ID'
password = 'PW'
#options = webdriver.ChromeOptions()
#options.add_argument("headless")
#options.add_argument("window-size=1920x1080")
#options.add_argument("--log-level=3")
driver = webdriver.Chrome("c:\\chromedriver")
driver.implicitly_wait(3)
driver.get('https://account.yanolja.bz/')
print("Title: %s\nLogin URL: %s" %(driver.title, driver.current_url))
id_elem = driver.find_element_by_name("id")
id_elem.clear()
id_elem.send_keys(id)
driver.find_element_by_name("password").clear()
driver.find_element_by_name("password").send_keys(password)
Most probably this thing is not working due to span in your a tag. I have tried to give some examples but I am not sure, if you are supposed to click the a tag or span. I have tried to click the span in all of them.I hope it does work. Only if you could give us what you have tried, it would be a great help to find out mistakes, if any.
Have you tried to find the element using class. Your link is named by so many classes, Is none of them unique?
driver.find_element(By.CLASS_NAME, "{class-name}").click()
using x-path:
driver.find_element_by_xpath("//span[#class='v-btn__content']").click()
driver.find_element_by_xpath("//a[#class='{class-name}']/span[#class='v-btn__content']").click()
if this xpath is not unique then you can use css selector
driver.find_element_by_css_selector("a[aria-current='page']>span.v-btn__content").click()
Firstly, I want to note that span class="v-btn__content">login</span> is not clickable. Thus, it raises an error.
Try to use this instead
driver.find_element_by_xpath('//a[#href="'+url+'"]')
Replace url with the url given in <a href='url'>
I am trying to scrape a webpage that opens up hyperlinks via javascript as shown below. I am using Selenium with Python.
<a href="javascript:openlink('120000020846')">
<subtitle>Blah blah blah</subtitle>
</a>
Using XPATH, I was able to open up the hyperlink using the following Python code.
driver = webdriver.Chrome()
xpath = '//a/subtitle[contains(text(),"Blah blah blah")]'
link_to_open = driver.find_element(By.XPATH,xpath);
link_to_open.click()
However, the link opens in the same tab. This is not what I want, as I want the links to open in a new tab, so that I can retain the information of the current page and continue processing the rest of the links.
Would greatly appreciate if anyone can give me some pointers if this can be done?
Thank you so much! :)
You can force a link to open in a new tab through the ActionChains implementation as follows :
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
action = ActionChains(driver)
link_to_open = driver.find_element(By.XPATH, "//a/subtitle[contains(.,'Blah blah blah')]")
action.key_down(Keys.CONTROL).click(link_to_open).key_up(Keys.CONTROL).perform()
I am quite new to python selenium and I am trying to click on a button which has the following html structure:
<div class="b_div">
<div class="button c_button s_button" onclick="submitForm('mTF')">
<input class="very_small" type="button"></input>
<div class="s_image"></div>
<span>
Search
</span>
</div>
<div class="button c_button s_button" onclick="submitForm('rMTF')" style="margin-bottom: 30px;">
<input class="v_small" type="button"></input>
<span>
Reset
</span>
</div>
</div>
I would like to be able to click both the Search and Reset buttons above (obviously individually).
I have tried a couple of things, for example:
driver.find_element_by_css_selector('.button .c_button .s_button').click()
or,
driver.find_element_by_name('s_image').click()
or,
driver.find_element_by_class_name('s_image').click()
but, I seem to always end up with NoSuchElementException, for example:
selenium.common.exceptions.NoSuchElementException: Message: u'Unable to locate element: {"method":"name","selector":"s_image"}' ;
I am wondering if I can somehow use the onclick attributes of the HTML to make selenium click?
Any thoughts which can point me in the right direction would be great.
Thanks.
Remove space between classes in css selector:
driver.find_element_by_css_selector('.button .c_button .s_button').click()
# ^ ^
=>
driver.find_element_by_css_selector('.button.c_button.s_button').click()
try this:
download firefox, add the plugin "firebug" and "firepath"; after install them go to your webpage, start firebug and find the xpath of the element, it unique in the page so you can't make any mistake.
See picture:
browser.find_element_by_xpath('just copy and paste the Xpath').click()
For python, use the
from selenium.webdriver import ActionChains
and
ActionChains(browser).click(element).perform()
open a website https://adviserinfo.sec.gov/compilation and click on button to download the file and even i want to close the pop up if it comes using python selenium
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
from selenium.webdriver.chrome.options import Options
#For Mac - If you use windows change the chromedriver location
chrome_path = '/usr/local/bin/chromedriver'
driver = webdriver.Chrome(chrome_path)
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--disable-popup-blocking")
driver.maximize_window()
driver.get("https://adviserinfo.sec.gov/compilation")
# driver.get("https://adviserinfo.sec.gov/")
# tabName = driver.find_element_by_link_text("Investment Adviser Data")
# tabName.click()
time.sleep(3)
# report1 = driver.find_element_by_xpath("//div[#class='compilation-container ng-scope layout-column flex']//div[1]//div[1]//div[1]//div[2]//button[1]")
report1 = driver.find_element_by_xpath("//button[#analytics-label='IAPD - SEC Investment Adviser Report (GZIP)']")
# print(report1)
report1.click()
time.sleep(5)
driver.close()
I had the same problem using Phantomjs as browser, so I solved in the following way:
driver.find_element_by_css_selector('div.button.c_button.s_button').click()
Essentially I have added the name of the DIV tag into the quote.
The following debugging process helped me solve a similar issue.
with open("output_init.txt", "w") as text_file:
text_file.write(driver.page_source.encode('ascii','ignore'))
xpath1 = "the xpath of the link you want to click on"
destination_page_link = driver.find_element_by_xpath(xpath1)
destination_page_link.click()
with open("output_dest.txt", "w") as text_file:
text_file.write(driver.page_source.encode('ascii','ignore'))
You should then have two textfiles with the initial page you were on ('output_init.txt') and the page you were forwarded to after clicking the button ('output_dest.txt'). If they're the same, then yup, your code did not work. If they aren't, then your code worked, but you have another issue.
The issue for me seemed to be that the necessary javascript that transformed the content to produce my hook was not yet executed.
Your options as I see it:
Have the driver execute the javascript and then call your find
element code. Look for more detailed answers on this on
stackoverflow, as I didn't follow this approach.
Just find a comparable hook on the 'output_dest.txt' that will produce the same result, which is what I did.
Try waiting a bit before clicking anything:
xpath2 = "your xpath that you are going to click on"
WebDriverWait(driver, timeout=5).until(lambda x:
x.find_element_by_xpath(xpath2))
The xpath approach isn't necessarily better, I just prefer it, you can also use your selector approach.
I had the same problem and with Firefox, I got button element with the following steps:
right click button of interest and select "Inspect Accessibility Properties"
this opens the inspector. Right click the highlighted line and click "Print to JSON"
this opens a new tab. Look for nodeCssSelector and copy the value
This allowed me to accept cookies of the website Yahoo by using.
url = "https://yahoo.com"
driver = Firefox(executable_path="geckodriver.exe")
driver.get(url)
driver.find_element_by_css_selector("button.btn:nth-child(5)").click()
I tested this further and it allowed me to accept individual cookies with ease. Simply repeat the mentioned steps from before to get the button names.
url = "https://yahoo.com"
driver = Firefox(executable_path="geckodriver.exe")
driver.get(url)
driver.find_element_by_css_selector("a.btn").click()
driver.find_element_by_css_selector(".firstPartyAds > div:nth-child(2) > label:nth-child(1)").click()
driver.find_element_by_css_selector(".preciseGeolocation > div:nth-child(2) > label:nth-child(1)").click()
driver.find_element_by_css_selector("button.btn").click()
Another method is to
right click button of interest and select "Inspect"
right click the highlighted line and click "Copy -> CSS Selector" or whatever you need (there are multiple options, including XPath)
However, I think the second method may include whitespaces depending on what you copy, so you might need to manually remove (some of) them. The first method seems to be more foolproof, but I don't know if/how it works on other browsers than Firefox. The second method should work for all browsers.
Use This code To Click On Button
# finding the button using ID
button = driver.find_element_by_id(ID)
# clicking on the button
button.click()
e = driver.find_element(By.XPATH, 's_image').click()
sometime it does not work!
you can try:
e = driver.find_element(By.XPATH, 's_image') driver.execute_script("arguments[0].click();", e)