What can I do to replace find_element_css_selector in Selenium Python to click on a specific onclick value
(javascript:unitsSelectUnit(1))
The browser is google chrome
browser.get("http://eatsmart.housing.illinois.edu/NetNutrition/46#")
browser.find_element_css_selector("a[onclick*=javascript:unitsSelectUnit(1);]").click()
html = browser.page_source
time.sleep(2)
print(html)
# close web browser
browser.close()
To click on the element with text as Ikenberry Dining Center (IKE) you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using LINK_TEXT:
driver.get("http://eatsmart.housing.illinois.edu/NetNutrition/46#")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.LINK_TEXT, "New"))).click()
Using CSS_SELECTOR:
driver.get("http://eatsmart.housing.illinois.edu/NetNutrition/46#")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[onclick*='unitsSelectUnit(1)']"))).click()
Using XPATH:
driver.get("http://eatsmart.housing.illinois.edu/NetNutrition/46#")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[contains(#onclick, 'unitsSelectUnit(1)')]"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot:
This CSS_SELECTOR
a[onclick*=javascript:unitsSelectUnit(1);]
that you are using in your code is wrong. Basically, you are missing ''
CSS_SELECTOR follow the below syntax:
node_name[attribute_name = 'attribute_value']
You can cross verify as well by following the below steps:
Steps to check:
Press F12 in Chrome -> go to element section -> do a CTRL + F -> then paste the css selector and see, if your desired element is getting highlighted with 1/1 matching node.
This a[onclick*=javascript:unitsSelectUnit(1);] won't match anything, where as a[onclick*='javascript:unitsSelectUnit(1);'] will match Ikenberry Dining Center (IKE) node.
Now If you want to click on it, please use the below code:
Code:
browser.maximize_window()
wait = WebDriverWait(browser, 30)
browser.get("http://eatsmart.housing.illinois.edu/NetNutrition/46#")
try:
btn = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[onclick*='javascript:unitsSelectUnit(1);']")))
btn.click()
except:
print("Could not click on button")
pass
Imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Related
I would like to scrape data from this page: https://www.investing.com/equities/nvidia-corp-financial-summary.
There are two buttons that I'd like to click:
Accept button.
Checking the XPath of the button:
XPath = //*[#id="onetrust-accept-btn-handler"]
Replicating the steps performed here: Clicking a button with selenium using Xpath doesn't work
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
wait = WebDriverWait(driver, 5)
link= wait.until(EC.element_to_be_clickable((By.XPATH, "//*[#id="onetrust-accept-btn-handler")))
I got the error: SyntaxError: invalid syntax
Annual button
there is a toggle between Annual and Quarterly (default is quarterly)
XPath is //*[#id="leftColumn"]/div[9]/a[1]
wait.until(EC.element_to_be_clickable((By.XPATH, "//*[#id="leftColumn"]/div[9]/a[1]")))
also returned invalid Syntax.
Updated Code
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
company = 'nvidia-corp'
driver = webdriver.Chrome(path)
driver.get(f"https://www.investing.com/equities/{company}-financial-summary")
wait = WebDriverWait(driver, 2)
accept_link= wait.until(EC.element_to_be_clickable((By.XPATH, '//*[#id="onetrust-accept-btn-handler"]')))
accept_link.click()
scrollDown = "window.scrollBy(0,500);"
driver.execute_script(scrollDown)
#scroll down to get the page data below the first scroll
driver.maximize_window()
time.sleep(10)
wait = WebDriverWait(driver, 2)
scrollDown = "window.scrollBy(0,4000);"
driver.execute_script(scrollDown)
#scroll down to get the page data below the first scroll
try:
close_popup_link= wait.until(EC.element_to_be_clickable((By.XPATH,'//*[#id="PromoteSignUpPopUp"]/div[2]/i')))
close_popup_link.click()
except NoSuchElementException:
print('No such element')
wait = WebDriverWait(driver, 3)
try:
annual_link = wait.until(EC.element_to_be_clickable((By.XPATH, '//*[#id="leftColumn"]/div[9]/a[1]')))
annual_link()
# break
except NoSuchElementException:
print('No element of that id present!')
The first accept button was successfully clicked,
but clicking the Annual button returns Timeout Exception error.
Annual button
At least for me I saw we need to use another locator to access that element.
I used scrolling until I can click that element.
The following code works for me:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("--start-maximized")
s = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=s)
company = 'nvidia-corp'
wait = WebDriverWait(driver, 5)
driver.get(f"https://www.investing.com/equities/{company}-financial-summary")
try:
wait.until(EC.element_to_be_clickable((By.XPATH, '//*[#id="PromoteSignUpPopUp"]/div[2]/i'))).click()
except:
pass
while True:
try:
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[#class='float_lang_base_1']//a[#data-ptype='Annual']"))).click()
break
except:
driver.execute_script("window.scrollBy(0, arguments[0]);", 1000)
You need to take care of a couple of things here as follows:
If you are supplying the xpath within double qoutes, i.e. "..." then the attribute values needs to be within single quotes, i.e. '...'
Similarly, if you are supplying the xpath within single qoutes, i.e. '...' then the attribute values needs to be within double quotes, i.e. "..."
This take care of both the SyntaxError: invalid syntax
Effectively, the lines of code will be:
link= wait.until(EC.element_to_be_clickable((By.XPATH, "//*[#id='onetrust-accept-btn-handler')))
and
wait.until(EC.element_to_be_clickable((By.XPATH, "//*[#id='leftColumn']/div[9]/a[1]")))
Solution
To click on the clickable elements you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Clicking on I Accept:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#id='onetrust-accept-btn-handler']"))).click()
Clicking on Annual:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[data-ptype='Annual']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[#data-ptype='Annual']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I am using Selenium with Python to webscrape a page which includes JavaScript.
The racecourse result tabs towards the top of the page eg "Ludlow","Dundalk" are manually clickable but do not have any obvious hyperlinks attached to them.
...
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome(executable_path='C:/A38/chromedriver_win32/chromedriver.exe')
driver.implicitly_wait(30)
driver.maximize_window()
# Navigate to the application home page
driver.get("https://www.sportinglife.com/racing/results/2020-11-23")
...
This works so far. I have used BeautifulSoup to find the label names of the NewGenericTabs eg "Ludlow","Dundalk" etc. However, the following code, to attempt to automate clicking a tab, times out each time.
WebDriverWait(driver, 60).until(EC.element_to_be_clickable((By.LINK_TEXT, "Ludlow"))).click()
Would welcome any help.
The WebElement aren't <a> tags but <span> tags so By.LINK_TEXT wouldn't work.
To click on the desired elements you can use either of the following xpath based Locator Strategies:
Ludlow:
driver.get("https://www.sportinglife.com/racing/results/2020-11-23")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#mode='primary']"))).click()
driver.find_element_by_xpath("//span[#data-test-id='generic-tab' and text()='Ludlow']").click()
Dundalk:
driver.get("https://www.sportinglife.com/racing/results/2020-11-23")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#mode='primary']"))).click()
driver.find_element_by_xpath("//span[#data-test-id='generic-tab' and text()='Dundalk']").click()
Ideally, to click on the elements you need to induce WebDriverWait for the element_to_be_clickable() and you can use the following xpath based Locator Strategies:
Ludlow:
driver.get("https://www.sportinglife.com/racing/results/2020-11-23")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#mode='primary']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[#data-test-id='generic-tab' and text()='Ludlow']"))).click()
Dundalk:
driver.get("https://www.sportinglife.com/racing/results/2020-11-23")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#mode='primary']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[#data-test-id='generic-tab' and text()='Dundalk']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
driver.get("https://twitter.com/i/flow/signup")
driver.implicitly_wait(10)
setname = driver.find_element_by_name("name")
setname.click()
setname.send_keys("NAME SURNAME")
driver.implicitly_wait(10)
emailoption = driver.find_element_by_xpath("/html/body/div/div/div/div[2]/main/div/div/div/div[2]/div[2]/div/div[4]") #selenium can not find the element XPATH (I found it manually)
emailoption.click() #need to click in the element :)
driver.close()
Selenium can not find the element XPATH (I found it manually) ---Use email instead---.
Tried to find this element other ways... same result.
Maybe there is another way to click in it?
Add a Selenium expected condition (element_to_be_clickable) to your code and use relative XPath. To input your name, click on the link and input your email, you can use :
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#autocomplete='name']"))).send_keys('name')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#autocomplete='tel']/following::span[1]']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#autocomplete='email']"))).send_keys('email')
Be sure to add the following imports :
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
If it still fails you can use Javascript :
name = "your_name"
email = "your_email"
elem = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#autocomplete='name']")))
self.driver.execute_script("arguments[0].setAttribute('value', '" + name +"')", elem)
elem2 = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#autocomplete='tel']/following::span[1]']")))
self.driver.execute_script("arguments[0].click();", elem2)
elem3 = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#autocomplete='email']")))
self.driver.execute_script("arguments[0].setAttribute('value', '" + email +"')", elem3)
The elements within Twitter Signup page are React elements. So to send a character sequence to the Name field you have to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
driver.get("https://twitter.com/i/flow/signup")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[name='name']"))).send_keys("Manoel Augusto")
Using XPATH:
driver.get("https://twitter.com/i/flow/signup")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#name='name']"))).send_keys("Manoel Augusto")
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot:
Reference
You can find a detailed relevant discussion in:
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element error sending text to Email field in twitter with Selenium Python
Try below code -
from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
driver = webdriver.Chrome()
action = ActionChains(driver)
wait = WebDriverWait(driver, 20)
driver.get('https://twitter.com/i/flow/signup')
time.sleep(5) # Add wait here so that twitter login page will load.
NameElement = driver.find_element_by_xpath("//input[#name='name']")
action.move_to_element(NameElement).click().perform()
NameElement.send_keys("Hello")
EmailOption = driver.find_element_by_xpath('//span[text()="Use email instead"]/parent::div')
EmailOption.click()
driver.close()
I ran your code on my computer and It worked perfectly, finding the email option and clicking on it. I also found the same xpath you provided, have you tried running your program in a debugger, and What IDE/text editor are you using? Also, are you leaving the window open when you are running your program? If you close the window it will throw an error. If you have any more details on what is happening when you run your code let me know I'd love to help.
Try below code :
wait = WebDriverWait(driver, 10)
driver.get("https://twitter.com/i/flow/signup")
driver.implicitly_wait(10)
setname = driver.find_element_by_name("name")
setname.click()
setname.send_keys("NAME SURNAME")
driver.implicitly_wait(10)
element1 = wait.until(EC.element_to_be_clickable((By.XPATH, "//body//div[4]")))
element1.click()
Output::
So the webpage has a button that after clicking will add an element to the webpage, in which I can't find using selenium
Some imaginary code as follows to explain the problem I experience:
from selenium import webdriver
d = webdriver.Chrome()
#Go to my target website
d.get("https://some_website_url") #ref1
#Okay now loading of the website is done. `d` will not be updated and this is the problem!!
#Click my target button and an element with id="SecretButton" is loaded.
d.find_element_by_css_selector("#secretlyupdatethewebpage").click()
#Find #SecretButton but to no avail.
#It can be found in the html panel of Chrome Developer Tools
#but cannot be found in the webdriver `d`, as `d` won't be
#updated after #ref1
d.find_element_by_css_selector("#SecretButton").click()
How can I find that #SecretButton?
To find and invoke click() on the secret button you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using ID:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.ID, "SecretButton"))).click()
Using CSS_SELECTOR:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#SecretButton"))).click()
Using XPATH:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//*[#id='SecretButton']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I am new to the world of python and I am trying to select a couple of options on the following website and then click the search button to update results. However, I cannot get the button to respond.
I tried using search button.click() and .submit() and I have tried to implicitly wait. I have also used the code below to wait until the button is clickable. When executing the code, it highlights the button but doesn't seem to release the click; almost like a half click.
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Safari()
driver.get('https://leasing.com/personal/car-leasing/')
element = driver.find_element_by_id('selUpfront')
select = Select(element)
select.select_by_value("3")
element = driver.find_element_by_id('selMileage')
select = Select(element)
select.select_by_value("8000")
searchbutton = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.ID, "search-button")))
searchbutton.click()
I would expect the search results to be updated with the conditions above.
Seems you were close. To click() on the element you have to induce WebDriverWait for the element to be clickable and you can use either of the following solutions:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#search-button>i.fa.fa-search#search-button-icon"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#id='search-button']/i[#class='fa fa-search' and #id='search-button-icon']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot:
There are 2 elements with the search-button you need to use the xpath to locate specific
searchbutton = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#id='search-button']")))
searchbutton.click()