I am using Selenium with Python to webscrape a page which includes JavaScript.
The racecourse result tabs towards the top of the page eg "Ludlow","Dundalk" are manually clickable but do not have any obvious hyperlinks attached to them.
...
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome(executable_path='C:/A38/chromedriver_win32/chromedriver.exe')
driver.implicitly_wait(30)
driver.maximize_window()
# Navigate to the application home page
driver.get("https://www.sportinglife.com/racing/results/2020-11-23")
...
This works so far. I have used BeautifulSoup to find the label names of the NewGenericTabs eg "Ludlow","Dundalk" etc. However, the following code, to attempt to automate clicking a tab, times out each time.
WebDriverWait(driver, 60).until(EC.element_to_be_clickable((By.LINK_TEXT, "Ludlow"))).click()
Would welcome any help.
The WebElement aren't <a> tags but <span> tags so By.LINK_TEXT wouldn't work.
To click on the desired elements you can use either of the following xpath based Locator Strategies:
Ludlow:
driver.get("https://www.sportinglife.com/racing/results/2020-11-23")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#mode='primary']"))).click()
driver.find_element_by_xpath("//span[#data-test-id='generic-tab' and text()='Ludlow']").click()
Dundalk:
driver.get("https://www.sportinglife.com/racing/results/2020-11-23")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#mode='primary']"))).click()
driver.find_element_by_xpath("//span[#data-test-id='generic-tab' and text()='Dundalk']").click()
Ideally, to click on the elements you need to induce WebDriverWait for the element_to_be_clickable() and you can use the following xpath based Locator Strategies:
Ludlow:
driver.get("https://www.sportinglife.com/racing/results/2020-11-23")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#mode='primary']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[#data-test-id='generic-tab' and text()='Ludlow']"))).click()
Dundalk:
driver.get("https://www.sportinglife.com/racing/results/2020-11-23")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#mode='primary']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[#data-test-id='generic-tab' and text()='Dundalk']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Related
I'm trying to click on a button, but selenium can't find the element.
I'm using the xpath:
//button[#title="Monitor Chart(Ctrl+Alt+G)"]
but selenium can't locate. I can find the xpath from Chrome manualy using the find(Ctrl+F) tool, as we can see in the image.
Here is my code and error. The xpath does't have an ID.
Code Snapshot:
To click on the clickable element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button.eui-btn.eui-btn-default.eui-btn-normal[title^='Monitor Chart']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#class='eui-btn eui-btn-default eui-btn-normal' and starts-with(#title, 'Monitor Chart')]"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
hiding_button = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[title ='Monitor Chart(Ctrl+Alt+G)']")))
hiding_button.click()
Thanks to all. But I figure it out the issue. The button was inside an iFrame. So I hadd to switch to the iFrame first.
Code used:
iframe = driver.find_element_by_id('Access_Performance_Monitor')
driver.switch_to.frame(iframe)
What can I do to replace find_element_css_selector in Selenium Python to click on a specific onclick value
(javascript:unitsSelectUnit(1))
The browser is google chrome
browser.get("http://eatsmart.housing.illinois.edu/NetNutrition/46#")
browser.find_element_css_selector("a[onclick*=javascript:unitsSelectUnit(1);]").click()
html = browser.page_source
time.sleep(2)
print(html)
# close web browser
browser.close()
To click on the element with text as Ikenberry Dining Center (IKE) you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using LINK_TEXT:
driver.get("http://eatsmart.housing.illinois.edu/NetNutrition/46#")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.LINK_TEXT, "New"))).click()
Using CSS_SELECTOR:
driver.get("http://eatsmart.housing.illinois.edu/NetNutrition/46#")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[onclick*='unitsSelectUnit(1)']"))).click()
Using XPATH:
driver.get("http://eatsmart.housing.illinois.edu/NetNutrition/46#")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[contains(#onclick, 'unitsSelectUnit(1)')]"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot:
This CSS_SELECTOR
a[onclick*=javascript:unitsSelectUnit(1);]
that you are using in your code is wrong. Basically, you are missing ''
CSS_SELECTOR follow the below syntax:
node_name[attribute_name = 'attribute_value']
You can cross verify as well by following the below steps:
Steps to check:
Press F12 in Chrome -> go to element section -> do a CTRL + F -> then paste the css selector and see, if your desired element is getting highlighted with 1/1 matching node.
This a[onclick*=javascript:unitsSelectUnit(1);] won't match anything, where as a[onclick*='javascript:unitsSelectUnit(1);'] will match Ikenberry Dining Center (IKE) node.
Now If you want to click on it, please use the below code:
Code:
browser.maximize_window()
wait = WebDriverWait(browser, 30)
browser.get("http://eatsmart.housing.illinois.edu/NetNutrition/46#")
try:
btn = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[onclick*='javascript:unitsSelectUnit(1);']")))
btn.click()
except:
print("Could not click on button")
pass
Imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I want to use selenium to login in the website https://www.winamax.es/account/login.php?redir=/apuestas-deportivas. The case is that I don't find the xpath/id/text to get te next code running successfully:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
options = webdriver.ChromeOptions()
options.add_argument("window-size=1920,1080")
options.add_argument('--disable-blink-features=AutomationControlled')
driver=webdriver.Chrome(options=options,executable_path=r"chromedriver.exe")
driver.get("https://www.winamax.es/account/login.php?redir=/apuestas-deportivas")
WebDriverWait(driver=driver, timeout=15).until(
lambda x: x.execute_script("return document.readyState === 'complete'")
)
upload_field = driver.find_element_by_xpath("//input[#type='email']")
I don't only want the specific xpath for this example, but I prefer a method to obtain the xpath or something similar to get the code working for other parts of the website
<iframe id="iframe-login" data-node="iframe" name="login" scrolling="auto" frameborder="1" style="min-height: 280px; width: 100%;"></iframe>
Your element is in an iframe. Switch to it.
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.ID, "iframe-login")))
Import
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Full working code.
wait = WebDriverWait(driver, 10)
driver.get("https://www.winamax.es/account/login.php?redir=/apuestas-deportivas")
wait.until(EC.frame_to_be_available_and_switch_to_it((By.ID, "iframe-login")))
upload_field = wait.until(EC.element_to_be_clickable((By.XPATH, "//input[#type='email']")))
upload_field.send_keys("stuff")
The email element is within an <iframe> so you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use either of the following Locator Strategies:
Using CSS_SELECTOR:
driver.get("https://www.winamax.es/account/login.php?redir=/apuestas-deportivas")
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe#iframe-login")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[type='email']"))).send_keys("scraper#stackoverflow.com")
Using XPATH:
driver.get("https://www.winamax.es/account/login.php?redir=/apuestas-deportivas")
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#id='iframe-login']")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#type='email']"))).send_keys("scraper#stackoverflow.com")
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot:
Reference
You can find a couple of relevant discussions in:
Ways to deal with #document under iframe
Switch to an iframe through Selenium and python
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element while trying to click Next button with selenium
selenium in python : NoSuchElementException: Message: no such element: Unable to locate element
As titled, seems like I just can't select the drop down menu from this website no matter what.
from selenium import webdriver
from selenium.webdriver.support.ui import Select
driver=webdriver.Chrome()
driver.get('https://assessing.nashuanh.gov/search.asp')
time.sleep(1)
select=Select(driver.find_element_by_xpath('//*[#id="cboSearchType"]'))
select.select_by_value('2')
First you have to handdle the frames in your page. Also looks like there is no value 2 inside this dropdown, so you have to pass a valid value.
driver.switch_to.frame("middle")
select = Select(driver.find_element_by_xpath('//*[#id="cboSearchType"]'))
select.select_by_value('Parcel')
The <option> element with value/text as Owner within the html-select is within an <frame> so you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use either of the following Locator Strategies:
Using CSS_SELECTOR and select_by_value():
driver.get('https://assessing.nashuanh.gov/search.asp')
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"frame[name='middle']")))
Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "select#cboSearchType")))).select_by_value("Owner")
Using XPATH and select_by_visible_text():
driver.get('https://assessing.nashuanh.gov/search.asp')
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//frame[#name='middle']")))
Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//select[#id='cboSearchType']")))).select_by_visible_text("Owner")
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot:
Reference
You can find a couple of relevant discussions in:
Ways to deal with #document under iframe
Switch to an iframe through Selenium and python
So the webpage has a button that after clicking will add an element to the webpage, in which I can't find using selenium
Some imaginary code as follows to explain the problem I experience:
from selenium import webdriver
d = webdriver.Chrome()
#Go to my target website
d.get("https://some_website_url") #ref1
#Okay now loading of the website is done. `d` will not be updated and this is the problem!!
#Click my target button and an element with id="SecretButton" is loaded.
d.find_element_by_css_selector("#secretlyupdatethewebpage").click()
#Find #SecretButton but to no avail.
#It can be found in the html panel of Chrome Developer Tools
#but cannot be found in the webdriver `d`, as `d` won't be
#updated after #ref1
d.find_element_by_css_selector("#SecretButton").click()
How can I find that #SecretButton?
To find and invoke click() on the secret button you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using ID:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.ID, "SecretButton"))).click()
Using CSS_SELECTOR:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#SecretButton"))).click()
Using XPATH:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//*[#id='SecretButton']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC