I'm trying to click a button using Selenium, but everytime I get the message that it can't find the element. This happens even when I put a time.sleep() in front of it.
time.sleep(5)
#Click on download
downloadButton = driver.find_element_by_css_selector('#downloadButton')
time.sleep(5)
downloadButton.click()
#Click on close
closeButton = driver.find_element_by_xpath('//*[#id="btnZorgAnnuleren"]')
closeButton.click()
The button is inside of a frame (not iframe), but I don't know how to switch to it. Here is the frame source code and page source code: https://www.codepile.net/pile/ZoG0REzQ. Can anyone help me out?
If it's iframe :
the I would suggest you to switch to frame like this (with explicit waits):
wait = WebDriverWait(driver, 10)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.ID, "iframe ID")))
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
once you have switched you can click like this :
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#downloadButton"))).click()
and click on close :
wait.until(EC.element_to_be_clickable((By.ID, "btnZorgAnnuleren"))).click()
and once you are done, you should switch to default content like this :
driver.switch_to.default_content()
Related
It closes when click runs.
and ı wanna just click button.
ı wanna do click that xpath. but its doesnt work. it close without doesnt click. so it open edge. waiting waiting and close. it didint run my click command. and there a error. its "webdriver object has no attribute find_element_by_xpath"
from selenium import webdriver
import time
driver = webdriver.Edge()
driver.get("https://www.instagram.com/")
time.sleep(7)
log_in = driver.find_element_by_xpath("//*[#id='mount_0_0_+h']/div/div/div/div[1]/div/div/div/div[1]/section/main/article/div[2]/div[2]/div/p/a/span")
log_in.click()
driver.close()
I think this is what you want:
# Needed libs
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
#Open the browser
driver = webdriver.Edge()
driver.get("https://www.instagram.com/")
# Wait and click for allow cookies button
allow_cookies_button = date = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, f"//button[text()='Only allow essential cookies']")))
allow_cookies_button.click()
# Wait and click for Sign in button
signup_button = date = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, f"//span[text()='Sign up']")))
signup_button.click()
# This time.sleep is to allow you to see what happened, you can delete it
time.sleep(5)
# Close the browser
driver.close()
Why your code did not work? Because as you console told you "find_element_by_xpath" is not a valid method/attribute for driver.
Advises for automation/Scraping:
Wait for the element you will use (it does not matter if you want to write, click or whatever, make sure the element is on the screen)
Try to use xpaths (Or any selector) that are understandable, you will reuse them in the future. What do you understand better, //span[text()='Sign up'] or //*[#id='mount_0_0_+h']/div/div/div/div[1]/div/div/div/div[1]/section/main/article/div[2]/div[2]/div/p/a/span? Good selectors will save a lot of time in the future
I am trying to search www.oddschecker.com using Python and Selenium (chromedriver) but am struggling to populate the search field with the search term.
The code below clicks the magnifying glass ok but the send_keys statement does not populate the expanded search box.
Can anyone see the issue?
driver.find_element_by_xpath('//*[#id="top-menu"]/li[8]/ul/li[1]/span/span').click()
sleep(5)
driver.find_element_by_xpath('/html/body/div[1]/header/div[1]/div/div[1]/div/ul/li[8]/ul/li[1]/div/div/div/form/fieldset/input').send_keys('Bicep')
xpath is brittle, please use relative xpath. see below.
Use explicit waits.
Close modal pop up may appear sometime, may not appear sometime so better to wrap them inside try and except block.
Code :
driver = webdriver.Chrome(driver_path)
driver.maximize_window()
#driver.implicitly_wait(30)
wait = WebDriverWait(driver, 30)
driver.get("http://www.oddschecker.com/")
try:
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "span.js-close-class"))).click()
print('Clicked on close button if it appears')
except:
pass
wait.until(EC.element_to_be_clickable((By.XPATH, "//span[starts-with(#data-ng-click,'NavSearchCtrl')]"))).click()
search = wait.until(EC.visibility_of_element_located((By.ID, "search-input")))
search.send_keys('Bicep', Keys.RETURN)
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I am trying to click on this button so that I can proceed to the next page but doesn't seem to be working for me. I am trying to click on the 'Search for an inspection report' button.
This is the xpath for this source. //[#id="btnOK"] . And so I was trying to use driver.find_element_by_xpath('//[#id="btnOK"]').click() and it gives me an error saying unable to locate element.
Here is the link to the site.
https://www.halton.ca/For-Residents/Food-Safety/Dinewise/Search-Directory-of-Food-Premises-Dinewise
Any help would be great.
It is because the button is present within a different iframe. In order to click that button, u have to switch focus to that iframe. This is how u do it:
iframe = driver.find_element_by_id("iframe-form")
driver.switch_to.frame(iframe)
btn = driver.find_element_by_xpath('//*[#id="btnOK"]')
btn.click()
Plus, another thing to note is that the xpath u have provided is wrong. U have missed a * in the xpath. The right xpath is '//*[#id="btnOK"]'.
Complete code:
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('https://www.halton.ca/For-Residents/Food-Safety/Dinewise/Search-Directory-of-Food-Premises-Dinewise')
iframe = driver.find_element_by_id("iframe-form")
driver.switch_to.frame(iframe)
btn = driver.find_element_by_xpath('//*[#id="btnOK"]')
btn.click()
Hope that this helps.
Alternative solution using webdriver wait:
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
#chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
wd = webdriver.Chrome('<PATH_TO_CHROMEDRIVER>', options=chrome_options)
# load page via selenium
wd.get("https://www.halton.ca/For-Residents/Food-Safety/Dinewise/Search-Directory-of-Food-Premises-Dinewise")
# wait for iframe, switch to it
frame = WebDriverWait(wd, 30).until(EC.frame_to_be_available_and_switch_to_it((By.ID,"iframe-form")))
# wait for button, scroll to it, click it
btn = WebDriverWait(wd, 30).until(EC.presence_of_element_located((By.ID, 'btnOK')))
wd.execute_script("arguments[0].scrollIntoView();", btn)
wd.execute_script("arguments[0].click();", btn)
I want to parse an IMDb film rating located here on around 8 pages. In order to do that I'm using Selenium, and I'm having trouble with clicks, proceeding algorithm to next page. In the end I need 1000 titles when I'll continue using BeautifulSoup. Code below isn't working, I need to use button 'NEXT' with this HTML:
<a class="flat-button lister-page-next next-page" href="/list/ls000004717/?page=2">
Next
</a>
This is the code:
from selenium import webdriver as wb
browser = wb.Chrome()
browser.get('https://www.imdb.com/list/ls000004717/')
field = browser.find_element_by_name("flat-button lister-page-next next-page").click()
Error is the following:
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".flat-button lister-page-next next-page"}
(Session info: chrome=78.0.3904.108)
I suppose I lack knowledge of syntax needed, or maybe I mixed it up a little. I tried searching on SO, though every example is pretty unique and I don't possess the knowledge to extrapolate these cases fully. Any way Selenium can handle that?
You could try using an XPath to query on the Next text inside the button. You should also probably invoke WebDriverWait since you are navigating across multiple pages, then scroll into view since this is at the bottom of the page:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
from time import sleep
browser = wb.Chrome()
browser.get('https://www.imdb.com/list/ls000004717/')
# keep clicking next until we reach the end
for i in range(0,9):
# wait up to 10s before locating next button
try:
next_button = WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "//a[contains(#class, 'page') and contains(text(), 'Next')]")))
# scroll down to button using Javascript
browser.execute_script("arguments[0].scrollIntoView(true);", next_button)
# click the button
# next_button.click() this throws exception -- replace with JS click
browser.execute_script("arguments[0].click();", next_button)
# I never recommend using sleep like this, but WebDriverWait is not waiting on next button to fully load, so it goes stale.
sleep(5)
# case: next button no longer exists, we have reached the end
except TimeoutException:
break
I also wrapped everything in a try / except TimeoutException block to handle the case where we have reached the end of pages, and Next button no longer exists, thus breaking out of the loop. This worked on multiple pages for me.
I also had to add an explicit sleep(5) because even after invoking WebDriverWait on element_to_be_clickable, next_button was still throwing StaleElementReferenceException. It seems like WebDriverWait was finishing before page was fully loaded, causing the status of next_button to change after it had been located. Normally adding sleep(5) is bad practice, but there did not seem to be another workaround here. If anyone else has a suggestion on this, feel free to comment / edit the answer.
There are a couple of ways that could work:
1. Use a selector for the next button and loop until the end:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as ec
browser = webdriver.Chrome()
browser.get('https://www.imdb.com/list/ls000004717/')
selector = 'a[class*="next-page"]'
num_pages = 10
for page in range(pages):
# Wait for the element to load
WebDriverWait(browser, 10).until(ec.presence_of_element_located((By.CSS_SELECTOR, selector)))
# ... Do rating parsing here
browser.find_element_by_css_selector(selector).click()
Instead of clicking on the element, the other option could be to navigate to the next page using broswer.get('...'):
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as ec
# Set up browser as before and navigate to the page
browser = webdriver.Chrome()
browser.get('https://www.imdb.com/list/ls000004717/')
selector = 'a[class*="next-page"]'
base_url = 'https://www.imdb.com/list/ls000004717/'
page_extension = '?page='
# Already at page = 1, so only needs to loop 9 times
for page in range(2, pages + 1):
# Wait for the page to load
WebDriverWait(browser, 10).until(ec.presence_of_element_located((By.CSS_SELECTOR, selector)))
# ... Do rating parsing here
next_page = base_url + page_extension + str(page)
browser.get(next_page)
As a note: field = browser.find_element_by_name("...").click() will not assign field to a webelement, as the click() method has no return value.
You could try a partial css selector.
browser.find_element_by_css_selector("a[class*='next-page']").click()
To click on the element with text as NEXT till the 901 - 1,000 of 1,000 page you have to:
scrollIntoView() the element once the visibility_of_element_located() is achieved.
Induce WebDriverWait for the element_to_be_clickable()
You can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get('https://www.imdb.com/list/ls000004717/')
driver.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "span.pagination-range"))))
while True:
try:
WebDriverWait(driver, 20).until(EC.invisibility_of_element((By.CSS_SELECTOR, "div.row.text-center.lister-working.hidden")))
driver.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "span.pagination-range"))))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.flat-button.lister-page-next.next-page"))).click()
print("Clicked on NEXT button")
except TimeoutException as e:
print("No more NEXT button")
break
driver.quit()
Console Output:
Clicked on NEXT button
Clicked on NEXT button
Clicked on NEXT button
Clicked on NEXT button
Clicked on NEXT button
Clicked on NEXT button
Clicked on NEXT button
Clicked on NEXT button
Clicked on NEXT button
No more NEXT button
I can't find a solution how this element cannot be found by using a selenium xpath. Other button on other websites always working just fine. Usually what I would normally do is, just open up a page and inspect the element and I would right click it and copy the xpath. Done.
But this website, www.gsc.com.my (a malaysian cinema booking site). Seems not able to find the button. Is it protected by another security layer?
Lets see the code below,
from selenium import webdriver
chromedriver_path = './chromedriver.exe'
driver = webdriver.Chrome(chromedriver_path)
driver.get('https://www.gsc.com.my')
driver.find_element_by_xpath("""//*[#id="btnNext"]""").click()
The error Message:
no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id="btnNext"]"}
Button is located inside an iframe, so you need to switch to that frame before clicking the button:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.webdriver.support import expected_conditions as EC
driver.switch_to.frame('getQuickFilter_ctrl_ifrBox')
wait(driver, 10).until(EC.element_to_be_clickable((By.ID, "btnNext"))).click()
Because there are two elements with id btnNext, you'll have to specify which of them using an index, 1 for the first, 2 for the second.
driver.find_element_by_xpath("""//*[#id="btnNext"][1]""").click()
You can try with this css selector :
div.container input#btnNext
Note that you will need to switch to iframe first , cause the check button is in a iframe.
For switching to iframe :
driver.switch_to.frame(driver.find_element_by_id("getQuickFilter_ctrl_ifrBox"))
and for clicking on check Button
wait = WebDriverWait(driver, 10)
check_button = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.container input#btnNext")))
check_button.click()