When I use mechanize, selenium libraries to run a url ("www.maps.google.com" in this case),
Chrome opens with a note saying that "Chrome is being controlled by an automated testing software".
But when I try to inspect element, there is only one element: body. All other inputs and buttons are not showing.
I want to automate the process to find distance between two addresses, so how do I solve the above problem?
from selenium import webdriver
driver = webdriver.Chrome(executable_path='C:/chromedriver.exe')
# Go to your page url
driver.get('https://www.google.com/maps')
# Get button you are going to click by its id ( also you could us find_element_by_css_selector to get element by css selector)
button_element = driver.find_element_by_id('searchbox-directions')
button_element.click()
dest_add = driver.find_element_by_class_name("tactile-searchbox-input")
dest_add.send_keys("Agra")
start_add = driver.find_element_by_class_name("tactile-searchbox-input")
start_add.send_keys("Jaipur")
For example this doesn't work. Since, there are no input fields so naturally, no elements with the class name "tactile-searchbox-input".
Induce WebDriverWait() and wait for element_to_be_clickable() and following css selector
driver.get('https://www.google.com/maps')
WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.ID,"searchbox-directions"))).click()
WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"input[aria-label*='starting point']"))).send_keys("Agra")
WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"input[aria-label*='destination']"))).send_keys("Jaipur")
You need to import below libraries.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
To click on search button try this.
WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.XPATH,"(//button[#aria-label='Search'])[last()]"))).click()
Related
Trying to automate one thing for my work, which was choosing one option from the dropdown list on the website below:
https://interparking-pl.my.site.com/abonament/s/?id=a0A58000000D7pZ
The Selenium automation didn't work in that case. After writing such code:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
chrome_driver_path = r"C:/Users/.../Projects/chromedriver.exe"
service = Service(executable_path=chrome_driver_path)
driver = webdriver.Chrome(service=service)
driver.get("https://interparking-pl.my.site.com/abonament/s/?id=a0A58000000D7pZ")
wait = WebDriverWait(driver, 10)
abo_button = wait.until(EC.element_to_be_clickable((By.XPATH, '//*[#id="combobox-button-53"]')))
After executing I've got a message:
TimeoutException
In case of finding element by tag name or any other options, the following message pops up:
Message: no such element: Unable to locate element
The list is built on button tags and has a structure of lightning-basecombobox. It looks like there is no possibility to click on the dropdown list and choose the required option automatically.
Is it needed to do something different with such stuff?
What I expect is to use Selenium to choose between the options in the list.
The combobox is a <button> element.
Additionally the id i.e. combobox-button-53 is dynamically generated and is bound to chage sooner/later. They may change next time you access the application afresh or even while next application startup. So can't be used in locators.
Solution
To click element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
driver.get("https://interparking-pl.my.site.com/abonament/s/?id=a0A58000000D7pZ")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[name='subscriptionsTypeId']"))).click()
Using XPATH:
driver.get("https://interparking-pl.my.site.com/abonament/s/?id=a0A58000000D7pZ")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#name='subscriptionsTypeId']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot;
When I load the page, I get the element ID (XPATH) as following. //*[#id="combobox-button-51"]
Maybe the number 51 isn't always the same? In that case, try:
//*[starts-with(#id,'combobox-button-5')]
Or just use //*[#name="subscriptionsTypeId"] , as another answer here allready mentioned.
My school has a system that tells us if our schedule has any changes.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait, Select
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.select import Select
url = "https://www.alliancetlv.com/עדכוני-מערכת"
driver = webdriver.Chrome()
driver.get(url)
driver.implicitly_wait(5.0)
examButton = driver.find_element(By.ID, 'TimeTableView1_btnChanges')
im trying to find an element, and later click it using selenium. every time i try to find literally anything it returns No Such Element error. I tried by ID, class name, name, and more.
this is the website: https://www.alliancetlv.com/עדכוני-מערכת
and im trying to click one of the tabs called "changes/שינויים"
my end goal is to click the dropdown to the side, select a class, click the changes tab, then get all the data inside of it, then maybe format it.
You can not seek the element because it is layed inside the iframe. So, you have to:
find iframe
switch to it
find your element
click on it
frame = driver.find_element(By.XPATH, value="//iframe[#title='Embedded Content']")
driver.switch_to.frame(frame)
elem = driver.find_element(By.XPATH, value="//a[contains(#id,'TimeTableView1_btnChanges')]")
elem.click()
See docs...
I have a python script, It look like this.
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.select import Select
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from os import path
import time
# Tried this code
chrome_options = webdriver.ChromeOptions()
prefs = {"profile.default_content_setting_values.notifications" : 2}
chrome_options.add_experimental_option("prefs",prefs)
browser = webdriver.Chrome(ChromeDriverManager().install(), chrome_options=chrome_options)
links = ["https://www.henleyglobal.com/", "https://markets.ft.com/data"]
for link in links:
browser.get(link)
#WebDriverWait(browser, 20).until(EC.url_changes(link))
#How do I disable/Ignore/remove/escape this "Accept all cookie" popup and then access the website to scrape data?
browser.quit()
So each website in the links array displays an "Accept all cookie" popup after navigating to the site. check the below image.
I have tried many ways nothing works, Check the one after imports
How do I exit/pass/escape this popup and then access the website to scrape data?
If you open your page in a new browser you'll note the page fully loads, then, a moment later your popup appears. The default wait strategy in selenium is just that the page is loaded.
One way to handle this is to simply inspect the page and find the xpath of the popup window. The below code should work for that.
browser.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS)
if link == 'https://www.henleyglobal.com/':
browser.findElement(By.XPATH("/html/body/div[7]/div/div/div/div[2]/div/div[2]/button[2]")).click()
else:
browser.findElement(By.XPATH("/html/body/div[4]/div/div/div[2]/div[2]/a")).click()
The code is waiting until the element of the pop-up is clickable and then clicking it.
For unknown sites you could try:
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--disable-notifications")
webdriver.Chrome(os.path.join(path, 'chromedriver'), chrome_options=chrome_options)
generally, you can not use some universal locator that will match the "Accept cookies" buttons for each and every web site in the world.
Even here, you have 2 different sites and the elements you need to click are totally different on these sites.
For https://www.henleyglobal.com/ site the correct locator may be something like this CSS Selector .confirmation button.primary-btn while for https://markets.ft.com/data site I'd advise to use CSS Selector .o-cookie-message__actions a.o-cookie-message__button.
These 2 elements are totally different: the first one is button while the second is a, they have totally different class names and all other attributes.
You may thing about the Accept text. It seems to be common, so you could use this XPath //*[contains(text(),'Accept')] but even this will not work since on the first page it matches 2 elements while the accept cookies element is the second between them...
So, there is no General locators, you will have to define separate locators for each page.
Again, for https://www.henleyglobal.com/ I would prefer
driver.find_element(By.CSS_SELECTOR, ".confirmation button.primary-btn").click()
While for the second page https://markets.ft.com/data I would prefer this
driver.find_element(By.CSS_SELECTOR, ".o-cookie-message__actions a.o-cookie-message__button").click()
Also, generally we always use WebDriverWait expected_conditions explicit waits, so the code will be as following:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
# for the first page
wait.until(EC.element_to_be_clickable((By.XPATH, ".confirmation button.primary-btn"))).click()
# for the second page
wait.until(EC.element_to_be_clickable((By.XPATH, ".o-cookie-message__actions a.o-cookie-message__button"))).click()
I am wondering if someone can point me in the right direction to get an address search to work. For reference, I am trying to enter an address in the Address Search input box and click the "Search" button on this page: https://beacon.schneidercorp.com/Application.aspx?AppID=44&LayerID=260&PageTypeID=2&PageID=297. A valid address is: "1770 Mobile Ave"
This page may or may not load for you, I think it might be using some kind of token.
Anyhow, I have tried WebDriverWait with xpath, find_element_by_css_selector, find_element_by_name, etc. to access the input field and then use send_keys but nothing seems to allow me to get past an "ElementNotInteractableException".
My working code starts at the home page and everything here works fine to get to the search page in the picture:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
options = webdriver.ChromeOptions()
options.headless = True
driver = webdriver.Chrome(options=options)
url = "https://beacon.schneidercorp.com/"
driver.get(url)
select_st = Select(driver.find_element_by_id('stateMenu'))
select_st.select_by_visible_text('Iowa')
select_st = Select(driver.find_element_by_id('areaMenu'))
select_st.select_by_visible_text('Harrison County, IA')
parentElement = driver.find_element_by_class_name('list-group')
elementList = parentElement.find_elements_by_tag_name("a")
elementList[0].click()
You can try to skip the above and start at the search page - I only mention it in case the search page won't load.
Again, some things I have tried to fill out the input include:
address = driver.find_element_by_css_selector('input[placeholder="enter address..."]')
address.send_keys("1770 Mobile Ave")
address = driver.find_element_by_name("ctlBodyPane$ctl01$ctl01$txtAddress")
address.send_keys("1770 Mobile Ave")
address = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.ID, 'ctlBodyPane_ctl01_ctl01_txtAddress')))
address.send_keys("1770 Mobile Ave")
Am I missing something? It seems like the answer here is obvious but I'm overlooking it.
I have noticed once move to next page from first page one pop up is appeared for terms & condition and you need to click on that first.
Induce WebDriverWait() and wait for element_to_be_clickable()
#click on terms & condition page
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//a[text()='Agree']"))).click()
#Enter data in address field
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.ID,"ctlBodyPane_ctl01_ctl01_txtAddress"))).send_keys("1770 Mobile Ave")
You need to import below libraries.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Let me know how this goes?
It worked for me. Using the method find_element_by_xpath(). Other selector that you mentioned was for other elements or incorrect. for example this ctlBodyPane_ctl01_ctl01_txtAddress points the outer element of the search box which is obviously not interactable. Using input[placeholder="enter address..."] this selector you were close but this returns 2 matching elements. To fix this I converted this to xpath and wrapped it for index to return unique matching element.
Here is the part of code fix -
address = driver.find_element_by_xpath('(.//input[#placeholder="enter address..."])[1]')
address.send_keys("1770 Mobile Ave")
Hoping you can help. I'm relatively new to Python and Selenium. I'm trying to pull together a simple script that will automate news searching on various websites. The primary focus was football and to go and get me the latest Manchester United news from a couple of places and save the list of link titles and URLs for me. I could then look through the links myself and choose anything I wanted to review.
In trying the the independent newspaper (https://www.independent.co.uk/) I seem to have come up against a problem with element not interactable when using the following approaches:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome('chromedriver')
driver.get('https://www.independent.co.uk')
time.sleep(3)
#accept the cookies/privacy bit
OK = driver.find_element_by_id('qcCmpButtons')
OK.click()
#wait a few seconds, just in case
time.sleep(5)
search_toggle = driver.find_element_by_class_name('icon-search.dropdown-toggle')
search_toggle.click()
This throws the selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable error
I've also tried with XPATH
search_toggle = driver.find_element_by_xpath('//*[#id="quick-search-toggle"]')
and I also tried ID.
I did a lot of reading on here and then also tried using WebDriverWait and execute_script methods:
element = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, '//*[#id="quick-search-toggle"]')))
driver.execute_script("arguments[0].click();", element)
This didn't seem to error but the search box never appeared, i.e. the appropriate click didn't happen.
Any help you could give would be fantastic.
Thanks,
Pete
Your locator is //*[#id="quick-search-toggle"], there are 2 on the page. The first is invisible and the second is visible. By default selenium refers to the first element, sadly the element you mean is the second one, so you need another unique locator. Try this:
search_toggle = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//div[#class="row secondary"]//a[#id="quick-search-toggle"]')))
search_toggle.click()
First you need to open search box, then send search keys:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
import os
chrome_options = Options()
chrome_options.add_argument("--start-maximized")
browser = webdriver.Chrome(executable_path=os.path.abspath(os.getcwd()) + "/chromedriver", options=chrome_options)
link = 'https://www.independent.co.uk'
browser.get(link)
# accept privacy
button = browser.find_element_by_xpath('//*[#id="qcCmpButtons"]/button').click()
# open search box
li = browser.find_element_by_xpath('//*[#id="masthead"]/div[3]/nav[2]/ul/li[1]')
search_tab = li.find_element_by_tag_name('a').click()
# send keys to search box
search = browser.find_element_by_xpath('//*[#id="gsc-i-id1"]')
search.send_keys("python")
search.send_keys(Keys.RETURN)
Can you try with below steps
search_toggle = driver.find_element_by_xpath('//*[#class="row secondary"]/nav[2]/ul/li[1]/a')
search_toggle.click()