I am wondering if someone can point me in the right direction to get an address search to work. For reference, I am trying to enter an address in the Address Search input box and click the "Search" button on this page: https://beacon.schneidercorp.com/Application.aspx?AppID=44&LayerID=260&PageTypeID=2&PageID=297. A valid address is: "1770 Mobile Ave"
This page may or may not load for you, I think it might be using some kind of token.
Anyhow, I have tried WebDriverWait with xpath, find_element_by_css_selector, find_element_by_name, etc. to access the input field and then use send_keys but nothing seems to allow me to get past an "ElementNotInteractableException".
My working code starts at the home page and everything here works fine to get to the search page in the picture:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
options = webdriver.ChromeOptions()
options.headless = True
driver = webdriver.Chrome(options=options)
url = "https://beacon.schneidercorp.com/"
driver.get(url)
select_st = Select(driver.find_element_by_id('stateMenu'))
select_st.select_by_visible_text('Iowa')
select_st = Select(driver.find_element_by_id('areaMenu'))
select_st.select_by_visible_text('Harrison County, IA')
parentElement = driver.find_element_by_class_name('list-group')
elementList = parentElement.find_elements_by_tag_name("a")
elementList[0].click()
You can try to skip the above and start at the search page - I only mention it in case the search page won't load.
Again, some things I have tried to fill out the input include:
address = driver.find_element_by_css_selector('input[placeholder="enter address..."]')
address.send_keys("1770 Mobile Ave")
address = driver.find_element_by_name("ctlBodyPane$ctl01$ctl01$txtAddress")
address.send_keys("1770 Mobile Ave")
address = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.ID, 'ctlBodyPane_ctl01_ctl01_txtAddress')))
address.send_keys("1770 Mobile Ave")
Am I missing something? It seems like the answer here is obvious but I'm overlooking it.
I have noticed once move to next page from first page one pop up is appeared for terms & condition and you need to click on that first.
Induce WebDriverWait() and wait for element_to_be_clickable()
#click on terms & condition page
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//a[text()='Agree']"))).click()
#Enter data in address field
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.ID,"ctlBodyPane_ctl01_ctl01_txtAddress"))).send_keys("1770 Mobile Ave")
You need to import below libraries.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Let me know how this goes?
It worked for me. Using the method find_element_by_xpath(). Other selector that you mentioned was for other elements or incorrect. for example this ctlBodyPane_ctl01_ctl01_txtAddress points the outer element of the search box which is obviously not interactable. Using input[placeholder="enter address..."] this selector you were close but this returns 2 matching elements. To fix this I converted this to xpath and wrapped it for index to return unique matching element.
Here is the part of code fix -
address = driver.find_element_by_xpath('(.//input[#placeholder="enter address..."])[1]')
address.send_keys("1770 Mobile Ave")
Related
I have a python script, It look like this.
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.select import Select
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from os import path
import time
# Tried this code
chrome_options = webdriver.ChromeOptions()
prefs = {"profile.default_content_setting_values.notifications" : 2}
chrome_options.add_experimental_option("prefs",prefs)
browser = webdriver.Chrome(ChromeDriverManager().install(), chrome_options=chrome_options)
links = ["https://www.henleyglobal.com/", "https://markets.ft.com/data"]
for link in links:
browser.get(link)
#WebDriverWait(browser, 20).until(EC.url_changes(link))
#How do I disable/Ignore/remove/escape this "Accept all cookie" popup and then access the website to scrape data?
browser.quit()
So each website in the links array displays an "Accept all cookie" popup after navigating to the site. check the below image.
I have tried many ways nothing works, Check the one after imports
How do I exit/pass/escape this popup and then access the website to scrape data?
If you open your page in a new browser you'll note the page fully loads, then, a moment later your popup appears. The default wait strategy in selenium is just that the page is loaded.
One way to handle this is to simply inspect the page and find the xpath of the popup window. The below code should work for that.
browser.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS)
if link == 'https://www.henleyglobal.com/':
browser.findElement(By.XPATH("/html/body/div[7]/div/div/div/div[2]/div/div[2]/button[2]")).click()
else:
browser.findElement(By.XPATH("/html/body/div[4]/div/div/div[2]/div[2]/a")).click()
The code is waiting until the element of the pop-up is clickable and then clicking it.
For unknown sites you could try:
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--disable-notifications")
webdriver.Chrome(os.path.join(path, 'chromedriver'), chrome_options=chrome_options)
generally, you can not use some universal locator that will match the "Accept cookies" buttons for each and every web site in the world.
Even here, you have 2 different sites and the elements you need to click are totally different on these sites.
For https://www.henleyglobal.com/ site the correct locator may be something like this CSS Selector .confirmation button.primary-btn while for https://markets.ft.com/data site I'd advise to use CSS Selector .o-cookie-message__actions a.o-cookie-message__button.
These 2 elements are totally different: the first one is button while the second is a, they have totally different class names and all other attributes.
You may thing about the Accept text. It seems to be common, so you could use this XPath //*[contains(text(),'Accept')] but even this will not work since on the first page it matches 2 elements while the accept cookies element is the second between them...
So, there is no General locators, you will have to define separate locators for each page.
Again, for https://www.henleyglobal.com/ I would prefer
driver.find_element(By.CSS_SELECTOR, ".confirmation button.primary-btn").click()
While for the second page https://markets.ft.com/data I would prefer this
driver.find_element(By.CSS_SELECTOR, ".o-cookie-message__actions a.o-cookie-message__button").click()
Also, generally we always use WebDriverWait expected_conditions explicit waits, so the code will be as following:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
# for the first page
wait.until(EC.element_to_be_clickable((By.XPATH, ".confirmation button.primary-btn"))).click()
# for the second page
wait.until(EC.element_to_be_clickable((By.XPATH, ".o-cookie-message__actions a.o-cookie-message__button"))).click()
I am still quite new to python and selenium, However have managed to get quite far with what I am doing. But I appear to now be stuck. The page in question is an internal business page. I have tried using ID, name and XPATH with very little success.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver import ActionChains
import time
PATH = r"C:\Users\p819364\Downloads\chromedriver.exe"
options = webdriver.ChromeOptions()
options.add_argument('--ignore-ssl-errors=yes')
options.add_argument('--ignore-certificate-errors')
driver = webdriver.Chrome(PATH, options=options)
driver.get("https://10.47.31.102/3/Login")
driver.implicitly_wait(15)
print (driver.title)
username = driver.find_element(By.NAME, value='j_username')
username.send_keys("username")
password = driver.find_element(By.NAME, value='j_password')
password.send_keys("password")
password.send_keys(Keys.RETURN)
driver.implicitly_wait(20)
Settings = driver.find_element(By.XPATH, value="//*[#id='evo_widget_TBFisheyeItem_4']")
Settings.click()
driver.implicitly_wait(20)
Filter = driver.find_element(By.XPATH, value="//*[#id='tableForm:authenticationPolicyTable:tableActions']")
driver.implicitly_wait(20)
Filter.click()
The problem I am having is with the filter, I think it may be because this is a page within a page. I am sorry I cannot share the page as its internal. But I need to be able to click the filter and click and options
I keep getting the following error
Message: no such element; Unable to locate element: {"method":"css selector","selector":["//*[#id='tableForm:authenticationPolicyTable:tableActions']"
The XPATH is as follows:
//*[#id="tableForm:authenticationPolicyTable:tableActions"]
This is the full XPATH (which I have tried):
/html/body/form[3]/table/thead/tr[1]/th/table/tbody/tr/td[2]/select
I appreciate any help given and I also am not sure if its caused this because the field I want to select is not in view until I scroll. However I cannot seem to scroll as the element is within another as pictured
Thanks
Edit:
The iFrame was stopping it i had to switch to it first
iframe = driver.find_element_by_xpath("//*[#id='consoleCanvas']")
driver.switch_to.frame(iframe)
I think if the page is within page then you need to check if that element is within iframe. If iframe is there then you should first switch to frame and then click on filter. If you can post html code then it will be easy to understand issue.
When I use mechanize, selenium libraries to run a url ("www.maps.google.com" in this case),
Chrome opens with a note saying that "Chrome is being controlled by an automated testing software".
But when I try to inspect element, there is only one element: body. All other inputs and buttons are not showing.
I want to automate the process to find distance between two addresses, so how do I solve the above problem?
from selenium import webdriver
driver = webdriver.Chrome(executable_path='C:/chromedriver.exe')
# Go to your page url
driver.get('https://www.google.com/maps')
# Get button you are going to click by its id ( also you could us find_element_by_css_selector to get element by css selector)
button_element = driver.find_element_by_id('searchbox-directions')
button_element.click()
dest_add = driver.find_element_by_class_name("tactile-searchbox-input")
dest_add.send_keys("Agra")
start_add = driver.find_element_by_class_name("tactile-searchbox-input")
start_add.send_keys("Jaipur")
For example this doesn't work. Since, there are no input fields so naturally, no elements with the class name "tactile-searchbox-input".
Induce WebDriverWait() and wait for element_to_be_clickable() and following css selector
driver.get('https://www.google.com/maps')
WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.ID,"searchbox-directions"))).click()
WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"input[aria-label*='starting point']"))).send_keys("Agra")
WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"input[aria-label*='destination']"))).send_keys("Jaipur")
You need to import below libraries.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
To click on search button try this.
WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.XPATH,"(//button[#aria-label='Search'])[last()]"))).click()
Hoping you can help. I'm relatively new to Python and Selenium. I'm trying to pull together a simple script that will automate news searching on various websites. The primary focus was football and to go and get me the latest Manchester United news from a couple of places and save the list of link titles and URLs for me. I could then look through the links myself and choose anything I wanted to review.
In trying the the independent newspaper (https://www.independent.co.uk/) I seem to have come up against a problem with element not interactable when using the following approaches:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome('chromedriver')
driver.get('https://www.independent.co.uk')
time.sleep(3)
#accept the cookies/privacy bit
OK = driver.find_element_by_id('qcCmpButtons')
OK.click()
#wait a few seconds, just in case
time.sleep(5)
search_toggle = driver.find_element_by_class_name('icon-search.dropdown-toggle')
search_toggle.click()
This throws the selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable error
I've also tried with XPATH
search_toggle = driver.find_element_by_xpath('//*[#id="quick-search-toggle"]')
and I also tried ID.
I did a lot of reading on here and then also tried using WebDriverWait and execute_script methods:
element = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, '//*[#id="quick-search-toggle"]')))
driver.execute_script("arguments[0].click();", element)
This didn't seem to error but the search box never appeared, i.e. the appropriate click didn't happen.
Any help you could give would be fantastic.
Thanks,
Pete
Your locator is //*[#id="quick-search-toggle"], there are 2 on the page. The first is invisible and the second is visible. By default selenium refers to the first element, sadly the element you mean is the second one, so you need another unique locator. Try this:
search_toggle = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//div[#class="row secondary"]//a[#id="quick-search-toggle"]')))
search_toggle.click()
First you need to open search box, then send search keys:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
import os
chrome_options = Options()
chrome_options.add_argument("--start-maximized")
browser = webdriver.Chrome(executable_path=os.path.abspath(os.getcwd()) + "/chromedriver", options=chrome_options)
link = 'https://www.independent.co.uk'
browser.get(link)
# accept privacy
button = browser.find_element_by_xpath('//*[#id="qcCmpButtons"]/button').click()
# open search box
li = browser.find_element_by_xpath('//*[#id="masthead"]/div[3]/nav[2]/ul/li[1]')
search_tab = li.find_element_by_tag_name('a').click()
# send keys to search box
search = browser.find_element_by_xpath('//*[#id="gsc-i-id1"]')
search.send_keys("python")
search.send_keys(Keys.RETURN)
Can you try with below steps
search_toggle = driver.find_element_by_xpath('//*[#class="row secondary"]/nav[2]/ul/li[1]/a')
search_toggle.click()
I'm attempting to automate some work with selenium and I'm having issues sending keys to the input id when attempting to login and I'm wondering if I'm in over my head.
I've tried locating the element by xpath, id as well as class, with no luck. I've also tried to wait to make sure the element is visible with no luck. Perhaps it's due to the div xmlnsis inside a body xmlns, resulting in the elements within the div xmlns initially not being visible? If so, how do I go about making the input box visible?
This is basically how far I've come:
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
# website url
url = "https://tx-production-eu-web.production.eu1.tx.klarna.net/radix/"
# driver
driver = webdriver.Chrome(r"C:\Users\ahmed.khairouni\Desktop\driver\chromedriver.exe")
driver.get(url)
# wait for page to load
wait = WebDriverWait(driver, 5)
#locate element and insert text into textbox
username = wait.until(EC.presence_of_element_located((By.ID, "wf_139389")))
search_input.send_keys('username')
Linked you can find part of the web code: https://imgur.com/WVKo7B7
Appreciate any help and thank you in advance.
The issue here might be the input is using a dynamic ID, so finding by ID may not work. I also noticed you are locating username, but then you are sending keys to search_input, which does not seem to exist in your code. That may be causing your issue, unless you made a typo in this question.
You could try using an XPath to locate the username input:
username = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//input[contains(#class, 'rwt-regular-value')]")))
username.send_keys('username')