I'm attempting to automate some work with selenium and I'm having issues sending keys to the input id when attempting to login and I'm wondering if I'm in over my head.
I've tried locating the element by xpath, id as well as class, with no luck. I've also tried to wait to make sure the element is visible with no luck. Perhaps it's due to the div xmlnsis inside a body xmlns, resulting in the elements within the div xmlns initially not being visible? If so, how do I go about making the input box visible?
This is basically how far I've come:
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
# website url
url = "https://tx-production-eu-web.production.eu1.tx.klarna.net/radix/"
# driver
driver = webdriver.Chrome(r"C:\Users\ahmed.khairouni\Desktop\driver\chromedriver.exe")
driver.get(url)
# wait for page to load
wait = WebDriverWait(driver, 5)
#locate element and insert text into textbox
username = wait.until(EC.presence_of_element_located((By.ID, "wf_139389")))
search_input.send_keys('username')
Linked you can find part of the web code: https://imgur.com/WVKo7B7
Appreciate any help and thank you in advance.
The issue here might be the input is using a dynamic ID, so finding by ID may not work. I also noticed you are locating username, but then you are sending keys to search_input, which does not seem to exist in your code. That may be causing your issue, unless you made a typo in this question.
You could try using an XPath to locate the username input:
username = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//input[contains(#class, 'rwt-regular-value')]")))
username.send_keys('username')
Related
I am still quite new to python and selenium, However have managed to get quite far with what I am doing. But I appear to now be stuck. The page in question is an internal business page. I have tried using ID, name and XPATH with very little success.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver import ActionChains
import time
PATH = r"C:\Users\p819364\Downloads\chromedriver.exe"
options = webdriver.ChromeOptions()
options.add_argument('--ignore-ssl-errors=yes')
options.add_argument('--ignore-certificate-errors')
driver = webdriver.Chrome(PATH, options=options)
driver.get("https://10.47.31.102/3/Login")
driver.implicitly_wait(15)
print (driver.title)
username = driver.find_element(By.NAME, value='j_username')
username.send_keys("username")
password = driver.find_element(By.NAME, value='j_password')
password.send_keys("password")
password.send_keys(Keys.RETURN)
driver.implicitly_wait(20)
Settings = driver.find_element(By.XPATH, value="//*[#id='evo_widget_TBFisheyeItem_4']")
Settings.click()
driver.implicitly_wait(20)
Filter = driver.find_element(By.XPATH, value="//*[#id='tableForm:authenticationPolicyTable:tableActions']")
driver.implicitly_wait(20)
Filter.click()
The problem I am having is with the filter, I think it may be because this is a page within a page. I am sorry I cannot share the page as its internal. But I need to be able to click the filter and click and options
I keep getting the following error
Message: no such element; Unable to locate element: {"method":"css selector","selector":["//*[#id='tableForm:authenticationPolicyTable:tableActions']"
The XPATH is as follows:
//*[#id="tableForm:authenticationPolicyTable:tableActions"]
This is the full XPATH (which I have tried):
/html/body/form[3]/table/thead/tr[1]/th/table/tbody/tr/td[2]/select
I appreciate any help given and I also am not sure if its caused this because the field I want to select is not in view until I scroll. However I cannot seem to scroll as the element is within another as pictured
Thanks
Edit:
The iFrame was stopping it i had to switch to it first
iframe = driver.find_element_by_xpath("//*[#id='consoleCanvas']")
driver.switch_to.frame(iframe)
I think if the page is within page then you need to check if that element is within iframe. If iframe is there then you should first switch to frame and then click on filter. If you can post html code then it will be easy to understand issue.
I have a question about Selenium.
My idea:
My idea is to make a Python script that logs in to this website.
Selenium sends the username and password to the HTML input field and submits it.
Problem:
My code keeps saying:
Message: no such element: Unable to locate element:
I have tried this code with google.com for example and that works.
Why is this not working with this login page?
Can anybody help me please?
My Python code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
login_URL = ""
driver = webdriver.Chrome()
driver.get(login_URL)
time.sleep(5)
inputElement = driver.find_element_by_name('uname')
inputElement.send_keys(username)
time.sleep(20)
driver.close()
i don't know how it works in pyton, im use js
but try to use xpath
driver.find_element_by_xpath('your xpath') [maybe use 1 more click) ('xpath element').click() - element is active.
and after use send keys
******.send_keys('username')
driver.switchTo().frame(driver.findElement(By.xpath('xpath frame'))) - in js
As already explained, the element is in an iframe. Need to switch to frame to interact with the element.
It would be better apply Explicit waits.
# Imports required:
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver.get("https://nl.infothek-sptk.com/isps/infothek/?1043")
wait = WebDriverWait(driver,30)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.NAME,"body_frame")))
wait.until(EC.element_to_be_clickable((By.NAME,"uname"))).send_keys("username#mail.com")
# Code to enter other fields.
# Switch back to default to interact with elements outside the iframe.
driver.switch_to.default_content()
I am having a terribly hard time referencing to a certain "next page" button on a website that I am trying to scrape links from [https://www.sreality.cz/adresar?strana=2]. If you scroll down you can see a red right arrow button that you can click to go to the next page and so the website load new dynamic content. Every approach seems to report the same exact error and I don't know how am I supposed to point to the element without running into it.
This is the code that I currently have :
from selenium import webdriver
chromedriver_path = "/home/user/Dokumenty/iCloud/RealityScraper/chromedriver"
driver = webdriver.Chrome(chromedriver_path)
print("WebDriver Successfully Initialized")
driver.get("https://www.sreality.cz/adresar?strana=2")
links = driver.find_elements_by_css_selector("h2.title a")
nextPage = driver.find_element_by_css_selector("li.paging-item a.btn-paging-pn.icof.icon-arr-right.paging-next")
for link in links:
print(link.get_attribute("href"))
nextPage.click()
The "nextPage" variable is holding a supposed value to be clicked on once the "links" variable search finishes scraping all the links from the company titles. However when I run this code I get an error :
selenium.common.exceptions.StaleElementReferenceException: Message:
stale element reference: element is not attached to the page document
I have been searching for various fixes online but none of them seemed to resolve the issue. I think that the issue at this point is not caused by the element not loading quickly enough but rather Selenium having trouble finding the element because of wrong reference.
Because of this I have tried using XPath to accurately point to the actual element and so I changed the "nextPage" variable to :
nextPage = driver.find_element_by_xpath("""/html/body/div[2]/div[1]/div[2]/div[2]/div[4]/div/div/div/div[2]/div/div[2]/ul[1]/li[12]/a""")
Which returns exactly the same error as stated above. I have been trying to find a solution to this for hours now and I can't understand where the issue lies. I would be grateful if anyone could explain to me what am I doing wrong. Thanks to anyone.
If you want to get all the ng-href tags from every page. Or you could look into their api.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from time import sleep
driver.get("https://www.sreality.cz/adresar?strana=2")
wait = WebDriverWait(driver, 10)
while True:
try:
links = wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "h2.title > a")))
#print(len(links))
for link in links:
print(link.get_attribute("ng-href"))
nextPage = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.btn-paging-pn.icof.icon-arr-right.paging-next")))
nextPage.click()
time.sleep(10)
except Exception as e:
print(e)
break
First of all never use the absolute xpath it will breakdown easily, Use the relative xpath.
Secondly, i think the error you are getting is because after clicking "Next" button for the first time it loads a new page. Which has a different DOM structure and that's why you are not able to find that element.
You can try searching for the element after every new page load (after clicking "Next" button everytime.)
// imports
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver import ActionChains
from selenium.webdriver.common.by import By
// initialize
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 20)
action = ActionChains(driver)
// Try to use the below code and see if it works.
Next_btn = wait.until(EC.presence_of_element_located((By.XPATH, '(//li[#class="paging-item"])[2]')))
action.move_to_element(Next_btn).click().perform()
I am wondering if someone can point me in the right direction to get an address search to work. For reference, I am trying to enter an address in the Address Search input box and click the "Search" button on this page: https://beacon.schneidercorp.com/Application.aspx?AppID=44&LayerID=260&PageTypeID=2&PageID=297. A valid address is: "1770 Mobile Ave"
This page may or may not load for you, I think it might be using some kind of token.
Anyhow, I have tried WebDriverWait with xpath, find_element_by_css_selector, find_element_by_name, etc. to access the input field and then use send_keys but nothing seems to allow me to get past an "ElementNotInteractableException".
My working code starts at the home page and everything here works fine to get to the search page in the picture:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
options = webdriver.ChromeOptions()
options.headless = True
driver = webdriver.Chrome(options=options)
url = "https://beacon.schneidercorp.com/"
driver.get(url)
select_st = Select(driver.find_element_by_id('stateMenu'))
select_st.select_by_visible_text('Iowa')
select_st = Select(driver.find_element_by_id('areaMenu'))
select_st.select_by_visible_text('Harrison County, IA')
parentElement = driver.find_element_by_class_name('list-group')
elementList = parentElement.find_elements_by_tag_name("a")
elementList[0].click()
You can try to skip the above and start at the search page - I only mention it in case the search page won't load.
Again, some things I have tried to fill out the input include:
address = driver.find_element_by_css_selector('input[placeholder="enter address..."]')
address.send_keys("1770 Mobile Ave")
address = driver.find_element_by_name("ctlBodyPane$ctl01$ctl01$txtAddress")
address.send_keys("1770 Mobile Ave")
address = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.ID, 'ctlBodyPane_ctl01_ctl01_txtAddress')))
address.send_keys("1770 Mobile Ave")
Am I missing something? It seems like the answer here is obvious but I'm overlooking it.
I have noticed once move to next page from first page one pop up is appeared for terms & condition and you need to click on that first.
Induce WebDriverWait() and wait for element_to_be_clickable()
#click on terms & condition page
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//a[text()='Agree']"))).click()
#Enter data in address field
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.ID,"ctlBodyPane_ctl01_ctl01_txtAddress"))).send_keys("1770 Mobile Ave")
You need to import below libraries.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Let me know how this goes?
It worked for me. Using the method find_element_by_xpath(). Other selector that you mentioned was for other elements or incorrect. for example this ctlBodyPane_ctl01_ctl01_txtAddress points the outer element of the search box which is obviously not interactable. Using input[placeholder="enter address..."] this selector you were close but this returns 2 matching elements. To fix this I converted this to xpath and wrapped it for index to return unique matching element.
Here is the part of code fix -
address = driver.find_element_by_xpath('(.//input[#placeholder="enter address..."])[1]')
address.send_keys("1770 Mobile Ave")
Hoping you can help. I'm relatively new to Python and Selenium. I'm trying to pull together a simple script that will automate news searching on various websites. The primary focus was football and to go and get me the latest Manchester United news from a couple of places and save the list of link titles and URLs for me. I could then look through the links myself and choose anything I wanted to review.
In trying the the independent newspaper (https://www.independent.co.uk/) I seem to have come up against a problem with element not interactable when using the following approaches:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome('chromedriver')
driver.get('https://www.independent.co.uk')
time.sleep(3)
#accept the cookies/privacy bit
OK = driver.find_element_by_id('qcCmpButtons')
OK.click()
#wait a few seconds, just in case
time.sleep(5)
search_toggle = driver.find_element_by_class_name('icon-search.dropdown-toggle')
search_toggle.click()
This throws the selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable error
I've also tried with XPATH
search_toggle = driver.find_element_by_xpath('//*[#id="quick-search-toggle"]')
and I also tried ID.
I did a lot of reading on here and then also tried using WebDriverWait and execute_script methods:
element = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, '//*[#id="quick-search-toggle"]')))
driver.execute_script("arguments[0].click();", element)
This didn't seem to error but the search box never appeared, i.e. the appropriate click didn't happen.
Any help you could give would be fantastic.
Thanks,
Pete
Your locator is //*[#id="quick-search-toggle"], there are 2 on the page. The first is invisible and the second is visible. By default selenium refers to the first element, sadly the element you mean is the second one, so you need another unique locator. Try this:
search_toggle = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//div[#class="row secondary"]//a[#id="quick-search-toggle"]')))
search_toggle.click()
First you need to open search box, then send search keys:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
import os
chrome_options = Options()
chrome_options.add_argument("--start-maximized")
browser = webdriver.Chrome(executable_path=os.path.abspath(os.getcwd()) + "/chromedriver", options=chrome_options)
link = 'https://www.independent.co.uk'
browser.get(link)
# accept privacy
button = browser.find_element_by_xpath('//*[#id="qcCmpButtons"]/button').click()
# open search box
li = browser.find_element_by_xpath('//*[#id="masthead"]/div[3]/nav[2]/ul/li[1]')
search_tab = li.find_element_by_tag_name('a').click()
# send keys to search box
search = browser.find_element_by_xpath('//*[#id="gsc-i-id1"]')
search.send_keys("python")
search.send_keys(Keys.RETURN)
Can you try with below steps
search_toggle = driver.find_element_by_xpath('//*[#class="row secondary"]/nav[2]/ul/li[1]/a')
search_toggle.click()