I am trying to access a dialog box on SharePoint using the Edge driver, however, my code doesn't find it. Using the Firefox driver, it works without problems:
[The reason I have to use Edge is because I am in a corporate environment, and Firefox is not supported by our single sign-on, proxies etc.]
from selenium import webdriver
import time
driver = webdriver.Edge()
#driver = webdriver.Firefox()
base_url = r"..../project/carb2_0/filedirectory/_layouts/15/groups.aspx"
driver.get(base_url)
time.sleep(1)
group = "AT01"
button= driver.find_element_by_xpath(r"//a[contains(text(),'File Directory - "+group+"')]")
button.click()
time.sleep(0.2)
button = driver.find_element_by_link_text("New")
button.click()
time.sleep(0.2)
driver.switch_to.frame(1)
time.sleep(0.2)
#t = driver.page_source
elem = driver.find_element_by_xpath(
r"//div[#id='ctl00_PlaceHolderMain_peoplePicker_TopSpan']/input[2]"
)
The last line throws an exception when using Edge,
NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"//div[#id='ctl00_PlaceHolderMain_peoplePicker_TopSpan']/input[2]"}
(Session info: MicrosoftEdge=85.0.564.41)
Firefox is perfectly fine. Digging deeper, reading the page source with driver.page_source yields two completely different results. For Edge it is
<html><head></head><body></body></html>
For Firefox I get a full page, including an iframe, which can be found by find_element_by_xpath.
Any help would be appreciated.
The solution to the above issue was 2-fold:
switch to the correct frame
find the exact xpath denoting the <input> field in the page
For the first part I used the following code:
for i, frame in enumerate(frames):
if re.search('ms-dlgFrame', frame.get_attribute('outerHTML')):
frame_id = i
break
driver.switch_to.frame(frame_id)
It turns out the the correct frame_id was 2, not 1. It is not clear to me why the firefox (gecko) driver would work anyways (and the selenium IDE would select 1).
The second part boiled down to searching through the source of the HTML page and find the exact input element. Interestingly enough, the selenium IDE had only located a <div> around that element, and the firefox webdriver was forgiving enough to type into the correct field. The edge driver would not do that and claim that element was not writable. Once I had found the correct <input> field, everything worked fine.
Related
I have been trying to locate a sidebar bar in this website https://covid19.who.int/?gclid=EAIaIQobChMIkoua643_6gIVDNd3Ch3qgApPEAAYASAAEgIfIvD_BwE
The search bar I am trying to locate is the one on top, which says "Search by Country, Territory, or Area"
I tried with XPATH but I always get error in locating the element, or sometimes it says that my element is not interactable (I guess that means that I am locating a different element)
Could please show a way to find that element?
here is what I saw when I inspected
Assuming that you want to search Country and see the data, So for that you need to locate the id of search box(i.e react-select-2-input) and enter the Country name using send_keys and hit enter.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
site = 'https://covid19.who.int/?gclid=EAIaIQobChMIkoua643_6gIVDNd3Ch3qgApPEAAYASAAEgIfIvD_BwE'
driver = webdriver.Chrome(executable_path = 'chromedriver.exe') # Here I am using Chrome's web driver
#For Firefox Web driver
#driver = webdriver.Firefox(executable_path = 'geckodriver.exe')
driver.get(site)
time.sleep(3)
elem = driver.find_element_by_id("react-select-2-input")
elem.send_keys("Brazil")
elem.send_keys(Keys.ENTER)
If you are only interested in locating search-bar then you can locate it by it's id(driver.find_element_by_id("react-select-2-input"))
How did I find search-bar id?
Basically search-bar is nothing but a text-box and in HTML text-box code starts with <input type="text">, So I had to just find the HTML code of text box in inspect element of browser.
By seeing your screen-shot I can say you were going in right direction, just you had to expand div tag more till you find code of text box.
I'm trying to automate the process of uploading contacts to my iCloud account.
But, I'm unable to get Selenium to find the Show Actions Menu button which is located at the bottom left.
I know the button is loaded within a frame, so I switched to the frame but still couldn't get it to find the button.
Here's the full code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
chrome_driver_path = '/chromedriver.exe'
chrome_options = Options()
chrome_options.add_argument("user-data-dir=selenium")
chrome_options.add_argument("--disable-features=NetworkService")
chrome_options.add_argument("--disable-csp")
chrome_options.add_argument("--log-level=3")
driver = webdriver.Chrome(
executable_path=chrome_driver_path, options=chrome_options
)
url = "https://www.icloud.com/contacts/"
with driver:
# Set timeout time
wait = WebDriverWait(driver, 10)
driver.get(url)
driver.maximize_window()
time.sleep(10)
#wait.until(presence_of_element_located((By.XPATH, '//*[#id="sc1737"]')))
driver.switch_to_frame(driver.find_element_by_id('contacts'))
#Tried locating the frame by xpath too, but didn't work
#driver.switch_to_frame(driver.find_element_by_xpath('//*[#id="contacts"]'))
driver.find_element_by_id('sc1737').click()
Could anyone please help me figure out what am I doing wrong?
Extra steps I tried as debugging:
Using pdb, I tried locating an element within the frame (the whole left side of the iCloud page) before switching to the frame, but I got the same error:
Unable to locate element
So I tried switching to the frame, then locate the same element, and it was found with no problem, which proves that switching to the frame worked.
Finally, tried to locate the button I'm after, but still gave me the same error.
I am trying to scrape some information off this website using python's selenium.
First, I log into the website and get to the page. Then, I would like to click on the tab "Quickscan" to scrape some info. However, that's where I get stuck. I can't find a way to click on the tab.
Note that this problem would be surmounted if I managed to navigate to the page, though when I log in, even if I put such page in my WebDriver, I still get redirected to this one.
To get to the desired page I have tried finding the element both through the xpath and through the link, but it does not find the element.
import requests
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
driver =webdriver.Chrome(executable_path ="mypath")
driver.get("https://vc4a.com/ventures/autocollect/#quickscan-tab")
#find username and password bar
username = driver.find_element_by_id("user_login")
password = driver.find_element_by_id("user_pass")
#Input password and username
username.send_keys("username")
password.send_keys("password")
#click on submit
driver.find_element_by_name("wp-submit").click()
driver.find_element_by_name("rememberme").click()
#try to find element using text in the link
driver.find_elements_by_link_text('#quickscan-tab')[0].click()
#try to find element using xpath from the inspected element
driver.find_element_by_xpath('//*[#id="subnav"]/li[3]/a').click()
I would like to be able to open the tab so that I can scrape the content.
When I use the first code it returns the following error:
IndexError: list index out of range
However, by inspecting the page I can see there is indeed 2 elements with the text "#quickscan-tab", so I don't understand why the index 0 would be out of range.
When I use the second code it returns the following error:
NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id="subnav"]/li[3]/a"}
(Session info: chrome=74.0.3729.169)
(Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729#{#29}),platform=Mac OS X 10.14.5 x86_64)
What I did was just copying the xpath.
I created an account on that page and tried this modified script and it works:
import requests
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
driver = webdriver.Chrome()
driver.get("https://vc4a.com/ventures/autocollect/#quickscan-tab")
#find username and password bar
username = driver.find_element_by_id("user_login")
password = driver.find_element_by_id("user_pass")
#Input password and username
username.send_keys("username")
password.send_keys("password")
#click on submit
driver.find_element_by_name("rememberme").click()
driver.find_element_by_name("wp-submit").click()
time.sleep(10)
#try to find element using text in the link
driver.find_elements_by_link_text('Quickscan')[0].click()
#try to find element using xpath from the inspected element
driver.find_element_by_xpath('//a[text()="Quickscan"]').click()
link_text means the text that you actually see. [Quickscan]
Login takes time and the script tries to locate before the tab is created thus causing error.
Your xpath would have worked if not for the login delay.
Click rememberme before submitting the form. Or don't, since selenium starts a clean session for every run.
driver.find_elements_by_link_text('#quickscan-tab')[0].click() - it's wrong
Link text doesn't work like this you need to create a different locator. try below XPath
driver.find_element_by_xpath((//*[#id='quickscan-tab'])[0])
Try this:
scanelements = driver.find_elements_by_xpath('//*[#id='quickscan-tab']')
for elt in scanelements :
elt.click()
break
I have a problem with selenium chromedriver which I cannot figure out what's causing it. Some weeks ago everything was working OK, and suddenly this error started to show up.
The problem is coming from the following function.
def login_(browser):
try:
browser.get("some_url")
# user credentials
user = browser.find_element_by_xpath('//*[#id="username"]')
user.send_keys(config('user'))
password = browser.find_element_by_xpath('//*[#id="password"]')
password.send_keys(config('pass'))
login = browser.find_element_by_xpath('/html/body/div[1]/div/button')
login.send_keys("\n")
time.sleep(1)
sidebar = browser.find_element_by_xpath('//*[#id="sidebar"]/ul/li[1]/a')
sidebar.send_keys("\n")
app_submit = browser.find_element_by_xpath('//*[#id="sidebar"]/ul/li[1]/ul/li[1]/a')
app_submit.send_keys("\n")
except TimeoutException or NoSuchElementException:
raise LoginException
This function works with no problem in the development environment (macOS 10.11), but throws the following error in the production environment:
Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id="sidebar"]/ul/li[1]/a"}
(Session info: headless chrome=67.0.3396.79)
(Driver info: chromedriver=2.40.565383 (76257d1ab79276b2d53ee97XXX),platform=Linux 4.4.0-116-generic x86_64)
I already updated both Chrome and chromedriver (v67 & 2.40, respectively) in each environment. I also gave it more time.sleep(15). But the problem persists. My latest guess is that maybe the initialization of the webdriver is not working properly:
def initiate_webdriver():
option = webdriver.ChromeOptions()
option.binary_location = config('GOOGLE_CHROME_BIN')
option.add_argument('--disable-gpu')
option.add_argument('window-size=1600,900')
option.add_argument('--no-sandbox')
if not config('DEBUG', cast=bool):
display = Display(visible=0, size=(1600, 900))
display.start()
option.add_argument("--headless")
else:
option.add_argument("--incognito")
return webdriver.Chrome(executable_path=config('CHROMEDRIVER_PATH'), chrome_options=option)
Because, if the Display is not working, then there may not be the mentioned sidebar but some other button.
So my questions are: does anybody have had a similar issue? Is there a way to know what is the page showing at the time the driver is looking for such an element?
It's report that the element not found error after you supplying the login , so I think the login failed and the page redirected to somewhere. You can use screenshot option to take a screenshot of the page and then see which page the driver load.
driver.save_screenshot("path to save screen.jpeg")
Also you can save the raw html code and inspect the same page.
Webdriver Screenshot
Using Selenium in Python to save a webpage on Firefox
A couple of things as per the login_(browser) method:
As you have identified the Login button through:
login = browser.find_element_by_xpath('/html/body/div[1]/div/button')
I would suggest rather invoking send_keys("\n") take help of the onclick() event through login.click() to mock the clicking of Login button as follows:
login = browser.find_element_by_xpath('/html/body/div[1]/div/button')
login.click()
Next when you identify the sidebar induce WebDriverWait for the element to be clickable as follows:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="sidebar"]/ul/li[1]/a'))).click()
As you mentioned your code code block works perfect in macOS 10.11 environment but throws the following error in the production environment (Linux) it is highly possible that different browsers renders the HTML DOM differently in different OS architecture. So instead of absolute xpath you must use relative xpath as follows:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[#attribute='value']"))).click()
A couple of things as per the initiate_webdriver() method:
As per Getting Started with Headless Chrome the argument --disable-gpu is applicable only for Windows but not a valid configuration for Linux OS. So need o remove:
option.add_argument('--disable-gpu')
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Whenever I encounter strange issues in Selenium like this, I prefer retrying to find the particular element which is causing intermittent troubles. One way is to wrap it around a try-except block:
try:
sidebar = browser.find_element_by_xpath('//*[#id="sidebar"]/ul/li[1]/a')
except NoSuchElementException:
time.sleep(10)
print("Unable to find element in first time, trying it again")
sidebar = browser.find_element_by_xpath('//*[#id="sidebar"]/ul/li[1]/a')
You could also put the try code in a loop with a suitable count variable to make the automation code work. (Check this). In my experience with JAVA, this idea has resolved multiple issues.
You need to wait until the element is visible or else you will get this error. Try something like this:
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support.expected_conditions import visibility_of_element_located
from selenium.webdriver.common.by import By
TIMEOUT = 5
...
xpath = '//*[#id="sidebar"]/ul/li[1]/a'
WebDriverWait(self.selenium, TIMEOUT).until(visibility_of_element_located((By.XPATH, xpath)))
browser.find_element_by_xpath(xpath)
...
I am trying to automate reading my phone bill from the carrier website. www.fido.ca
However, the site is built with angularjs and I can't find the element using python and selenium webdriver. Please see below for the codes I've tried.
driver = webdriver.Firefox()
url = 'https://www.fido.ca/pages/#/login?m=login'
driver.get(url)
wait = WebDriverWait(driver, 10)
wait.until(EC.visibility_of_element_located((By.XPATH, "//a[#id='BC']")))
It returns selenium.common.exceptions.TimeoutException: Message:
Note: I can see the element from the front end, but no idea why webdriver can't see it.
When you navigate to a page, you would see the overlay "Welcome to Fido!" screen which makes your desired element invisible - hence the timeout error.
Handle the screen by selecting a region and clicking "Continue" or "X" (close).