I have been trying to locate a sidebar bar in this website https://covid19.who.int/?gclid=EAIaIQobChMIkoua643_6gIVDNd3Ch3qgApPEAAYASAAEgIfIvD_BwE
The search bar I am trying to locate is the one on top, which says "Search by Country, Territory, or Area"
I tried with XPATH but I always get error in locating the element, or sometimes it says that my element is not interactable (I guess that means that I am locating a different element)
Could please show a way to find that element?
here is what I saw when I inspected
Assuming that you want to search Country and see the data, So for that you need to locate the id of search box(i.e react-select-2-input) and enter the Country name using send_keys and hit enter.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
site = 'https://covid19.who.int/?gclid=EAIaIQobChMIkoua643_6gIVDNd3Ch3qgApPEAAYASAAEgIfIvD_BwE'
driver = webdriver.Chrome(executable_path = 'chromedriver.exe') # Here I am using Chrome's web driver
#For Firefox Web driver
#driver = webdriver.Firefox(executable_path = 'geckodriver.exe')
driver.get(site)
time.sleep(3)
elem = driver.find_element_by_id("react-select-2-input")
elem.send_keys("Brazil")
elem.send_keys(Keys.ENTER)
If you are only interested in locating search-bar then you can locate it by it's id(driver.find_element_by_id("react-select-2-input"))
How did I find search-bar id?
Basically search-bar is nothing but a text-box and in HTML text-box code starts with <input type="text">, So I had to just find the HTML code of text box in inspect element of browser.
By seeing your screen-shot I can say you were going in right direction, just you had to expand div tag more till you find code of text box.
Related
This question already has answers here:
Click on ember.js enabled element using Selenium
(5 answers)
Closed 1 year ago.
At this stage, all the script aims to do is to locate the search bar in a digital library, send the name of the resource I am looking for, click on it and get the current url. The first bit (sending keys to search bar) works fine. But I have noticed that the second try/finally block only executes when I right click and inspect any element on the page. If I don't, I get the following error message :
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="ember767"]"}
My code:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.action_chains import ActionChains
PATH = "/Applications/chromedriver"
ser = Service(PATH)
driver = webdriver.Chrome(service=ser)
driver.maximize_window()
driver.implicitly_wait(20)
driver.get("https://browzine.com/libraries/1374/subjects")
print("Enter targeted Journal name:")
targeted_journal = input()
wait = WebDriverWait(driver, 10)
try:
button = driver.find_element(By.ID, "ember648")
driver.implicitly_wait(10)
ActionChains(driver).move_to_element(button).click(button).perform()
button.send_keys(targeted_journal)
button.send_keys(Keys.RETURN)
finally:
pass
# step 2 : click targeted journal and get current url
try:
driver.implicitly_wait(20)
click_journal = driver.find_element(By.ID, "ember767")
ActionChains(driver).move_to_element(click_journal).click(click_journal).perform()
targeted_url = driver.current_url
finally:
pass
print(targeted_url)
driver.quit()
Update - answer found
The ID initially collected for the targeted element was a dynamic one. It was only as such when the developer tools window was open. Therefore the script was only able to detect that particular ID when "Inspect element" was clicked as it was the only case in which the targeted element was assigned this particular ID ("ember767" in my case).
To avoid this issue, an Xpath was used to locate the element instead. See Mayank Shukla's answer below.
In some cases, element's ID or other attributes changes or attribute's value appears on mouse hover. So it's preferred to use xpath for locating such elements
In this scenario, I think the element's id is changing after you are hovering the mouse. Try using xpath and let me know if that works. Because might be after browser refresh this is happening i.e. element's id is changing.
I am trying to access a dialog box on SharePoint using the Edge driver, however, my code doesn't find it. Using the Firefox driver, it works without problems:
[The reason I have to use Edge is because I am in a corporate environment, and Firefox is not supported by our single sign-on, proxies etc.]
from selenium import webdriver
import time
driver = webdriver.Edge()
#driver = webdriver.Firefox()
base_url = r"..../project/carb2_0/filedirectory/_layouts/15/groups.aspx"
driver.get(base_url)
time.sleep(1)
group = "AT01"
button= driver.find_element_by_xpath(r"//a[contains(text(),'File Directory - "+group+"')]")
button.click()
time.sleep(0.2)
button = driver.find_element_by_link_text("New")
button.click()
time.sleep(0.2)
driver.switch_to.frame(1)
time.sleep(0.2)
#t = driver.page_source
elem = driver.find_element_by_xpath(
r"//div[#id='ctl00_PlaceHolderMain_peoplePicker_TopSpan']/input[2]"
)
The last line throws an exception when using Edge,
NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"//div[#id='ctl00_PlaceHolderMain_peoplePicker_TopSpan']/input[2]"}
(Session info: MicrosoftEdge=85.0.564.41)
Firefox is perfectly fine. Digging deeper, reading the page source with driver.page_source yields two completely different results. For Edge it is
<html><head></head><body></body></html>
For Firefox I get a full page, including an iframe, which can be found by find_element_by_xpath.
Any help would be appreciated.
The solution to the above issue was 2-fold:
switch to the correct frame
find the exact xpath denoting the <input> field in the page
For the first part I used the following code:
for i, frame in enumerate(frames):
if re.search('ms-dlgFrame', frame.get_attribute('outerHTML')):
frame_id = i
break
driver.switch_to.frame(frame_id)
It turns out the the correct frame_id was 2, not 1. It is not clear to me why the firefox (gecko) driver would work anyways (and the selenium IDE would select 1).
The second part boiled down to searching through the source of the HTML page and find the exact input element. Interestingly enough, the selenium IDE had only located a <div> around that element, and the firefox webdriver was forgiving enough to type into the correct field. The edge driver would not do that and claim that element was not writable. Once I had found the correct <input> field, everything worked fine.
i'm trying to click the "download Results" button on this website .
i'm using below python code to click this button
from selenium import webdriver
chromedriver_path = 'E:/software/python/chromedriver'
url = 'https://www.cms.gov/apps/physician-fee-schedule/search/search-results.aspx?Y=0&T=4&HT=2&CT=3&H1=74750&H2=74800&M=5'
driver= webdriver.Chrome(executable_path=chromedriver_path )
driver.get(url)
driver.find_element_by_xpath('//*[#title="Download Results"]').click()
i'm getting below error
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//frame[#name="aspnetForm"]"}
(Session info: headless chrome=83.0.4103.116)
i'm thinking if the button is within a iframe but how do I find out the iframe?
This may help you out: Unable to locate element using selenium webdriver in python
Try and switch to the frame first, and then search for the element.
i realized that there's a agreement page that i need to click 'Agree' button first.
i didn't see this in browser because i already clicked 'Agree' weeks ago. but in webdriver, i need to click that each time.
Just after the URL opens, a page comes up asking you to agree the terms. You need to click on that Agree first. Since it is possible that once clicking on the Agree button, it won't come again if you accepted it on default browser. So just be sure with code, you can first check the presence of the agree button first.
This kind of function you may make to check presence:
def elementPresent(locatorType, locator):
#present = true
#not present = false
wait = WebDriverWait(driver, 20)
try:
wait.until(EC.presence_of_element_located((locatorType, locator)))
except Exception:
return False
return True
And then using if condition you may proceed further like:
if(elementPresent("xpath", "//a[#title='Accept']/span")):
driver.find_element_by_xpath("//a[#title='Accept']/span").click()
and then you may click on the element you want, there is no frame that is required to be switched to.
driver.find_element_by_xpath("//a[#title = 'Download Results']/span").click()
Just extract the element and click on it and your required file will be downloaded.
I am trying to scrape some information off this website using python's selenium.
First, I log into the website and get to the page. Then, I would like to click on the tab "Quickscan" to scrape some info. However, that's where I get stuck. I can't find a way to click on the tab.
Note that this problem would be surmounted if I managed to navigate to the page, though when I log in, even if I put such page in my WebDriver, I still get redirected to this one.
To get to the desired page I have tried finding the element both through the xpath and through the link, but it does not find the element.
import requests
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
driver =webdriver.Chrome(executable_path ="mypath")
driver.get("https://vc4a.com/ventures/autocollect/#quickscan-tab")
#find username and password bar
username = driver.find_element_by_id("user_login")
password = driver.find_element_by_id("user_pass")
#Input password and username
username.send_keys("username")
password.send_keys("password")
#click on submit
driver.find_element_by_name("wp-submit").click()
driver.find_element_by_name("rememberme").click()
#try to find element using text in the link
driver.find_elements_by_link_text('#quickscan-tab')[0].click()
#try to find element using xpath from the inspected element
driver.find_element_by_xpath('//*[#id="subnav"]/li[3]/a').click()
I would like to be able to open the tab so that I can scrape the content.
When I use the first code it returns the following error:
IndexError: list index out of range
However, by inspecting the page I can see there is indeed 2 elements with the text "#quickscan-tab", so I don't understand why the index 0 would be out of range.
When I use the second code it returns the following error:
NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id="subnav"]/li[3]/a"}
(Session info: chrome=74.0.3729.169)
(Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729#{#29}),platform=Mac OS X 10.14.5 x86_64)
What I did was just copying the xpath.
I created an account on that page and tried this modified script and it works:
import requests
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
driver = webdriver.Chrome()
driver.get("https://vc4a.com/ventures/autocollect/#quickscan-tab")
#find username and password bar
username = driver.find_element_by_id("user_login")
password = driver.find_element_by_id("user_pass")
#Input password and username
username.send_keys("username")
password.send_keys("password")
#click on submit
driver.find_element_by_name("rememberme").click()
driver.find_element_by_name("wp-submit").click()
time.sleep(10)
#try to find element using text in the link
driver.find_elements_by_link_text('Quickscan')[0].click()
#try to find element using xpath from the inspected element
driver.find_element_by_xpath('//a[text()="Quickscan"]').click()
link_text means the text that you actually see. [Quickscan]
Login takes time and the script tries to locate before the tab is created thus causing error.
Your xpath would have worked if not for the login delay.
Click rememberme before submitting the form. Or don't, since selenium starts a clean session for every run.
driver.find_elements_by_link_text('#quickscan-tab')[0].click() - it's wrong
Link text doesn't work like this you need to create a different locator. try below XPath
driver.find_element_by_xpath((//*[#id='quickscan-tab'])[0])
Try this:
scanelements = driver.find_elements_by_xpath('//*[#id='quickscan-tab']')
for elt in scanelements :
elt.click()
break
I am trying to automate reading my phone bill from the carrier website. www.fido.ca
However, the site is built with angularjs and I can't find the element using python and selenium webdriver. Please see below for the codes I've tried.
driver = webdriver.Firefox()
url = 'https://www.fido.ca/pages/#/login?m=login'
driver.get(url)
wait = WebDriverWait(driver, 10)
wait.until(EC.visibility_of_element_located((By.XPATH, "//a[#id='BC']")))
It returns selenium.common.exceptions.TimeoutException: Message:
Note: I can see the element from the front end, but no idea why webdriver can't see it.
When you navigate to a page, you would see the overlay "Welcome to Fido!" screen which makes your desired element invisible - hence the timeout error.
Handle the screen by selecting a region and clicking "Continue" or "X" (close).