Go to google.com
Type a search keyword
I want to select 3rd/4th value from the auto suggestions list. What method should I use in selenium python ?
I don't know python, but i do have code in C# which i was able to succeed. You can give it try.
IWebDriver driver = new InternetExplorerDriver();
driver.Navigate().GoToUrl("https://www.google.com/");
IWebElement txtboxSearch = driver.FindElement(By.Id("lst-ib"));
txtboxSearch.SendKeys("ap");
IList<IWebElement> autosaerchList = driver.FindElements(By.CssSelector(".sbsb_c.gsfs"));
autosaerchList[1].Click();
Google search page contains a <div class="gstl_0 sbdd_a">
When you start typing into the search box, that div becomes populated with an <ul role="listbox">. The <li> in that list contain the 4 suggestions. Pick one, and call the .click() method.
from selenium import webdriver
import time
driver = webdriver.Chrome('Path to chromedriver\chromedriver.exe')
driver.get('http://google.com')
driver.maximize_window()
driver.find_element_by_name('q').send_keys('Shah') #pass whatever you want to search
time.sleep(5)
# to click on third element of search suggestion
driver.find_element_by_xpath('//div/div[3]/form/div[2]/div[2]/div[1]/div[2]/div[2]/div[1]/div/ul/li[3]/div/div[2]').click()
# to click on fourth element of search suggestion, uncomment the next line and comment the previous one
#driver.find_element_by_xpath('//div/div[3]/form/div[2]/div[2]/div[1]/div[2]/div[2]/div[1]/div/ul/li[4]/div/div[2]').click()
Hope this help
Related
I'm very new and learning web scraping in python by trying to get the search results from the website below after a user types in some information, and then print the results. Everything works great up until the very last 2 lines of this script. When I include them in the script, nothing happens. However, when I remove them and then just try typing them into the shell after the script is done running, they work exactly as I'd intended. Can you think of a reason this is happening? As I'm a beginner I'm also super open if you see a much easier solution. All feedback is welcome. Thank you!
#Setup
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
import time
#Open Chrome
driver = webdriver.Chrome()
driver.get("https://myutilities.seattle.gov/eportal/#/accountlookup/calendar")
#Wait for site to load
time.sleep(10)
#Click on street address search box
elem = driver.find_element(By.ID, 'sa')
elem.click()
#Get input from the user
addr = input('Please enter part of your address and press enter to search.\n')
#Enter user input into search box
elem.send_keys(addr)
#Get search results
elem = driver.find_element(By.XPATH, ('/html/body/app-root/main/div/div/account-lookup/div/section/div[2]/div[2]/div/div/form/div/ul/li/div/div[1]'))
print(elem.text)
I haven't used Selenium in a while, so I can only point you in the right direction. It seems to me you need to iterate over the individual entries, and print those, as opposed to printing the entire div as one element.
You should remove the parentheses from the xpath expression
You can shorten the xpath expression as follows:
Code:
elems = driver.find_element(By.XPATH, '//*[#class="addressResults"]/div')
for elem in elems:
print(elem.text)
You are using an absolute XPATH, what you should be looking into are relative XPATHs
Something like this should do it:
elems = driver.find_elements(By.XPATH, ("//*[#id='addressResults']/div"))
for elem in elems:
...
I ended up figuring out my problem - I just needed to add in a bit that waits until the search results actually load before proceeding on with the script. tossing in a time.sleep(5) did the trick. Eventually I'll add a bit that checks that an element has loaded before proceeding with the script, but this lets me continue for now. Thanks everyone for your answers!
I'm kinda of a newbie in Selenium, started learning it for my job some time ago. Right now I'm working with a code that will open the browser, enter the specified website, put the products ID in the search box, search, and them open it. Once it opens the product, it needs to extract its name and price and write it in a CSV file. I'm kinda struggling with it a bit.
The main problem right now is that Selenium is unable to open the product after searching it. I've tried by ID, name and class and it still didn't work.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import csv
driver = webdriver.Firefox()
driver.get("https://www.madeiramadeira.com.br/")
assert "Madeira" in driver.title
elem = driver.find_element_by_id("input-autocomplete")
elem.clear()
elem.send_keys("525119")
elem.send_keys(Keys.RETURN)
product_link = driver.find_element_by_id('variant-url').click()
The error I get is usually this:
NoSuchElementException: Message: Unable to locate element: [id="variant-url"]
There are multiple elements with the id="variant-url", So you could use the index to click on the desired element, Also you need to handle I think the cookie pop-Up. Check the below code, hope it will help
#Disable the notifications window which is displayed on the page
option = Options()
option.add_argument("--disable-notifications")
driver = webdriver.Chrome(r"Driver_Path",chrome_options=option)
driver.get("https://www.madeiramadeira.com.br/")
assert "Madeira" in driver.title
elem = driver.find_element_by_id("input-autocomplete")
elem.clear()
elem.send_keys("525119")
elem.send_keys(Keys.RETURN)
#Clicked on the cookie popup button
accept= driver.find_element_by_xpath("//button[#id='lgpd-button-agree']")
accept.click()
OR with explicitWait
accept=WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//button[#id='lgpd-button-agree']")))
accept.click()
#Here uses the XPath with the index like [1][2] to click on the specific element
product_link = driver.find_element_by_xpath("((//a[#id='variant-url'])[1])")
product_link.click()
OR with explicitWait
product_link=WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"((//a[#id='variant-url'])[1])")))
product_link.click()
I used to get this error all the time when starting. The problem is that there are probably multiple elements with the same ID "variant-url". There are two ways you can fix this
By using "driver.find_elementS_by_id"
product_links = driver.find_elements_by_id('variant-url')
product_links[2].click()
This will create an index off all the elements with id 'variant-url' then click the index of 2. This works but it is annoying to find the correct index of the button you want to click and also takes a long time if there are many elements with the same ID.
By using Xpaths or CSS selectors
This way is a lot easier as each element has a specific Xpath of Selector. It will look like this.
product_link = driver.get_element_by_xpath("XPATH GOES HERE").click()
To get an Xpath or selector 1. go into developer mode on your browser by inspecting the element 2. right click the element in the F12 menu 3. hover over copy 4. move to copy Xpath and click on it.
Hope this helps you :D
i am using a school class schedule website and i want to access the div element that contains info on how many seats are in a class and who is teaching it in order to scrape it. i first find the element which contains the div element i want, after that i try to find the div element i want by using xpaths. the problem i face is when i try to use either the find_element_by_xpath or find_elements_by_xpath to get the div i want i get this error:
'list' object has no attribute 'find_element_by_xpath'
is this error happening because the div element i want to find is nested? is there a way to get nested elements using a div tag?
here is the code i have currently :
driver = webdriver.Chrome(ChromeDriverManager().install())
url = "https://app.testudo.umd.edu/soc/202008/INST"
driver.get(url)
section_container = driver.find_elements_by_id('INST366')
sixteen_grid = section_container.find_element_by_xpath(".//div[#class = 'sections sixteen colgrid']").text
the info i want is this:
<div class = "sections sixteen colgrid"</div>
its currently inside this id tag:
<div id="INST366" class="course"</div>
greatly appreciated if anyone could help me out with this
From documentation of find_elements_by_id:
Returns : list of WebElement - a list with elements if any was found. An empty list if not
Which means section_container is a list. You can't call find_element_by_xpath on a list but you can on each element within the list because they are WebElement.
What says the documentation about find_element_by_id?
Returns : WebElement - the element if it was found
In this case you can use find_element_by_xpath directly. Which one you should use? Depends on your need, if need to find the first match to keep digging for information or you need to use all the matches.
After fixing that you will encounter a second problem: your information is displayed after executing javascript code when clicking on "Show Sections", so you need to do that before locating what you want. For that go get the a href and click on it.
The new code will look like this:
from selenium import webdriver
from time import sleep
driver = webdriver.Chrome()
url = "https://app.testudo.umd.edu/soc/202008/INST"
driver.get(url)
section_container = driver.find_element_by_id('INST366')
section_container.find_element_by_xpath(".//a[#class='toggle-sections-link']").click()
sleep(1)
section_info = section_container.find_element_by_xpath(".//div[#class='sections sixteen colgrid']").text
driver.quit()
I'm trying to use Pythons Selenium module to click on an element whose link has the text "xlsx" at the end of it. Below is the code I'm using and the details of the element. Can someone please see why Python is unable to find this element?
driver.find_element_by_partial_link_text('xlsx').click()
Here is the element details:
<a name="URL$2" id="URL$2" ptlinktgt="pt_new" tabindex="43" onfocus="doFocus_win0(this,false,true);" href="http:******/HSC8_CNTRCT_ITEMS_IMPRVD-1479218.xlsx" onclick="window.open('http:********/HSC8_CNTRCT_ITEMS_IMPRVD-1479218.xlsx','','');cancelBubble(event);return false;" class="PSHYPERLINK">HSC8_CNTRCT_ITEMS_IMPRVD-1479218.xlsx</a>
I had to remove some parts of the URL for confidentiality purposes, however, it should not impact the answering of the question.
Thanks.
Thanks for the replies. Turns out, as #Andersson mentioned, the window was in a different frame.
I solved the problem using the following code before the find_element: driver.switch_to.frame('ptModFrame_0').
You can use a CSS selector:
driver.find_element_by_css_selector("a[href*='xlsx']")
If the element still cannot be located, I would suggest using a wait statement, to ensure that the element is visible, before you interact with it.
Please try:
driver.find_element_by_xpath(".//a[contains(#href,'xlsx')]").
You can grab it by class name (class name = PSHYPERLINK).
This should work:
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
url = ''
driver = webdriver.Chrome('/path/to/chromedriver/executable')
if __name__=='__main__':
driver.get(url)
time.sleep(3)
driver.find_element_by_class_name('PSHYPERLINK').click()
When finding the attribute, make sure to use a singular '"element". Like:
driver.find_element_by_class_name('PSHYPERLINK').click()
not:
driver.find_elements_by_class_name('PSHYPERLINK').click()
Hope this helps.
I'm using Selenium and coding with Python.
I'm trying to do the following: for a flight search website, under Flight 1's 'Enter routing code' text box, type 'AA'
This is the code that I have at the moment:
flight1_routing = driver.find_element_by_xpath(".//*[#id='ita_form_location_RouteLanguageTextBox_0']")
flight1_routing.clear()
flight1_origin.send_keys("AA")
But instead, I get this error message: "invalid element state: Element is not currently interactable and may not be manipulated". How can this be with a regular text field that is also not an autocomplete field, AFAIK?
if you get Element is not currently interactable check if the element is not disabled and its visible. if you want to hack it execute JS to enable it.
i visited the homepage id ita_form_location_RouteLanguageTextBox_0 doesnt exist also under flight one there's no Enter routing code. i can see the text box saying airport city or city name
Also if you have the id prefer to use find_element_by_id if not try to use css selector if you can rather than xpath. Its much cleaner.
Update
here's a working script:
As recomended above, the elements selected are not visible. what is actualy done, is that there's 5-6 different elements all hidden and when you click on show advanced route it picks 2 random ones and makes them visible.
So the id is not always the same. If you use the same id you will get a hidden element some times(because it picks random ids) so selenium is not able to deal with it. i made a selector that gets the 2 hidden elements
from selenium import webdriver
import selenium.webdriver.support.ui as ui
driver = webdriver.Firefox()
driver.get("http://matrix.itasoftware.com/")
#click on the multi tab
tab = driver.find_element_by_id("ita_layout_TabContainer_0_tablist_ita_form_multislice_MultiSliceForm_0").click()
#click on the advanced routes
advanced_routing=ui.WebDriverWait(driver, 10).until(
lambda driver : driver.find_element_by_id("sites_matrix_layout_RouteLanguageToggleLink_1")
)
advanced_routing.click()
#get all visible elements with id like ita_form_location_RouteLanguageTextBox. its similar to regex ita_form_location_RouteLanguageTextBox.*
element = ui.WebDriverWait(driver, 10).until(
lambda driver : driver.find_elements_by_css_selector("[id*=ita_form_multislice_MultiSliceRow] [id*=ita_form_location_RouteLanguageTextBox]")
)
element[0].send_keys("foo")
element[1].send_keys("bar")
import time
time.sleep(20)
Did you click into the correct tab first & enable advanced routing codes?? e.g.
#Go to right tab
driver.find_element_by_css_selector("div#ta_layout_TabContainer_0_tablist_ita_form_multislice_MultiSliceForm_0 > span").click()
#Enable routing
driver.find_element_by_css_selector("a.itaToggleLink").click()
#note I seem to get a different id to the one you're using, assuming its dynamic numbering so handling all cases
#if you know how the dynamic numbering works youmay be able to deduce a single id that will work for your test case
#Instead I'm going for finding all elements matching a pattern then searching through them, assuming only one will be visible
flight1_routings = driver.find_elements_by_css_selector("input[id^='ita_form_location_RouteLanguageTextBox_']")
#probably better finding it then using it separately, but I was feeling lazy sorry.
for route in flight1_routings:
if route.is_displayed():
route.clear()
route.send_keys("AA")
Also you can probably skip the .clear() call as it looks like the box starts with no text to overwrite.
Edit: Updated the enable routing toggling to handle not knowing the id, the class name stays the same, should work. Handling finding the input despite variable id as suggested by foo bar with the css selector, just then iterating over that list and checking if its on top