I am very new to web scraping, so I still have lots of trouble. Currently, I am trying to web scrape from https://www.enterprisetrucks.com/truckrental/en_US.html by setting the pickup time by running this code:
pickupTime = d.find_element_by_id('fldPickuptime_msdd')
pickupTime.click();
select = Select(d.find_element_by_id('fldPickuptime'))
select.select_by_value('20:00')
But I get the error saying that the element is not currently visible and may not be manipulated.
The dropdown which is present is not of the Select type, so you cannot use the Select method here. You need to click on the time using the xpath of that element directly.
You can use the xpath:
pickupTime = d.find_element_by_id("fldPickuptime_msdd")
pickupTime.click();
selectTime = d.find_element_by_xpath("//*[#id='fldPickuptime_msdd']//span[text()='8:00 PM']")
selectTime.click();
Code for JavaScriptExecutor Click:
element = driver.find_element_by_xpath("Enter the xpath here")
driver.execute_script("arguments[0].click();", element)
Related
Good time of the day!
Faced with a seemingly simple problem,
But it’s been a while, and I’m asking for your help.
I work with Selenium on Python and I need to curse about
20 items on google search page by random request.
And I’ll give you an example of the elements below, and the bottom line is, once the elements are revealed,
Google generates new such elements
Problem:
Cannot click on the element. I will need to click on existing and then on new, generated elements in this block: click for open see blocks with element
Tried to click on xpath, having collected all the elements:
xpath = '//*[#id="qmCCY_adG4Sj3QP025p4__16"]/div/div/div[1]/div[4]'
all_elements = driver.find_element(By.XPATH, value=xpath)
for element in all_elements:
element.click()
sleep(2)
Important note!
id xpath has constantly changing and is generated by another on the google side
Tried to click on the class
class="r21Kzd"
Tried to click on the selector:
#qmCCY_adG4Sj3QP025p4__16 > div > div > div > div.wWOJcd > div.r21Kzd
Errors
This is when I try to click using xpath:
Message: no such element: Unable to locate element: {"method":"xpath","selector"://*[#id="vU-CY7u3C8PIrgTuuJH4CQ_9"]/div/div[1]/div[4]}
In other cases, the story is almost the same, the driver does not find the element and cannot click on it. Below I apply a scratch tag on which I need to click
screenshot tags on google search
Thanks for the help!
In case iDjcJe IX9Lgd wwB5gf are a fixed class name values of that element all you need is to use CSS_SELECTOR instead of CLASS_NAME with a correct syntax of CSS Selectors.
So, instead of driver.find_element(By.CLASS_NAME, "iDjcJe IX9Lgd wwB5gf") try using this:
driver.find_element(By.CSS_SELECTOR, ".iDjcJe.IX9Lgd.wwB5gf")
(dots before each class name, no spaces between them)
I try to click on dropdown options using selenium and python. The problem is that the dropdown values are different each time I open the browser therefore, i can not use the HTML tags from the webpage.
I'm wondering if there's any way of clicking the values?
I tried to use this line of code using answered provided by Sers in this case as an example How to select/click in a dropdown content using selenium chromewebdriver / python but I'm not sure how should I change/compose the class here.
att4 = WebDriverWait(driver, 2).until(ec.visibility_of_element_located((
By.CLASS_NAME, f"div.awsui-select-option awsui-select-option-selectable[title='{l}']"))).click()
l is for each option in my dropdown, I need to click on each of them
You are using a wrong, non-stable locator.
Try this:
att4 = WebDriverWait(driver, 2).until(ec.visibility_of_element_located((By.XPATH, "//div[#data-value='Bambino']"))).click()
Or
att4 = WebDriverWait(driver, 2).until(ec.visibility_of_element_located((By.XPATH, "//div[#title='Bambino']"))).click()
I'm kinda of a newbie in Selenium, started learning it for my job some time ago. Right now I'm working with a code that will open the browser, enter the specified website, put the products ID in the search box, search, and them open it. Once it opens the product, it needs to extract its name and price and write it in a CSV file. I'm kinda struggling with it a bit.
The main problem right now is that Selenium is unable to open the product after searching it. I've tried by ID, name and class and it still didn't work.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import csv
driver = webdriver.Firefox()
driver.get("https://www.madeiramadeira.com.br/")
assert "Madeira" in driver.title
elem = driver.find_element_by_id("input-autocomplete")
elem.clear()
elem.send_keys("525119")
elem.send_keys(Keys.RETURN)
product_link = driver.find_element_by_id('variant-url').click()
The error I get is usually this:
NoSuchElementException: Message: Unable to locate element: [id="variant-url"]
There are multiple elements with the id="variant-url", So you could use the index to click on the desired element, Also you need to handle I think the cookie pop-Up. Check the below code, hope it will help
#Disable the notifications window which is displayed on the page
option = Options()
option.add_argument("--disable-notifications")
driver = webdriver.Chrome(r"Driver_Path",chrome_options=option)
driver.get("https://www.madeiramadeira.com.br/")
assert "Madeira" in driver.title
elem = driver.find_element_by_id("input-autocomplete")
elem.clear()
elem.send_keys("525119")
elem.send_keys(Keys.RETURN)
#Clicked on the cookie popup button
accept= driver.find_element_by_xpath("//button[#id='lgpd-button-agree']")
accept.click()
OR with explicitWait
accept=WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//button[#id='lgpd-button-agree']")))
accept.click()
#Here uses the XPath with the index like [1][2] to click on the specific element
product_link = driver.find_element_by_xpath("((//a[#id='variant-url'])[1])")
product_link.click()
OR with explicitWait
product_link=WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"((//a[#id='variant-url'])[1])")))
product_link.click()
I used to get this error all the time when starting. The problem is that there are probably multiple elements with the same ID "variant-url". There are two ways you can fix this
By using "driver.find_elementS_by_id"
product_links = driver.find_elements_by_id('variant-url')
product_links[2].click()
This will create an index off all the elements with id 'variant-url' then click the index of 2. This works but it is annoying to find the correct index of the button you want to click and also takes a long time if there are many elements with the same ID.
By using Xpaths or CSS selectors
This way is a lot easier as each element has a specific Xpath of Selector. It will look like this.
product_link = driver.get_element_by_xpath("XPATH GOES HERE").click()
To get an Xpath or selector 1. go into developer mode on your browser by inspecting the element 2. right click the element in the F12 menu 3. hover over copy 4. move to copy Xpath and click on it.
Hope this helps you :D
I want to enter a value into a field
I find the element by:
driver.find_element_by_xpath('//*[#id="cell--gE-Ez1qtyIA"]')
And I do this to display the text and click:
driver.find_element_by_xpath('//*[#id="cell--gE-Ez1qtyIA"]').click()
driver.find_element_by_xpath('//*[#id="cell--gE-Ez1qtyIA"]').text
So far so good, but I use this code when I want to send a value to this field:
driver.find_element_by_xpath('//*[#id="cell--gE-Ez1qtyIA"]').send_keys('test')
Nothing happens and the field value does not change
Since we are unaware about the nature of the text box you are trying to access, there are some possibilities because of which you are facing this situation. The send_keys() works on all the text box which ever you use. Look for the points below and try applying it on the element:
The element is not extracted properly, try some other locator like id, name, css_selector.
The element is not ready to use, put some explicit wait and check for presence and visibility of element.
A sample code can be using explicit wait:
wait = WebDriverWait(driver, 10)
element = wait.until(EC.presence_of_all_elements_located(("xpath", "//*[#id='cell--gE-Ez1qtyIA']")))
element.send_keys("test")
Try to use javascriptexecutor instead of sendkeys
WebDriver driver = new ChromeDriver();
JavascriptExecutor je = (JavascriptExecutor)driver;
je.executeScript("document.getElementById('IDofElement').setAttribute('value', 'your value here')");
I am trying to retrieve all possible lists from this website
Model will only open when year is selected. Similarly Make will only open when Model is selected. I want to store all combination of Year, Model and Make for learning purpose.
However I am not able to click year field only. It seems it is hidden and without this I can't go ahead with rest of code.
from selenium import webdriver
import time
from selenium.webdriver.support.ui import Select
from bs4 import BeautifulSoup
driver = webdriver.Chrome()
driver.get("https://www.osram-americas.com/en-us/applications/automotive-lighting-systems/Pages/lrgmain.aspx")
# Switch to new window opened
driver.switch_to.window(driver.window_handles[-1])
# Close the new window
driver.close()
# Switch back to original browser (first window)
driver.switch_to.window(driver.window_handles[0])
el = driver.find_element_by_id('sbToggle_27562807')
el.click()
It gives error :-
Message: no such element: Unable to locate element: {"method":"id","selector":"sbToggle_27562807"}
Message: no such element: Unable to locate element: {"method":"id","selector":"sbToggle_27562807"}
year element is not hidden, actually your locator to locate element using find_element_by_id('sbToggle_27562807') is not correct. In this element id attribute value is dynamically changing that's why you're unable to locate this element.
Instead of id locator you should try using some different locator. I would suggest, for better way try using find_element_by_css_selector() as below :-
el = driver.find_element_by_css_selector("div#fldYear a.sbToggle")
el.click()
Use this CSS selector to get to the arrow pointing downwards on the select year dropdown
"div[id='fldYear'] > div[class='sbHolder'] > a[class='sbToggle']"
or this xpath
"//div[#id='fldYear']/div[#class='sbHolder']/a[#class='sbToggle']"
Click on this to webelement to get the options
The element IDs change on each reload of the page, you'll have to find a different way to find the dropdown.
You can always find the <a> link with the "-- Select Year --" text, for example.