I am trying to download the csv file from this link with selenium: https://ourworldindata.org/grapher/annual-co-emissions-by-region and am encountering issues regarding finding the actual elements necessary to download the file. Here is my code so far:
driver.get("https://ourworldindata.org/grapher/annual-co-emissions-by-region")
frame = driver.find_element_by_id("GTM-N2D4V8S")
driver.switch_to.frame(frame)
downloadLink = driver.find_element_by_xpath("/html/body/main/figure/div/div[4]/div/nav/ul/li[4]/a")
driver.execute_script("arguments[0].click();", downloadLink)
Specifically, I am trying to find the second div underneath the element with this css selector body > main > figure > div > div.DownloadTab, which contains the download link. Unfortunatley, while I am able to find this parent element, I am unable to find the div I am looking for. From doing research, I think that this issue might have to do with the iframe, so I am trying to switch it with the code above (although it is not working). This link may be helpful: Selenium can't find elements even if they exist
Thanks so much!
You should wait for the page / elements fully loaded before accessing them and to use the correct locators.
Try this:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 20)
driver.get("https://ourworldindata.org/grapher/annual-co-emissions-by-region")
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "li.download-tab-button"))).click()
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "[data-track-note='chart-download-csv']"))).click()
Related
I have found webelement using xpath text and performed some actions after finding the webelement. the webelement is in a dropdown. Its a big dropdown with more than 500 options in it. Now I want to perform same set of actions for every webelement which is found using the xpath text I have mentioned.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
driver = webdriver.Chrome()
driver.get("url")
driver.find_element_by_xpath("//*[contains(text(), 'abc')]")
driver.find_element_by_xpath("//*[#id='btnsubmit']").click()
driver.find_element_by_xpath("//*[#src='Images/pdf.jpg']").click()
with this code I am able to locate the drop down element which has text abc in it and download the pdf file which appears after clicking the submit button. Now I want to loop this and find other elements which have abc in it and download pdfs for each dropdown element.
I have tried this:
element = driver.find_element_by_xpath("//*[contains(text(), 'abc')]")
for i in element :
element.click()
driver.find_element_by_xpath("//*[#id='btnsubmit']").click()
driver.find_element_by_xpath("//*[#src='Images/pdf.jpg']").click()
but this only finding the first element in the dropdown and downloading the pdf. I want to iterate this for all the elements that have abc text in it and download the pdf.
Also tried this:
element = driver.find_elements_by_xpath(".//*[contains(text(), 'abc')]")
for i in element :
element.click()
driver.find_element_by_xpath("//*[#id='btnsubmit']").click()
driver.find_element_by_xpath("//*[#src='Images/pdf.jpg']").click()
it returns an exception that click cannot be performed on list.
can anyone help me with this problem? it would help me reduce a lot of manual work of clicking on each drop down element and downloading the pdf.
You should use i.click instead of element.click there.Or use the below code
elements = driver.find_elements_by_xpath(".//*[contains(text(), 'abc')]")
for element in elements:
element.click()
driver.find_element_by_xpath("//*[#id='btnsubmit']").click()
driver.find_element_by_xpath("//*[#src='Images/pdf.jpg']").click()
Here I have given the name element so it would work.
i would like to get the element and click on it by the text under tag a:
1791
The text "1791" is what I used to locate this element and click on it.
My code looks like this:
driver.find_element(By.XPATH,("//div[text()='1791']")).click()
and it does not work. My selenium version is 4.3.0, so it does not support command such as find_element_by_xpath. Hence, the answers I found on the internet offers no help.
Thank you!
You are trying to click on the div with the text '1791'.
But in the provided info this text is inside a tag.
Try the following:
driver.find_element(By.XPATH,("//a[text()='1791']")).click()
UPD
Except for the incorrect XPath the problem was in unused WebDriverWait for the element (the expectation for checking that an element is present on the DOM of a page before clicking on it).
Webdriver tries to click an element immediately after opening the desired page, but some elements may not be loaded yet.
The final code will be:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH,("//a[text()='1791']"))).click()
With imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I am very new to programing and am currently messing around trying to make a bot that will add items to my cart for me. I am using python with the selenium module. The problem I have came across is getting the program to find find an item using with src or href code.
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located(())
)
element.click()
This is the code I am using and I dont know what to put in the brackets to find the src or href. I am wondering which one is easier to use and how to use it in my code. Any advice/criticism at all could help me as I am still learning python. Thank you.
So your question is as follows:
[How] to find find an item using with src or href code[?]
Let's take a code snippet as an example.
This is from the https://www.google.com/ startpage. If you want to find this anchor element with said href attribute by specifically looking for the href url, you need to search with e.g.:
driver.find_element_by_xpath('//a[#href="https://mail.google.com/mail/?tab=rm&ogb"]')
Same goes for your code snippet:
element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, ('//a[#href="https://mail.google.com/mail/?tab=rm&ogb"]'))))
element.click()
Imported modules:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
I am trying to extract some information from the amazon website using selenium. But I am not able to scrape that information using xpath in selenium.
In the image below I want to extract the info highlighted.
This is the code I am using
try:
path = "//div[#id='desktop_buybox']//div[#class='a-box-inner']//span[#class='a-size-small')]"
seller_element = WebDriverWait(driver, 5).until(
EC.visibility_of_element_located((By.XPATH, path)))
except Exception as e:
print(e)
When I run this code, it shows that there is an error with seller_element = WebDriverWait(driver, 5).until( EC.visibility_of_element_located((By.XPATH, path))) but does not say what exception it is.
I tried looking online and found that this happens when selenium is not able to find the element in the webpage.
But I think the path I have specified is right. Please help me.
Thanks in advance
[EDIT-1]
This is the exception I am getting
Message:
//div[class='a-section a-spacing-none a-spacing-top-base']//span[class='a-size-small a-color-secondary']
XPath could be something like this. You can shorten this.
CSS selector could be and so forth.
.a-section.a-spacing-none.a-spacing-top-base
.a-size-small.a-color-secondary
I think the reason is xpath expression is not correct.
Take the following element as an example, it means the span has two class:
<span class="a-size-small a-color-secondary">
So, span[#class='a-size-small') will not work.
Instead of this, you can ues xpath as
//span[contains(#class, 'a-size-small') and contains(#class, 'a-color-secondary')]
or cssSelector as
span.a-size-small.a-color-secondary
Amazon is updating its content on the basis of the country you are living in, as I have clicked on the link provided by you, there I did not find the element you are looking for simply because the item is not sold here in India.
So in short if you are sitting in India and try to find your element, it is not there, but as you change the location to "United States". it is appearing there.
Solution - Change the location
To print the Ships from and sold by Amazon.com of an element you have to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR and get_attribute():
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.a-section.a-spacing-none.a-spacing-top-base > span.a-size-small.a-color-secondary"))).get_attribute("innerHTML"))
Using XPATH and text attribute:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[#class='a-section a-spacing-none a-spacing-top-base']/span[#class='a-size-small a-color-secondary']"))).text)
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python
Outro
Link to useful documentation:
get_attribute() method Gets the given attribute or property of the element.
text attribute returns The text of the element.
Difference between text and innerHTML using Selenium
I am trying to make a program which logs into my depop (A second-hand clothing selling app) and searches for a user. I've managed to make it log in so far. However, it can't find the element of the search button.
I've tried multiple didn't methods but none have worked. Keep in mind I have started learning this today so I am a beginner.
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys #allows keyboard to be used
from selenium.webdriver.common.by import By #allow waiting
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as cond
from selenium.webdriver.support import expected_conditions as ec
from selenium.common.exceptions import NoAlertPresentException
from selenium.common.exceptions import TimeoutException
def search():
driver.implicitly_wait(4)
searchbox = driver.find_element_by_name('a')
Message: no such element: Unable to locate element
I just get variations of this error message depending on how I do it.
EDIT:
The element is :
a
< span > Search < /span >
/a
EDIT 2: TO add some more detail onto this to make it easier to understand, when I click on the button, it makes a pull down menu of the actual search bar. So if it eventually finds the element. It gives me this "Message: stale element reference: element is not attached to the page document"
Your find_element_by_name function locates a WebElement by its name attribute, in your case neither span nor a tag have the name.
So you either need to go for find_element_by_tag_name attribute:
driver.find_element_by_tag_name("a")
but be aware that it will return the first link from the DOM, if you need to match the link having inner span with Search text you will need to go for XPath locator like:
driver.find_element_by_xpath("//span[contains(text(),'Search')]/parent::a")
It would be also good to wrap your locator into an Explicit Wait
a is not a name and hence it would not result in finding an element by using method find_element_by_name. Please try using xpath //span[text()='Search'] to find the element search. Please check(in developer console or some other browser extension like xpath, chropath etc) how many elements are being returned using this xpath. If there is just one element, you are good to go. Else, find some other strategy to reach to the desired element before proceeding with performing actions on it.