I am actually trying to scrap a website. In fact I have a table like this below :
Table
I would like to navigate to the line that contains the word "Maître", but I am not able to find this line with selenium. Below the html code of the page :
Html code 1
Html code 2
And this is my code :
objets_de_risque = driver.find_element(By.ID,'sidebar-link-0-1')
driver.execute_script("arguments[0].click();", objets_de_risque)
code_h = driver.find_element(By.ID,'input-search-1')
code_h.send_keys(Keys.CONTROL + "a")
code_h.send_keys(Keys.DELETE)
code_h.send_keys("H1404")
code_h.send_keys(Keys.ENTER)
driver.switch_to_frame("main1")
line_maitre = driver.find_elements_by_xpath("//*[contains(text(), 'Maître')]")
driver.execute_script("arguments[0].click();", line_maitre)
The last 2 lines doesn't seem to work. Is there a way to go to this line with selenium ?
Thank you very much
find_elements() returns a list. Instead you need find_element() to locate the element.
Solution
To locate a visible element you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:
Using XPATH:
line_maitre = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//td[text()=''Maître'']")))
However to click on it as per your code in a single line:
driver.execute_script("arguments[0].click();", WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//td[text()=''Maître'']"))))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Related
So I'm Fairley new to selenium and I'm attempting click on a table but I can't seem to find it.
Below I have posted my code along with a screen shot of what I'm trying to click on.
Here is my code:
competitorPrices = WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.LINK_TEXT, "Competitor Prices"))).click()
HTML snapshot:
Error:
Element snapshot:
I would try something like this:
...
import time
time.sleep(5)
xpath = "//*[contains(text(),'Competitor Prices')]"
element = driver.find_element(by=By.XPATH, value=xpath)
competitorPrices = driver.execute_script("$(arguments[0]).click();", element)
Sometimes the method element_to_be_clickable doesn't work out of the box. In this cases to debug I use time.sleep so I can understand better the problem
The desired element is a <td> not an <a> tag. Hence LINK_TEXT won't work.
Moreover click() doesn't returns anything, so competitorPrices will be always None which you may remove.
Solution
You can use either of the following locator strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "table#tblTabStrip td[ch^='CompetitorPrices']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//table[#id='tblTabStrip']//td[text()='Competitor Prices']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I am trying to get the following count of an Instagram account. I believe that the XPATH is correct an exists. Here's a screenshot showing it exists when I search for it:
This is my code:
wait = WebDriverWait(driver, 30)
followers = wait.until(EC.presence_of_element_located((By.XPATH, "/html/body/div[1]/div/div/div/div[1]/div/div/div/div[1]/div[1]/section/main/div/ul/li[2]/button/div/span")))
print(followers.get_attribute("title"))
I have even looked at similar projects that find the following count and our code is almost exactly the same.
The desired element is a dynamic element, so to locate the element instead of presence_of_element_located() you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategy:
Using XPATH:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[contains(., 'followers')]//span[#class and #title and text()]"))).get_attribute("title"))
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I have been trying to make a Selenium bot that searches for a word on github, clicks on the first link and then downloads it with Python 3.8 and I got stuck with making the bot click on the hyperlink. I understand that I can make the bot click on it with driver.find_element(By.XPATH, "Xpath").click() but I want to be able to find the path of the href with another method for the sake of learning, in this case CSS_SELECTOR. Source code of the first hyperlink result is like this:
HTML:
Since every single result is under the same "a" selector with a class of "v-align-middle", I thought of using this code: driver.find_element(By.CSS_SELECTOR, ".v-align-middle").click() but it did not seem to work. What am I missing?
The desired element is a dynamic element, so to click() on the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.v-align-middle[href='/asdf-vm/asdf'][data-hydro-click][data-hydro-click-hmac]"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[#class='v-align-middle' and #href='/asdf-vm/asdf'][#data-hydro-click and #data-hydro-click-hmac]"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I am trying to scrape the tables from the below dynamic webpage. I am using the below code to find the data in tables (they are under tag name tr). But I am getting empty list as output. Is there anything that I am missing here?
https://www.taipower.com.tw/tc/page.aspx?mid=206&cid=406&cchk=b6134cc6-838c-4bb9-b77a-0b0094afd49d
from selenium import webdriver
chrome_path = r"C:\Users\upko\Downloads\My Projects\Ibrahim Projects\chromedriver.exe"
driver = webdriver.Chrome(chrome_path)
driver.get("https://www.taipower.com.tw/tc/page.aspx?mid=206&cid=406&cchk=b6134cc6-838c-4bb9-b77a-0b0094afd49d")
driver.find_elements_by_tag_name('tr')
Website have iframes, you need switch into desired iframe to access data. Didnt tested code, but should work
iframe = driver.find_element_by_xpath("//iframe[#id='IframeId']")
driver.switch_to_frame(iframe)
#Now you can get data
trs = driver.find_elements_by_tag_name('tr')
The desired elements are within an <iframe> so you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired visibility_of_all_elements_located.
You can use either of the following Locator Strategies:
Using XPATH:
driver.get("https://www.taipower.com.tw/tc/page.aspx?mid=206&cid=406&cchk=b6134cc6-838c-4bb9-b77a-0b0094afd49d")
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"//iframe[#id='IframeId']")))
print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[#class='container-fluid']//div[#class='span6']/strong")))])
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Console Output:
['核能(Nuclear)', '燃煤(Coal)', '汽電共生(Co-Gen)', '民營電廠-燃煤(IPP-Coal)', '燃氣(LNG)', '民營電廠-燃氣(IPP-LNG)', ' 燃油(Oil)', '輕油(Diesel)', '水力(Hydro)', '風力(Wind)', '太陽能(Solar)', '抽蓄發電(Pumping Gen)']
Reference
You can find a couple of relevant discussions in:
Switch to an iframe through Selenium and python
I am trying to download an excel file using Selenium Python using the below code:
download=driver.find_element_by_css_selector("span[data-reactid='.0.1.0.0.1.1.1.0.1.0.1'>Download Orders]")
The element of the download button look like:
The piece of code above isnt working. Can someone help?
Your CSS locator is wrong. Please try to click with below xpath.
//span[.='Download Orders']
The desired element is a React element so to locate and click() on the element you have to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using XPATH with normalize-space():
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[normalize-space()='Download Orders']"))).click()
Using XPATHand contains():
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[contains(., 'Download Orders')]"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I agree with Muzzamil that the CSS locator you have used seems wrong. # is needed to select the attribute data-reactid.
You can try XPath:
//span[#data-reactid='.0.1.0.0.1.1.1.0.1.0.1']
If you want to include also the text, try:
//span[#data-reactid='.0.1.0.0.1.1.1.0.1.0.1'][contains(text(),'Download Orders')]