Using Selenium in Python, I'd like to download a page, and save the HTML code of a particular div, identified by its id. I've got the following:
from selenium.webdriver import Firefox
from selenium.webdriver.support.ui import WebDriverWait
...
with closing(Firefox()) as browser:
browser.get(current_url)
WebDriverWait(browser, timeout=3).until(lambda x: x.find_element_by_id('element_id'))
element = browser.find_element_by_id('element_id')
element is of type selenium.webdriver.remote.webelement.WebElement. Is there a way to get the HTML code (not processed in any way) from element? Is there some better way, using Selenium, of accomplishing this task?
Right from pydoc selenium.webdriver.remote.webelement.WebElement:
| text
| Gets the text of the element.
Use the .text attribute.
If you really are after the HTML source of the element
then please see: Get HTML Source of WebElement in Selenium WebDriver using Python
As stated above, it's not as straight forward as you'd like.
Related
i'm trying to create python automation scripts using selenium to make appointment times, however i'm having some trouble with clicking on certain elements. to confirm an appointment the steps are going to the webpage > selecting the date > selecting the time > hitting confirm
this is what i've got so far:
driver = webdriver.Chrome()
driver.get('<url.date>') -- this works
driver.find_element(By.ID,'corresponding time id').click() -- this works
driver.find_element(By.DATA-TEST-ID,'order_summary_page-button-book').click() -- this is where i'm struggling
i've also tried doing it by the button class, using the css selector and looking for 'Reserve Now' text, and using the xpath
when i do it by xpath, i get the error NoSuchElementException
i tried adding a webdriver wait function but that didn't seem to work either.
the appointment times are really hard to get, so i'd rather not but any wait times into the code so it can execute and book as soon as slots open
here are the elements:
https://i.stack.imgur.com/S62kR.png
any help is appreciated!
Selenium doesn't have option to identify the element by DATA-TEST-ID locator.
In this case You have to use either by xpath or css selector
driver.find_element(By.XPATH,'//button[#data-test-id="order_summary_page-button-book"]').click()
OR
driver.find_element(By.CSS_SELECTOR,'button[data-test-id="order_summary_page-button-book"]').click()
Ideally you should use explicit wait to click on the element.
WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, '//button[#data-test-id="order_summary_page-button-book"]'))).click()
you need following imports.
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
I'm trying to click on an element in a dropdown list, and Selenium is able to locate and click the button that makes the list appear, but it doesn't seem to to be able to locate the list itself when it pops up. I have found the HTML that controls the list and tried various find_element_by methods, and none are able to locate it.
I've tried using the Select module, as well as switch_to_frame (which I'm not sure I'm doing correctly), and I've had no luck at all. Any help would be appreciated.
EDIT:
Upon further investigation, there appears to be something "off" with the HTML. When I copied the XPATH from the Chrome inspector, it gave me /html/body/div[5]/div/div/ul/li[3]/div/button, however if you notice, the first div element is div[5], but looking at the html, there are only 4 div elements that are children of the body. Also, when I copy the CSS Selector, I get body > div:nth-child(13) > div > div > ul > li:nth-child(3) > div > button, but there are only 12 total children of the body. Even stranger is the fact that both leaving these locators as they are and attempted to correct for them yield the same results, which is the error.
Here is my code:
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.select import Select
browser = webdriver.Chrome()
url = <url>
browser.get(url)
export1 = browser.find_element_by_xpath("//*[#id=\"react-main-content\"]/div[1]/section/div[1]/div[3]/div[2]/div/button")
time.sleep(1)
export1.click()
# This is the element it's unable to locate
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[4]/div/div/ul/li[3]/li/button")))
# Execution never reaches this point
export2 = browser.find_element_by_xpath("/html/body/div[4]/div/div/ul/li[3]/li/button")
export2.click()
Here is the HTML (I can't get it to format correctly, so here's a screenshot. The Highlighted section is the drop down I'm looking for):
Here is the webpage with the dropdown open and the element in the dropdown I'm trying to locate highlighted.
There seems to be a mistake in an xpath. you have already traversed through the list and you need not do it again. Try this one instead.
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[4]/div/div/ul/li[3]/button")))
I figured it out. Silly me assumed that the HTML from the website being opened in one browser window would match the HTML from the browser window Selenium opens (and seeing as I used the HTML from that same window to locate several other elements, why wouldn't I?). That, however, was not the case.
After pausing execution of my program and examining the HTML of the website in the browser window Selenium opens, I noticed that the location of the element I'm trying to locate is different, so I copied the XPATH from the Selenium browser window and the issue is now resolved.
I have been trying to get an account balance text to print in python using selenium. I go to chrome, inspect the element, copy the xpath, put it in VS and it says cant find the xpath. I did the same thing for CSS selector, full xpath, tried to create my own xpath, tried by class name, etc. Ive tried every option it has. nothing is working. What do you do when you run into this situation? It is there on the website, its gotta be accessible in the background somewhere. The has to be a way to grab it. I have been able to get other text on the site to display properly.
This is just a stock simulator game, I have it open to the public if you want to log in and try your hand at getting the correct inspect element.
The xpath it generates:
/html/body/div[4]/div[3]/div[1]/div[2]/ul/li[1]/span
How I have it in VS:
driver.find_element_by_xpath('/html/body/div[4]/div[3]/div[1]/div[2]/ul/li[1]/span')
The error in VS:
Unable to locate element: {"method":"xpath","selector":"/html/body/div[4]/div[3]/div[1]/div[2]/ul/li[1]/span"}
To get the amount Net Worth Use WebDriverWait() and wait for visibility_of_element_located() and following xpath.
print(WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.XPATH,"//div[./span[text()='Net Worth']]/following::span[1]"))).text)
OR
print(WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.XPATH,"//ul[#class='list list--kv']/li[1]/span"))).text)
You need to import below libraries.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
I try to script some test on an internal website in selenium in python.
And this site contains a lot of iframes.
I have no issue to switch iframe, but I try to set the url of the current iframe, and I do not find any method to attain this.
iframe1 = WebDriverWait (driver, 10).until (EC.presence_of_element_located((By.ID, 'iframe1')))
driver.switch_to_frame (iframe1)
iframe2 = WebDriverWait (driver, 10).until (EC.presence_of_element_located((By.ID, 'iframe2')))
driver.switch_to_frame (iframe2)
# I was expecting something like this
driver.get("/new_url_inside_my_frame.html")
But it does not work, at first because get does not works with relative url, but even if I use a complete url, it sets the all the page and not just the iframe.
I am pretty sure, this is possible, I just not find how anywhere.
Im trying to scrape a webpage using selenium. The xpaths suggested by inspecting the page and right clicking are of an unstable kind (/html/body/table[2]/tbody/tr[1]/td/form/table/tbody/tr[2]) . So I tried the following solution instead:
driver = webdriver.Chrome("path")
driver.get("https://www.bundesfinanzhof.de/entscheidungen/entscheidungen-online")
time.sleep(1)
links=driver.find_element_by_xpath('//tr[#class="SuchForm"]')
or even
links=driver.find_elements_by_xpath('//*[#class="SuchForm"]')
don't return any results. However earlier on in the page I can obtain:
links=driver.find_element_by_xpath('//iframe')
links.get_attribute('src')
It seems that after:
<script language="JavaScript" src="/rechtsprechung/jscript/list.js" type="text/javascript"></script>
I can no longer get to any of the elements.
How do I determine the correct XPath?
suggests that parts within a script are impossible to parse. However, the path I am after seems to me not to be within a path. Am I misinterpretting how scripts work on a page ?
For instance, later on there is a path:
/html/body/table[2]/tbody/tr[1]/td/script
I would expect this to create such a problem. I am by no means a programmer, so my understanding of this subject is limited. Can someone explain what the problem is and if possible a solution ?
Attempted using solutions from:
Find element text using xpath in selenium-python NOt Working
xpath does not work with this site, pls verify
The table is located inside an iframe, so you need to switch to that iframe before handling required tr:
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver.get("https://www.bundesfinanzhof.de/entscheidungen/entscheidungen-online")
wait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#src='https://juris.bundesfinanzhof.de/cgi-bin/rechtsprechung/list.py?Gericht=bfh&Art=en']")))
link = driver.find_element_by_xpath('//tr[#class="SuchForm"]')
Use driver.switch_to.default_content() to switch back from iframe