I'm trying to get some information about a book of Amazon, which worked great for me until I got to the point where I wanted to scrape the content description. The content description is in an iframe-container in which a new HTML-code is started.
I have no problem catching the container via
content = driver.find_element_by_xpath("//iframe[#id='bookDesc_iframe']")
but I can't seem to get the part with the content. I tried
content_text = content.find_element_by_xpath("//div[#id='iframeContent']")
since is where it's hidden but it doen't work for me.
In order to access the iframe content you need to switch into that iframe.
driver.switch_to.frame(driver.find_element_by_xpath("//iframe[#id='bookDesc_iframe']"))
To continue working with other elements, not inside that iframe you will have to switch out of the iframe, to the default content, like this:
driver.switch_to.default_content()
Adding to the #Prophet answer, in case if you would like to do the same with explicit wait (which will eventually be a more reliable), you could do this :
wait = WebDriverWait(driver, 10)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//iframe[#id='bookDesc_iframe']")))
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Related
I would like to click on one of these tabs but I am not able to access it.
How can I access the data-toggle with Selenium in python?
Please try something like this:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[#class=='section-tabs']//a[contains(#href,'timeseries-tab')]"))).click()
The code above should be able to select the timeseries tab.
You did not share the xml of the other tab, but I guess this should work:
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[#class=='section-tabs']//a[contains(#href,'heatplot-tab')]"))).click()
This can be also done with CSS Selectors.
To give better answer we need the link to that page or at least the entire XML of that page as a text, not picture
Let's say there is a page and there is a "next page" button in it.
browser is initialized using webdriver.Chrome(chrome_driver_path).
I use page_source attribute to get the orginal page's source code and locate the button for next page. Then I send click to it, the browser will turn to next page in the same tab in Chrome.
browser.get(origin_page_url)
browser.find_element_by_xpath(btn_xpath).click()
time.sleep(5)
After above codes then I call page_source again, but what I get is the same as the original page's source code.
How can I get the newly opened page's source code?
You need to explicitly wait for the new page to load before getting the page source. This usually depends on the webpage you are working with. For instance, you may wait until a particular element becomes visible:
element = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "myXpath")))
element.click();
After that call page_source
Import
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
I have not tested this but, sure this will work for you.
I am trying to enter a field with text (email), in the username box. However, it is not interacting with it. I have it in a try & except statement but it only spits out excepts.
Website is account.protonmail.com/signup
I have tried everything: xpath, class, id, but nothing works. I am losing my mind!
Password works fine:
Does not work:
driver.find_element_by_xpath('//*[#id="username"]').send_keys(password)
Does work:
driver.find_element_by_xpath('//*[#id="repeat-password"]').send_keys(password)
And yes, I do wait until JavaScript finishes running.
it is in iframe, and needs explicit waits :-
driver = webdriver.Chrome(driver_path)
driver.maximize_window()
driver.get("https://account.protonmail.com/signup")
wait = WebDriverWait(driver, 20)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe[sandbox^='allow-scripts']")))
ele = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#username"))).send_keys('someemail#gmail.com')
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Element you are trying to access is inside an iframe. First you need to switch to the frame like this
driver.switch_to.frame(driver.find_element_by_xpath("//iframe[contains(#src,'account-api.protonmail.com')]"))
It's very difficult for you to write everything, make it easier
click = driver.find_element_by_xpath('//*[#id="username"]').click()
click.send_keys('')
click.send_keys(password)
I'm attempting to automate some work with selenium and I'm having issues sending keys to the input id when attempting to login and I'm wondering if I'm in over my head.
I've tried locating the element by xpath, id as well as class, with no luck. I've also tried to wait to make sure the element is visible with no luck. Perhaps it's due to the div xmlnsis inside a body xmlns, resulting in the elements within the div xmlns initially not being visible? If so, how do I go about making the input box visible?
This is basically how far I've come:
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
# website url
url = "https://tx-production-eu-web.production.eu1.tx.klarna.net/radix/"
# driver
driver = webdriver.Chrome(r"C:\Users\ahmed.khairouni\Desktop\driver\chromedriver.exe")
driver.get(url)
# wait for page to load
wait = WebDriverWait(driver, 5)
#locate element and insert text into textbox
username = wait.until(EC.presence_of_element_located((By.ID, "wf_139389")))
search_input.send_keys('username')
Linked you can find part of the web code: https://imgur.com/WVKo7B7
Appreciate any help and thank you in advance.
The issue here might be the input is using a dynamic ID, so finding by ID may not work. I also noticed you are locating username, but then you are sending keys to search_input, which does not seem to exist in your code. That may be causing your issue, unless you made a typo in this question.
You could try using an XPath to locate the username input:
username = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//input[contains(#class, 'rwt-regular-value')]")))
username.send_keys('username')
I have looked at some questions on here that are somewhat related but none of them are concerned with finding a class and then extracting text that is part of that class.
My goal is to automate the process of entry into a website that finds emails, given name and domain names. So far I have the code to automate the entry of the name and domain, and then click on "search", but my issue is waiting for a result to load. I basically want my code wot wait until the CSS class "one" is present and then extract the text related to this class, e.g. for the following snippet:
<h3 class="one">Success</h3>
Extract the text "Success".
You need to explicitly wait for the element to become visible:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
h3 = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "h3.one")))
print h3.text