I am trying this code to click on the javascript button. In the first frame you find the content, but when I place myself in the second html there is no menu.
How do I click on the button?
Code trials:
driver.switch_to.default_content()
driver.find_element_by_tag_name('frameset')
driver.switch_to_frame(1)
driver.find_element_by_tag_name('html')
print(driver.page_source)
driver.find_element_by_tag_name('frameset')
driver.switch_to_frame(1)
driver.find_element_by_tag_name('html')
print(driver.page_source)
<html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" />
</head><frameset cols="170px,100%" framespacing="0" border="0" frameborder="0">
<frame scrolling="no" name="menu" id="menu" src="menu.jsp" />
<frame scrolling="yes" name="main" id="main" src="main.jsp" />
</frameset>
As the the desired element is within an multiple <iframe>s so you have to:
Induce WebDriverWait for the parent frame to be available and switch to it.
Induce WebDriverWait for the child frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//frame[contains(#src, 'login')]")))
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//frame[#id='menu' and #name='menu'][contains(#src, 'menu')]")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "squeda Avanzada"))).click()
Here you can find a relevant discussion on Ways to deal with #document under iframe
Related
I'm trying to download a bunch of images and categorize them into folders using Selenium. To do so, I need to grab two ID's associated with each image within the URL. However I'm having trouble scraping the image link from the src attribute. Whether I try to grab by tag, Xpath, or other method the end result is merely "None".
Here's an example of an inspected image page:
<html style="height: 100%;"
><head><meta name="viewport" content="width=device-width, minimum-scale=0.1">
<title>index.php (2448×3264)</title>
</head>
<body style="margin: 0px; background: #0e0e0e; height: 100%">
<img style="-webkit-user-select: none;margin: auto;cursor: zoom-in;background-color: hsl(0, 0%, 90%);transition: background-color 300ms;" src="https://haalsi.net/haalsi_pride2/custom/picture/index.php?id=LQCMY&fieldname=DT006_picture&p=show" width="444" height="593">
</body>
</html>
For this example, I would need to grab "LQCMY" and "DT006_picture" as strings from the URL above. The code below shows my attempt at scraping the URL link (edited down since prior screens I click through are locked behind passwords that I can't give out).
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Image = '/html/body/div[1]/div[2]/div/table/tbody/tr[1]/td[1]/a'
driver.find_element_by_xpath(Image).click()
Image_URL = WebDriverWait(driver, 100).until(EC.element_to_be_clickable((By.XPATH, Image))).get_attribute('src')
print(Image_URL)
Are there certain src's that can't be scraped, or am I scraping the wrong tag?
I've tried grabbing by tag but that also returns "None" as well.
Image_URL = driver.find_element_by_xpath(Image).get_attribute('src')
Other posts said WebDriverWait would help, but I've tried adjusting the wait time and am still receiving "None" too
To print the value of the src attribute you can use either of the following locator strategies:
Using css_selector:
print(driver.find_element_by_css_selector("body img[style*='webkit-user-select'][src^='https://haalsi.net/haalsi_pride2/custom/picture/index.php?id=']").get_attribute("src"))
Using xpath:
print(driver.find_element_by_xpath("//body//img[contains(#style, 'webkit-user-select') and starts-with(#src, 'https://haalsi.net/haalsi_pride2/custom/picture/index.php?id=')]").get_attribute("src"))
Ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "body img[style*='webkit-user-select'][src^='https://haalsi.net/haalsi_pride2/custom/picture/index.php?id=']"))).get_attribute("src"))
Using XPATH:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//body//img[contains(#style, 'webkit-user-select') and starts-with(#src, 'https://haalsi.net/haalsi_pride2/custom/picture/index.php?id=')]"))).get_attribute("src"))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You can find a relevant discussion in Python Selenium - get href value
I am trying to select the frame 'mainFrame'.
The page source is :
<frameset rows="89,*" frameborder="NO" border="0" framespacing="0">
<frame name="topFrame" scrolling="NO" noresize src="inc-webpage/b-topnav.asp">
<frameset rows="*,20" frameborder="NO" border="0" framespacing="0">
<frameset cols="175,*" frameborder="NO" border="0" framespacing="0">
<frame name="leftFrame" scrolling="AUTO" noresize src="inc-webpage/b-sidenav-3.asp">
<frame name="mainFrame" src="b-default.asp">
</frameset>
<frame name="bottomFrame" scrolling="NO" noresize src="inc-webpage/b-footer.asp">
</frameset>
</frameset>
<noframes>
The element I wish to select is in 'mainFrame'. Therefore my code is:
time.sleep(5)
driver.switch_to.frame("mainFrame");
driver.find_element_by_xpath("//a[contains(text(),'I Agree')]").click()
Yes. time.sleep() is not ideal so I'm just using it for the time being.
Here is the HTML for the element I wish to select within 'mainFrame':
<input type="button" value="I Agree"
class="btn" onmouseover="blueBtnOver(this)" onmouseout="blueBtnOut(this)"
onclick="javascript:location.href='b-3c-pLessonBooking.asp?limit=pl'" style="background: rgb(0, 102, 204);">
Currently getting this Error:
NoSuchFrameException(frame_reference)
selenium.common.exceptions.NoSuchFrameException: Message: mainFrame
I am an absolute beginner. The driver.find_element_by_xpath is probably wrong
Also why do some websites use frames//framesets while others use iframes and some don't use either?
Basically wait for the frame switch to it and then click the input with matching value I agree.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it("mainFrame"))
driver.find_element_by_xpath("//input[#value='I Agree']").click()
The desired element is nested within multiple <frame> elements so you have to:
Induce WebDriverWait for the parent frame to be available and switch to it.
Induce WebDriverWait for the child frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"frame[name='topFrame']")))
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"frame[name='mainFrame']")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.btn[value='I Agree']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//frame[#name='topFrame']")))
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//frame[#name='mainFrame']")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#class='btn' and #value='I Agree']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Reference
You can find a couple of relevant discussions in:
Ways to deal with #document under iframe
Switch to an iframe through Selenium and python
How To sign in to Applemusic With Python Using Chrome Driver With Selenium
I am using selenium and chrome to automate the clicks of a page with python.
I am getting stuck not being able to click on the following href link:
<a href="javascript:void(0)" class="addSuppData-trigger pts" data-target="edit_3-1" style="padding-right:6px;float:right;">
<i class="material-icons black-text tiny-small">edit</i></a>
I have tried using xpath, css, and linktext to no avail.
sample code:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="row-group-1"]/td[1]/div/div[2]/a/i' ))).click()
The goal is to click the pen, then select the item from the drop down.
screen shot of button
The second highlight line is the pen.
html tree
The element seems to be a dynamic element and to click on it you need to induce WebDriverWait for the desired element to be clickable and you can use either of the following solutions:
Using CSS_SELECTOR:
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.addSuppData-trigger.pts[data-target^='edit_']>i.material-icons.black-text.tiny-small"))).click()
Using XPATH:
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[#class='addSuppData-trigger pts' and starts-with(#data-target, 'edit_')]/i[#class='material-icons black-text tiny-small' and contains(., 'edit')]"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I am learning Selenium and trying to click the GO button:
https://speedtest.telstra.com/
<button class="button background-primary-hover text-primary" aria-label="start your speedtest">
<span style="font-size: unset; overflow-wrap: unset;">GO</span></button>
What are all possible Selenium methods to get that button clicked,
elem = driver.find_element_by_....???
I also would like to see what I found, so should print(elem.text) then be used?
As per the website https://speedtest.telstra.com/ the desired element is within an <iframe> so you need to induce WebDriverWait to switch to the <iframe> and then look out for the element and you can use the following solution:
Using XPATH:
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#class='speed-test' and #src='//telstra-nbn.speedtestcustom.com']")))
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//button[#class='button background-primary-hover text-primary']/span[contains(.,'GO')]"))).click()
Using CSS_SELECTOR:
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"//iframe.speed-test[src*='speedtestcustom']")))
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button.button.background-primary-hover.text-primary[aria-label='start your speedtest']>span"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You have to use xpath, there is xpath helper tool for chrome. You can install it.
button = driver.find_element_by_xpath("your xpath")
button.click()
Try this:
browser.find_element_by_class_name("button background-primary-hover text-primary").click()
Since it will select the element and click it.
I was trying to do a automated script to download some foreign exchange price historical data. These data are available at
https://www.dukascopy.com/swiss/english/marketwatch/historical/
However, if I just use Selenium to retrieve part of the website, I can't find this element in the website no matter how long I waited:
<iframe src="https://freeserv.dukascopy.com/2.0/?path=historical_data_feed/index&header=false&availableInstruments=l%3A&width=100%25&height=600&adv=popup" border="0" marginwidth="0" marginheight="0" frameborder="0" scrolling="no" width="100%" height="600"></iframe>
Apparently, the only thing selenium can read is the two elements ahead of the :
<script type="text/javascript">DukascopyApplet = {"type":"historical_data_feed","params":{"header":false,"availableInstruments":"l:","width":"100%","height":"600","adv":"popup"}};</script>
<script type="text/javascript" src="https://freeserv.dukascopy.com/2.0/core.js"></script>
Anyone know how to solve this problem? Many thanks.
Wait for the presence of the frame and then switch to it. Then, you can look for the elements inside:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox()
driver.get("https://www.dukascopy.com/swiss/english/marketwatch/historical/")
# wait for the frame to load and switch
wait = WebDriverWait(driver, 10)
iframe = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, ".mainContentBody iframe")))
driver.switch_to.frame(iframe)
# print out the bold radio button style texts
for item in driver.find_elements_by_css_selector("ul > li[data-group][data-instrument] strong"):
print(item.text)
driver.close()
Prints:
ADS.DE
ALV.DE
AUD/CAD
AUD/CHF
AUD/JPY
...
VOW3.DE
XAG/USD
XAU/USD
ZAR/JPY