Hi, I am able to switch between tabs, access all elements. I am unable to scroll in this iframe. Script executes without error. But scrolling is not happening. Please help. Code I am using is as follows.
# switching to iframe
iframe = self.browser.find_elements_by_tag_name('iframe')[0]
self.browser.switch_to_frame(iframe)
time.sleep(1)
#clicking tab 4
self.force_click('xpath=/html/body/div/md-content/md-tabs/md-tabs-wrapper/md-tabs-canvas/md-pagination-wrapper/md-tab-item[4]/span')
time.sleep(4)
#scrolling
elm = self.browser.find_elements_by_tag_name('html')
elm[0].send_keys(Keys.END)
HTML of the iframe is as follows.
<iframe id="widget-iframe" class="widget-iframe" frameborder="0" ap-onunload="vm.onFrameUnload()" ap-onload="vm.onFrameLoad()" ng-src="/apps/launchpad-view-widget/" src="/apps/launchpad-view-widget/">
<!DOCTYPE html>
<html class="ng-scope" ng-app="launchpadViewWidget">
<head>
<body>
</html>
</iframe>
I propose you to set window size bigger, so you wont have to scroll
self.browser.set_window_size(1920,1080)#(4096, 3112) <-4k resolution(if needed)
--
Edit:
Also can get the HTML code from browser and manage it with BeautifulSoup
html_source = browser.page_source
For python2.x i recomend
html_source = browser.page_source.encode('utf-8')
Then find the table that you want without care about scrolling.
I've run into this before. I couldn't see the values past the scroll area. The solution was to get the javascript object that populates the table via JSON. You can do this using the javascriptexecutor: driver.execute_script
Hope that helps!
Related
I am trying to get the source code of a particular site with the help of Selenium with:
Python code:
driver.page_source
But it returns it after it has been encoded.
The raw file:
<html>
<head>
<title>AAAAAAAA</title>
</head>
<body>
</body>
When press 'View page source' inside Chrome, I saw the correct source raw without encoding.
How can this be achieved?
You can try using Javascript instead of Python builtin code to get the page source.
javascriptPageSource = driver.execute_script("return document.body.outerHTML;")
I am trying to scrape an html table which has two frames. When switching to the first one, the code works well but when switching to default and then to the second frame, I can´t get the full html code.
driver = webdriver.Chrome('/Users/Administrador/Documents/chromedriver')
main_url = 'https://www.justiciacordoba.gob.ar/Estatico/JEL/Escrutinios/ReportesEleccion20190512/default.html'
driver.get(main_url)
#This works fine:
driver.switch_to.frame("topFrame")
# This doesnt:
driver.switch_to.default_content()
driver.switch_to.frame('mainFrame')
page = driver.page_source
page
Output:
'<html><head></head><body></body></html>'
That is the full page!
<frame src="about:blank" name="mainFrame" align="center">
#document
<html>
<head></head>
<body></body>
</html>
</frame>
Click with the right mouse button, select "inspect" or "inspect element", and you'll see in the Elements tab of the Development window that that is all the frame has.
In Chrome, you can also press Ctrl+Shift+I and you'll get directly to this tab.
Seems you are seeing the right behavior. When the WebDriver's focus is within the <frame> with name as topFrame, unless you select values from the <select> elements and initiate a search, the elements within the <frame> with name as mainFrame aren't redendered. Hence you see the following behaviour:
Code Block:
driver.get('https://www.justiciacordoba.gob.ar/Estatico/JEL/Escrutinios/ReportesEleccion20190512/default.html')
driver.switch_to.frame("topFrame")
driver.switch_to.default_content()
driver.switch_to.frame('mainFrame')
print(driver.page_source)
Console Output:
<html><head></head><body></body></html>
In this case if you still want to extract the full HTML from the Top Level Content you can switch to the default_content() as follows:
Code Block:
driver.get('https://www.justiciacordoba.gob.ar/Estatico/JEL/Escrutinios/ReportesEleccion20190512/default.html')
driver.switch_to.frame("topFrame")
driver.switch_to.default_content()
driver.switch_to.frame('mainFrame')
print(driver.page_source)
driver.switch_to.default_content()
print(driver.page_source)
Console Output:
<html><head></head><body></body></html>
<html><head></head><frameset rows="190,*" cols="*" framespacing="0" frameborder="NO" border="0" id="fset">
<frame src="Index.html" name="topFrame" scrolling="NO" cd_frame_id_="887435be8ea834d3aec3a905bb2f8019">
<frame src="about:blank" name="mainFrame" align="center" cd_frame_id_="a1abd873a60c8db45dc83e5334321cbc">
</frameset><noframes></noframes>
</html>
<iframe id="xyz" src="https://www.XXXXXX.com/" allowfullscreen="yes" style="width: 100%; height: 100%;">
#document
<!DOCTYPE html>
<html>...</html> // a whole new HTML document
</iframe>
I tried the below code, but I am not able to access the inner HTML content. Please guide.
docu=driver.find_element_by_xpath("//*[#id='asdfghg']").find_element_by_tag_name("iframe")
print(docu.get_attribute("innerHTML"))
Not sure if you are particular to an element or need full source , check and upvote if below lines can help you...
from selenium import webdriver
driver = webdriver.Chrome(executable_path="C:\\driver\\chromedriver.exe")
driver.get('https://yoururl')
# HTML Source before getting in frame
print(driver.page_source)
# Switch to Frame
driver.switch_to.frame('yourframeID')
# HTML Source after getting in frame
print(driver.page_source)
I see that my selenium cannot execute codes except to launch Chrome.
I don't know why my selenium is not working. It just open the browser (Chrome) with the URL and then doing nothing even to maximize the window, not even inserting the form.
Is there anything wrong with my code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import re, time, csv
driver = webdriver.Chrome("C:\\Users\\Ashraf%20Misran\\Installer\\chromedriver.exe")
driver.get("file:///C:/Users/Ashraf%20Misran/Devs/project-html/learning-html/selenium sandbox.html")
driver.maximize_window()
username = driver.find_element_by_xpath(".//input")
username.click()
username.send_keys("000200020002")
The page I opened is coded as below:
<!DOCTYPE html>
<html>
<head>
<title>Sandbox</title>
</head>
<body>
<form>
<input type="text" name="username">
</form>
</body>
</html>
I think the problem is with web-page, you are trying to open. Would suggest to try first with simple test, like Open Google page, enter something in search field. With this you will be able to verify, if you correctly implemented driver initialization.
Update: try to use this css selector: input[name='username'], if page is loaded correctly, then you have a problem with your web element selector.
I think, there is a problem with using relative xpath locator. Please try that one:
username = driver.findElement(By.xpath("//input"))
I had this simple login script to facebook that used to work perfectly until about a month ago. But yesterday when I tried running it again I got this dummy page:
<html xmlns="http://www.w3.org/1999/xhtml">
<head></head>
<body><pre style="word-wrap: break-word; white-space: pre-wrap;">
</pre>
<iframe name="chromedriver dummy frame" src="about:blank"></iframe>
</body>
</html>
I guess they've added some new detections. Is there a way to avoid those?
This is my simplified code:
browser = webdriver.Chrome(executable_path=path, service_args=['--ignore-ssl-errors=true', '--ssl-protocol=TLSv1'])
browser.get("https://www.facebook.com/")
for line in browser.page_source.split('\n'):
print line
I have a similar problem which is not Facebook but our developing pages.
I might be ssl problem. (which might be solved --ignore-ssl-... option.)
Mostly, This is waiting problem.
The Selenium bot captures whole HTML PAGE before the server print out their contexts.
Thus, it might be solved, using same wait options (See this)
If there is some unique ID html elements, please insert following codes:
wait = WebDriverWait(driver, 5)
element = wait.until(EC.visibility_of_element_located((By.ID, 'unique')))