Selenium cannot get iframe - python

I am trying to retrieve the Job Description and Job Requirements from this link:
https://www.jobsbank.gov.sg/ICMSPortal/portlets/JobBankHandler/SearchDetail.do?id=JOB-2016-0008238
using selenium. This is my code:
driver = webdriver.Firefox()
url = "https://www.jobsbank.gov.sg/ICMSPortal/portlets/JobBankHandler/SearchDetail.do?id=JOB-2016-0008238"
driver.get(url)
iframes = driver.find_elements_by_xpath('//*[#id="divMainJobDescription"]')
print len(iframes)
driver.switch_to_frame(iframes)
print driver.page_source
I get the following error:
Traceback (most recent call last): File "D:\scraper.py", line 79, in
main() File "D:\Abigail MTI Projects\Singapore Skills\JobsBankScraper\jobsbank14012016_abi.py", line 74, in main
driver.switch_to_frame(iframes) File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py",
line 537, in switch_to_frame
self._switch_to.frame(frame_reference) File "C:\Python27\lib\site-packages\selenium\webdriver\remote\switch_to.py",
line 67, in frame
self._driver.execute(Command.SWITCH_TO_FRAME, {'id': frame_reference}) File
"C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py",
line 201, in execute
self.error_handler.check_response(response) File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py",
line 181, in check_response
raise exception_class(message, screen, stacktrace) NoSuchFrameException: Message: Unable to locate frame: [object Object]
Stacktrace:
at FirefoxDriver.prototype.switchToFrame (file:///c:/users/mti_wl~1/appdata/local/temp/tmpwelh3x/extensions/fxdriver#googlecode.com/components/driver-component.js:10745)
at DelayedCommand.prototype.executeInternal_/h (file:///c:/users/mti_wl~1/appdata/local/temp/tmpwelh3x/extensions/fxdriver#googlecode.com/components/command-processor.js:12551)
at DelayedCommand.prototype.executeInternal_ (file:///c:/users/mti_wl~1/appdata/local/temp/tmpwelh3x/extensions/fxdriver#googlecode.com/components/command-processor.js:12556)
at DelayedCommand.prototype.execute/< (file:///c:/users/mti_wl~1/appdata/local/temp/tmpwelh3x/extensions/fxdriver#googlecode.com/components/command-processor.js:12498)

iframes is list, you need to switch to a single frame.
driver.find_elements return list while driver.find_element return single web element (note the 's' in element/s).
In addition, divMainJobDescription is not iframe, I believe you are looking for frameJobDescription
iframe = driver.find_element_by_id("frameJobDescription")

You need to target the iframe, switch to that frame, and then you can start identifying other elements, like divs, which exist inside that frame. You need to specify the iframe tag itself and not an element inside the iframe.
So, instead of targeting a div immediately inside the frame like
iframes = driver.find_elements_by_xpath('//*[#id="divMainJobDescription"]')
target the iframe tag, then switch. (You can use the singular find_element_by_xpath if you want only one iframe.)
iframe = driver.find_element_by_xpath('//iframe[#id="frameJobDescription"]')
driver.switch_to_frame(iframe)
job_description = driver.find_element_by_xpath('//*[#id="divMainJobDescription"]')
Bonus: You can use other methods to target the iframe, such as finding the element by ID instead of writing the xpath, if you are only looking at IDs.
iframe = driver.find_element_by_id('frameJobDescription')
If you need to get details in BOTH iframes during your script, you will need to take turns switching between frames.

Related

Not able to click Checkbox Element using Python Selenium integration

I am trying to iterate over pages having a program List and then I have to select checkbox defined in an image below in order to get the run length of a loop on-page.
But when I am trying to navigate over pages via the next button it will stop clicking on checkboxes and throws me below error.
I have tried with xpath,css selector and id method But all get failed.
DOM element SS along with checkbox highlighted
Line of Code Used-
CheckBoxclick = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#workflow_table > thead > tr > th.first.checkbox-column.checkbox > input"))).click()
Error:-
File "D:/Vipul Garg Backup/XXXX/TestingPardotExportProgramsData.py", line 74, in <module>
ProgramsPathlist = getDataForPrograms(RecordsonOnePage,totalnofpages)
File "D:/Vipul Garg Backup/XXXX/TestingPardotExportProgramsData.py", line 63, in getDataForPrograms
RecordsonOnePage = getrecordsoNpage()
File "D:/Vipul Garg Backup/XXXX/TestingPardotExportProgramsData.py", line 42, in getrecordsoNpage
CheckBoxclick = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#workflow_table > thead > tr > th.first.checkbox-column.checkbox > input"))).click()
File "C:\Users\Vipul.Garg\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "C:\Users\Vipul.Garg\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "C:\Users\Vipul.Garg\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\Vipul.Garg\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
(Session info: chrome=80.0.3987.122)
But Above Line of Code is Working Intermetantly and throwing error as above.
Any suggestions for this line of code replacement as why I am getting such inconsistency.
You are using ID of the Table and checkbox is inside that table, so you need to pass locator of the check box.
As there is no id defined for the checkbox, so using xpath to reach the locator like - //input[#type='checkbox']

How to edit chromes search and homepage with selenium/python?

I am trying to edit chrome browsers search and homepage using selenium/python. After navigating to chrome://settings/searchEngines and targeting the 'add' button with the ID 'addSearchEngine', I get an error when I run a .click function. How do I target this element correctly, or is there another way to update chromes search/startpage with python?
I'm guessing this element is trapped inside an iframe but I'm unable to find one on the page using the dev tools, xpath noted the following about the absolute xpath: "It might be a child of iframe from different src & it is not supported currently."
from selenium import webdriver
driver = webdriver.Chrome()
driver.set_page_load_timeout(10)
driver.get("chrome://settings/searchEngines")
driver.find_element_by_id("addSearchEngine").click()
Traceback (most recent call last):
File "C:/Users/Jonathan/PycharmProjects/test_project/test_project/Main.py", line 20, in <module>
driver.find_element_by_id("addSearchEngine").click()
File "C:\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 360, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "C:\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
'value': value})['value']
File "C:\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Python37-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="addSearchEngine"]"}
(Session info: chrome=75.0.3770.142)
chrome://settings/searchEngines has Shadow DOM elements . You will need to use the driver.execute_script() to get handle of the shadowRoot element and ultimately get to the 'addSearchEngine' element.
Example python : shadowRoot python
Example Java : For chrome://downloads/ shadowRoot java
Refer to this post for detailed explanation.
In your case you can do the below.
url = "chrome://settings/searchEngines"
driver.get(url)
addButton = driver.execute_script("return document.querySelector('settings-ui')"
".shadowRoot.querySelector('#main')"
".shadowRoot.querySelector('settings-basic-page.showing-subpage')"
".shadowRoot.querySelector('settings-search-page')"
".shadowRoot.querySelector('settings-search-engines-page')"
".shadowRoot.querySelector('#addSearchEngine')")
addButton.click()
Screenshot:

Selenium can't click on a element in an iframe

So im trying to click a button in an iframe but it just errors saying that the button can't be found.
recaptcha = driver.find_element_by_tag_name('iframe')
driver.switch_to.frame(recaptcha)
submit1 = driver.find_element_by_id('recaptcha-anchor')
actions = ActionChains(driver)
actions.move_to_element(submit1)
actions.click(submit1).perform()
time.sleep(3)
driver.find_element_by_id('recaptcha-audio-button').click()
On the last line I try to get an element by id recaptcha-audio-button.
As I said above it just errors with that it can't find it.
The website that I am trying to do this with is: https://patrickhlauke.github.io/recaptcha/
Why can't I click this button?
Thanks, Ira.
EDIT
Here is the error
Traceback (most recent call last):
File "main.py", line 62, in <module>
driver.find_element_by_id('recaptcha-audio-button').click()
File "/home/ira/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 269, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "/home/ira/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 752, in find_element
'value': value})['value']
File "/home/ira/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "/home/ira/.local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: An element could not be located on the page using the given search parameters.
Element is in different iframe so you can do:
switch to main window
find all iframes and switch to second iframe
find and click your button
Code:
# go back to main window
driver.switch_to.default_content()
# find all frames and switch to second frame
all_frames = driver.find_elements_by_tag_name('iframe')
driver.switch_to.frame(all_frames[1])
driver.find_element_by_id('recaptcha-audio-button').click()
Minimal working code so everyone can copy and test it
import selenium.webdriver
import time
driver = selenium.webdriver.Firefox()
driver.get('https://patrickhlauke.github.io/recaptcha/')
recaptcha = driver.find_element_by_tag_name('iframe')
driver.switch_to.frame(recaptcha)
submit1 = driver.find_element_by_id('recaptcha-anchor')
actions = selenium.webdriver.ActionChains(driver)
actions.move_to_element(submit1)
actions.click(submit1).perform()
time.sleep(3)
# go back to main window
driver.switch_to.default_content()
# find all frames and switch to second frame
all_frames = driver.find_elements_by_tag_name('iframe')
driver.switch_to.frame(all_frames[1])
driver.find_element_by_id('recaptcha-audio-button').click()

Python Selenium cannot find an element on a page by xpath

I am using python and selenium to test some things with fantasy football. Here is my code so far (I just started).
import time
from selenium import webdriver
driver = webdriver.Chrome('C:\\Users\\202300Fontenot\\Desktop\\3\\chromedriver.exe')
driver.get('http://games.espn.com/ffl/signin?redir=http%3A%2F%2Fgames.espn.com%2Fffl%2Fclubhouse%3FseasonId%3D2018%26leagueId%3D49607%26teamId%3D4');
driver.implicitly_wait(10)
time.sleep(10)
search_box = driver.find_element_by_xpath('//*[#id="did-ui-view"]/div/section/section/form/section/div[1]/div/label/span[2]/input')
search_box.send_keys('email#icloud.com')
search_box.submit()
time.sleep(5)
driver.quit()
This just tries to enter an email address into the box. I am getting this error every time:
Traceback (most recent call last):
File "C:\Users\202300Fontenot\Desktop\3\ESPN.py", line 8, in <module>
search_box = driver.find_element_by_xpath('//*[#id="did-ui-view"]/div/section/section/form/section/div[1]/div/label/span[2]/input')
File "C:\Users\202300Fontenot\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 393, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "C:\Users\202300Fontenot\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 966, in find_element
'value': value})['value']
File "C:\Users\202300Fontenot\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 320, in execute
self.error_handler.check_response(response)
File "C:\Users\202300Fontenot\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id="did-ui-view"]/div/section/section/form/section/div[1]/div/label/span[2]/input"}
(Session info: chrome=69.0.3497.92)
(Driver info: chromedriver=2.42.591088 (7b2b2dca23cca0862f674758c9a3933e685c27d5),platform=Windows NT 10.0.16299 x86_64)
Thanks for your help.
Your problem is that the input you are trying to select exists inside of an iframe. You must first tell the driver to switch to the iframe and then execute the xpath selection.
driver.switch_to.frame("disneyid-iframe")
driver.find_element_by_xpath("//input[#type='email']").send_keys('email#icloud.com')
First of all you need to switch to your iframe using its name:
disneyid-iframe
Also if XPATH is not the only choice, you may use the CSS Selector:
div.field-username-email input[type='email']
This will get the driver to find & fill your field.
Using XPath in python You need to provide the string for
driver.find_element_by_xpath
Example Code below
("//*[#id=\"did-ui-view\"]/div/section/section/form/section/div[1]/div/label/span[2]/input")
Please refer below document regarding String
String Gude first Para

NoSuchElement Exception using find_element_by_link_text when implicitly_wait doesn't work?

New to python and Selenium and programming for that matter.
I am trying to automate selenium to hit a specific link. In this case, I want it to hit the link associated with the link text "B":
<li>B
on this website:
http://www.lw.com
I am using this code:
def get_single_link_using_find_elements_by_link_name(url, link_name):
driver = webdriver.Firefox()
driver.get(url)
driver.implicitly_wait(10)
time.sleep(20)
element = driver.find_element_by_link_text(link_name)
element.click()
I added some wait conditions, because I thought the problem might have been a rendering problem, but they didnt help.
I am getting the following error:
Traceback (most recent call last):
File "C:\Python27\programs\selenium commands.py", line 50, in <module>
get_single_link_using_find_elements_by_link_name(url, link_name)
File "C:\Python27\programs\selenium commands.py", line 47, in get_single_link_using_find_elements_by_link_name
element = driver.find_element_by_link_text(link_name)
File "C:\Python27\lib\site-packages\selenium-2.25.0-py2.7.egg\selenium\webdriver\remote\webdriver.py", line 237, in find_element_by_link_text
return self.find_element(by=By.LINK_TEXT, value=link_text)
File "C:\Python27\lib\site-packages\selenium-2.25.0-py2.7.egg\selenium\webdriver\remote\webdriver.py", line 671, in find_element
{'using': by, 'value': value})['value']
File "C:\Python27\lib\site-packages\selenium-2.25.0-py2.7.egg\selenium\webdriver\remote\webdriver.py", line 156, in execute
self.error_handler.check_response(response)
File "C:\Python27\lib\site-packages\selenium-2.25.0-py2.7.egg\selenium\webdriver\remote\errorhandler.py", line 147, in check_response
raise exception_class(message, screen, stacktrace)
NoSuchElementException: Message: u'Unable to locate element: {"method":"link text","selector":"B"}'
Oddly enough, the same code WORKS on the following url, which is part of the same site:
http://www.lw.com/people?searchIndex=A
Any ideas?
Your code works for me, when I pass it a page that has the alphabetic listing index. This means that you're passing the wrong variables to the function-- the page that you're passing in, doesn't have a link named 'B', plain and simple.
You can check whether or not the alphabetic thing is on the page by calling driver.find_element_by_id("IndexControl1"). IndexControl1 is the name of the id in which the alphabetic thing is contained.
alphabet = driver.find_element_by_id("IndexControl1")
link_b = alphabet.find_element_by_link_text("B")
Incidentally, something else to watch out for is that if you're already on the page with "B" selected, e.g. http://www.lw.com/people?searchIndex=B&esmode=1, the letter B does not show up as a link and you will end up with a NoSuchElementException in this case, as well.
I think that covers pretty much every case where NoSuchElementException could pop up. Good luck.

Categories

Resources