I have to use the selenium for my project.when i am doing using selenium.i am giving the at each step like this
time.sleep(6)
i am giving the sufficient interval for loading particular page, but still I am facing the below error at somepoint.
Mostly i am getting the exceptions like
Regular Exceptions:
Exception: Message: u'Unable to locate element: {"method":"xpath","selector":"//textarea[#style=\\"overflow-y: hidden; height: 50px; width: 300px;\\"]"}' ; Stacktrace: Method WebDriverError threw an error in file:///tmp/tmpBxGp2P/extensions/fxdriver#googlecode.com/resource/modules/atoms.js
Traceback (most recent call last):
File "crawl_optimize.py", line 206, in main
word_query = browser.find_element_by_xpath('//textarea[#style="overflow-y: hidden; height: 50px; width: 300px;"]')
File "/usr/local/lib/python2.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 210, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "/usr/local/lib/python2.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 643, in find_element
{'using': by, 'value': value})['value']
File "/usr/local/lib/python2.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 153, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python2.6/dist-packages/selenium/webdriver/remote/errorhandler.py", l ine 147, in check_response
raise exception_class(message, screen, stacktrace)
I am also getting No Such Implementation Error because of WebDriverWait().Until():
NoSuchElementException: Message: u'Unable to locate element: { "method":"xpath","selector":"//textarea[#style=\\"overflow-y: hidden; height: 50px; width: 300px;\\"]"}' ; Stacktrace: Method WebDriverError threw an error in file:///tmp/tmpBxGp2P/extensions/fxdriver#googlecode.com/resource/modules/atoms.js
keyword : balance transfer ad pos : 2
Exception: Message: None
Traceback (most recent call last):
File "crawl_optimize.py", line 228, in main
start_cpc,end_cpc = set_initial_cpc(initial_max_cpc,ad_position)
File "crawl_optimize.py", line 95, in set_initial_cpc
ep = crawl_position(cost)
File "crawl_optimize.py", line 82, in crawl_position
WebDriverWait(browser, 10).until(lambda driver : driver.find_element_by_link_text(word.lower()))
File "/usr/local/lib/python2.6/dist-packages/selenium/webdriver/support/wait.py", line 55, in until
raise TimeoutException()
TimeoutException: Message: None
What is the way to get rid of such exceptions?
Could be 2 things.
Your element hasn't finished loading within the allocated 6 seconds
Your locator is malformed. This looks suspect ...
{ "method":"xpath","selector":"//textarea[#style=\"overflow-y: hidden; height: 50px; width: 300px;\"]"}'
Can you check that the locator returns what you expect using FireFinder or similar?
If the locator is ok, you could try using WebDriverWait & ExpectedCondition to confirm the page has finished loading rather than gambling that you page will always load in 6s
Related
Here is the code which I use to open the new tab using Selenium Python,
self.driver.execute_script("window.open(arguments[0])", self.listing_url)
self.driver.switch_to.window(self.driver.window_handles[1])
time.sleep(10)
and it works perfectly. But I am planning to use WebDriverWait instead of time.sleep
self.driver.execute_script("window.open(arguments[0])", self.listing_url)
self.driver.switch_to.window(self.driver.window_handles[1])
WebDriverWait(self.driver, 20).until(EC.presence_of_element_located((By.XPATH, '//div[#class="honbun"]')))
and when I am using it, it's throwing an error. Here is the error log:
Traceback (most recent call last):
File "D:\tender_scraper\engine\Spider.py", line 308, in parse_index_page
index_handle = self.load_listing_page(listing, i, index_listing)
File "D:\tender_scraper\spiders\gunmapref_listing_spider.py", line 298, in load_listing_page
WebDriverWait(self.driver, 20).until(EC.presence_of_element_located((By.XPATH, '//div[#class="honbun"]')))
File "D:\scraper\lib\site-packages\selenium\webdriver\support\wait.py", line 78, in until
value = method(self._driver)
File "D:\scraper\lib\site-packages\selenium\webdriver\support\expected_conditions.py", line 64, in _predicate
return driver.find_element(*locator)
File "D:\scraper\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 1251, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "D:\scraper\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 430, in execute
self.error_handler.check_response(response)
File "D:\scraper\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: missing or invalid columnNumber
(Session info: headless chrome=104.0.5112.48)
Any idea what's wrong with it & how to solve it? Previously, I used chrome 103 and changed it to 104 but still got the same error. It seems the error only occurs when changing into the new tab.
I am trying to click a button but I am having an error when finding the element. In general I am trying to refresh a page until the link to join the waiting list goes live. When I run the script the webpage pulls up but does not refresh or click the join waitlist button when it is live.
I've tried finding element by class, link text, xpath, each time I get an error saying the element wasn't found.
code
while not joinList:
try:
addToCartBtn = addButton = driver.find_element_by_class_name("disabled")
print("Button isnt ready yet.")
time.sleep(2)
driver.refresh()
except:
addToCartBtn = addButton = driver.find_element_by_class_name("signup-btn")
print("Join")
addToCartBtn.click()
joinList = True
Where the element for the disabled join waitlist button is:
button _ngcontent-yur-c149="" disabled="" class="signup-btn ng-tns-c149-6 ng-star-inserted" style="background-color: rgb(0, 59, 153);"> Waitlist is currently closed </button
and the element for the enabled waitlist is:
button _ngcontent-yur-c149="" class="signup-btn ng-tns-c149-4 ng-star-inserted" tabindex="0" style="background-color: rgb(0, 59, 153);"> Join waitlist </button
The error I get is:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Python\Pycharm\MCattempt2\Main test folder\Test 1.py", line 20, in <module>
addToCartBtn = addButton = driver.find_element_by_class_name("signup-btn")
File "C:\Python\Pycharm\MCattempt2\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 564, in find_element_by_class_name
return self.find_element(by=By.CLASS_NAME, value=name)
File "C:\Python\Pycharm\MCattempt2\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Python\Pycharm\MCattempt2\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Python\Pycharm\MCattempt2\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".signup-btn"}
(Session info: chrome=88.0.4324.190)
Any advice on what I am doing wrong?
I am trying to return false if an element exist, but I don't want to end the program if the element doesn't exist.
My problem is that when Selenium doesn't find the element, it closes all but I don't want to close it, I want to run the rest of the code.
Is it possible?
My Python code:
if driver.find_element_by_xpath("//a[#class='btn btn-primary im_edit_forward_btn disabled']") != 0 :
print("Break here")
return False
print("Running the rest")
encaminhar = driver.find_element_by_xpath("//a[#class='btn btn-primary im_edit_forward_btn']")
encaminhar.click()
Output:
Traceback (most recent call last):
File "webscraping.py", line 94, in <module>
if forwardMessages():
File "webscraping.py", line 57, in forwardMessages
if driver.find_element_by_xpath("//a[#class='btn btn-primary im_edit_forward_btn disabled']") != 0 :
File "C:\Users\gusta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\selenium\webdriver\remote\webdriver.py", line 394, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "C:\Users\gusta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\selenium\webdriver\remote\webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Users\gusta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\gusta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//a[#class='btn btn-primary im_edit_forward_btn disabled']"}
You can catch the exception.
from selenium.common.exception import NoSuchElementException
try:
driver.find_element_by_xpath("//a[#class='btn btn-primary im_edit_forward_btn disabled']")
except NoSuchElementException:
pass
I need to get text from emails and save for send in webwhatsapp, but the text is in a iframe and returns a error when I try to get him
Thats the html code:
<div id="mailContent" class="mail-content" style="height: 724px;">
<div id="phishingLinkScanContent"></div>
<iframe id="bodyMailViewer"
class="body-mail-viewer" style="height: 719px;"
cd_frame_id_="18a0312a828e12092646c9a6b1ca58c4">
#document
<html><head></head>
<body>MetaTrader email test -> cilabassa#terra.com.br<br><div id="ozoneAttachments"></div></body>
</html>
</iframe>
</div>
Thats my code:
driver.switch_to.frame(driver.find_element_by_tag_name("iframe"))
texto = driver.find_element_by_id('bodyMailViewer').text
Thats the error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 360, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
'value': value})['value']
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\ProgramData\Anaconda3\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="bodyMailViewer"]"}
(Session info: chrome=75.0.3770.142)
First switch to the iframe and then locate the body tag under iframe and finally get the text from body tag. Try the below code.
driver.switch_to.frame(driver.find_element_by_tag_name("iframe"))
textInMail = driver.find_element_by_tag_name('body').text
I am having an issue with Selenium and not being able to switch to a frame. I have tried using the xpath of iframe but it replies that the element was not found. This was the code I was using:
driver.switch_to_frame(driver.find_element_by_xpath('//[#id="VF"]'))
I have also tried using a css-selector but it gives me the same result. I suspect that the div element that holds the iframe that I want to select is causing this issue but I'm not sure. Here's what that div element looks like:
<div id="dee_0" class="ar BAResults" fframe='<iframe title="SRC" src="javascript:"<HTML></HTML>"" onload="DVFol()"></iframe>
Any help would be greatly appreciated.
edit
this is the iframe I'm trying to target
<iframe name="VF" title="SRC" id="VF" src='javascript:"<HTML></HTML>"' frameborder="0" scrolling="auto" style="left: 0px; top: 0px; width: 830px; height: 490px; background-color: transparent;" allowtransparency="true" onload="F(0,302899400).ol()">
stack trace
Traceback (most recent call last):
File "C:/Users/kthao.ISC/PycharmProjects/AutoComplete/test2.py", line 32, in
<module>
chrome.switch_to_frame(chrome.find_element_by_xpath('//*
[#id="VF"]'))
File "C:\Users\kthao.ISC\Desktop\AutoComplete\lib\site-
packages\selenium\webdriver\remote\webdriver.py", line 368, in
find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "C:\Users\kthao.ISC\Desktop\AutoComplete\lib\site-
packages\selenium\webdriver\remote\webdriver.py", line 858, in find_element
'value': value})['value']
File "C:\Users\kthao.ISC\Desktop\AutoComplete\lib\site-
packages\selenium\webdriver\remote\webdriver.py", line 311, in execute
self.error_handler.check_response(response)
File "C:\Users\kthao.ISC\Desktop\AutoComplete\lib\site-
packages\selenium\webdriver\remote\errorhandler.py", line 237, in
check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element:
Unable to locate element: {"method":"xpath","selector":"//*
[#id="VF"]"}
The method to switch to an iframe is switch_to.frame() and the <iframe> tag contains the attribute title as SRC. So you can use the following line of code :
driver.switch_to.frame(driver.find_element_by_xpath("//iframe[#title='SRC']"))