I'm getting an "Unable to locate element" exception while running the below code. My expected output is "Button not found. This should not happen."
My code:
driver = webdriver.Chrome()
driver.get(byPassUrl)
driver.find_element_by_xpath("//*[#id='__next']/div/div[1]/div[2]/div/div[1]/div[2]/div/div[1]/div[1]/span/button").click()
time.sleep(3)
emails = driver.find_element_by_css_selector("input[type='email']")
emails.send_keys("mymail.com")
driver.find_element_by_css_selector("input[type='password']").send_keys("abcxyz")
driver.find_element_by_xpath('//button[text()="Log In"]').click()
time.sleep(3)
def delay(waiting_time=5):
driver.implicitly_wait(waiting_time)
audioBtnFound = False
def bypasscaptcha():
try:
# switch to recaptcha audio control frame
driver.switch_to.default_content()
frames = driver.find_element_by_xpath("/html/body/div[3]/div[2]").find_elements_by_tag_name("iframe")
driver.switch_to.frame(frames[0])
delay()
driver.find_element_by_id("recaptcha-audio-button").click()
# get the mp3 audio file
src = driver.find_element_by_id("audio-source").get_attribute("src")
print("[INFO] Audio src: %s" % src)
# download the mp3 audio file from the source
urllib.request.urlretrieve(src, os.path.normpath(os.getcwd() + "\\sample.mp3"))
delay()
# load downloaded mp3 audio file as .wav
try:
sound = pydub.AudioSegment.from_mp3(os.path.normpath(os.getcwd() + "\\sample.mp3"))
sound.export(os.path.normpath(os.getcwd() + "\\sample.wav"), format="wav")
sample_audio = sr.AudioFile(os.path.normpath(os.getcwd() + "\\sample.wav"))
except Exception:
print("[ERR] Please run program as administrator or download ffmpeg manually, "
"http://blog.gregzaal.com/how-to-install-ffmpeg-on-windows/")
# translate audio to text with google voice recognition
r = sr.Recognizer()
with sample_audio as source:
audio = r.record(source)
key = r.recognize_google(audio)
print("[INFO] Recaptcha Passcode: %s" % key)
delay()
# key in results and submit
driver.find_element_by_id("audio-response").send_keys(key.lower())
driver.find_element_by_id("audio-response").send_keys(Keys.ENTER)
except Exception as e:
print(e)
try:
driver.switch_to.default_content()
frames = driver.find_element_by_xpath("/html/body/div[3]/div[2]").find_elements_by_tag_name("iframe")
driver.switch_to.frame(frames[0])
delay()
driver.find_element_by_id("recaptcha-audio-button").click()
audioBtnFound = driver.find_element_by_id("recaptcha-audio-button").is_displayed()
except Exception as e:
print("audioBtn not found")
num = 1
if audioBtnFound:
try:
while True:
print(num)
num += 1
bypasscaptcha()
delay()
#if eye button found do bypasscaptcha
try:
driver.find_element_by_id("recaptcha-image-button").click()
print("Fail captcha")
delay()
bypasscaptcha()
except Exception as e:
print("Pass captcha")
audioBtnFound = False
break
except Exception as e:
print(e)
print('Caught. Need to change proxy now')
else:
print('Button not found. This should not happen.')
If I run the above code I get "Unable to Locate Element Exception".I know that when i log in and no need to bypass captcha this element can not be found "
driver.find_element_by_xpath("/html/body/div[3]/div[2]").find_elements_by_tag_name("iframe")
but how can I deal with this problem.
Thank you so much
iframes tags can be directly found using their tag name or //iframe xpath. you do not have to look for them after /html/body/div[3]/div[2]
Directly try :
all_iframes = find_elements_by_tag_name("iframe")
and now you have the list of iframe on the page.
if you know the number, then you can switch it like the way you have switched in your code.
it may be that selenium does not see the tag it needs, or you have registered an incorrect search. Set the search by id or css_selector. Don't do everything in one line, first find
driver.find_element_by_xpath("/html/body/div[3]/div[2]")
then look for
driver.find_elements_by_tag_name("iframe")
Related
I'm trying to make a little Facebook group auto post script using python and selenium
First of all I'm uploading 3/4 photos using the following code:
l = driver.find_elements_by_tag_name('input')
for g in l:
print(g)
try:
if g == driver.find_element_by_xpath("//input[#type='file']"):
print("Found")
logging.debug("Found input for image uploading")
g.send_keys(
'/var/www/html/v1/insta-post/AutoPostFB/images/0.jpg \n/var/www/html/v1/insta-post/AutoPostFB/images/1.jpg \n/var/www/html/v1/insta-post/AutoPostFB/images/2.jpg')
print("File/s Uploaded")
logging.debug("Images uploaded")
time.sleep(5)
# break
except:
print("Element not found after upload")
logging.debug("Element for upload not found")
Then I push the post text, since the textbox is already on focus
actions = ActionChains(driver)
actions.send_keys(info)
actions.perform()
time.sleep(1)
The issue is that every space in the info variable is abnormally converted into a return key
so if the text is like
info = "HEI HOW'S IS GOING?"
what I get in the textbox is
IS
GOING
HEY
HOW'S
Someone can help me out? I've tried like everything
I've compiled this code to perform an iteration of downloads from a webpage which has multiple download links. Once the download link is clicked, the webpage produces a webform which has to be filled and submitted for the download to start. I've tried running the code and face issue in 'try'& 'except' block code (Error: Too broad exception clause) and towards the end there is an error associated with the 'submit' (Error: method submit maybe static) both of these subsequently result in 'SyntaxError: invalid syntax '. Any suggestions / help will be much appreciated. Thank you.
import os
from selenium import webdriver
fp = webdriver.FirefoxProfile()
fp.set_preference("browser.download.folderList",2)
fp.set_preference("browser.download.manager.showWhenStarting",False)
fp.set_preference("browser.download.dir", os.getcwd())
fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "application/x-msdos-program")
driver = webdriver.Firefox(firefox_profile=fp)
driver.get('http://def.com/catalog/attribute')
#This is to find the download links in the webpage one by one
i=0
while i<1:
try:
driver.find_element_by_xpath('//*[#title="xml (Open in a new window)"]').click()
except:
i=1
#Once the download link is clicked this has to fill the form for submission which fill download the file
class FormPage(object):
def fill_form(self, data):
driver.find_element_by_xpath('//input[#type = "radio" and #value = "Non-commercial"]').click()
driver.find_element_by_xpath('//input[#type = "checkbox" and #value = "R&D"]').click()
driver.find_element_by_xpath('//input[#name = "name_d"]').send_keys(data['name_d'])
driver.find_element_by_xpath('//input[#name = "mail_d"]').send_keys(data['mail_d'])
return self
def submit(self):
driver.find_element_by_xpath('//input[#value = "Submit"]').click()
data = {
'name_d': 'abc',
'mail_d': 'xyz#gmail.com',
}
FormPage().fill_form(data).submit()
driver.quit()
Actually you have two warnings and a error:
1 - "Too broad exception" this is a warning telling you that you should except espefic errors, not all of them. In your "except" line should be something like except [TheExceptionYouAreTreating]: an example would be except ValueError:. However this should not stop your code from running
2 - "Error: method submit maybe static" this is warning telling you that the method submit is a static method (bassically is a method that don't use the self attribute) to supress this warning you can use the decorator #staticmethod like this
#staticmethod
def submit():
...
3 - "SyntaxError: invalid syntax" this is what is stopping your code from running. This is a error telling you that something is written wrong in your code. I think that may be the indentation on your class. Try this:
i=0
while i<1:
try:
driver.find_element_by_xpath('//*[#title="xml (Open in a new window)"]').click()
except:
i=1
#Once the download link is clicked this has to fill the form for submission which fill download the file
class FormPage(object):
def fill_form(self, data):
driver.find_element_by_xpath('//input[#type = "radio" and #value = "Non-commercial"]').click()
driver.find_element_by_xpath('//input[#type = "checkbox" and #value = "R&D"]').click()
driver.find_element_by_xpath('//input[#name = "name_d"]').send_keys(data['name_d'])
driver.find_element_by_xpath('//input[#name = "mail_d"]').send_keys(data['mail_d'])
return self
def submit(self):
driver.find_element_by_xpath('//input[#value = "Submit"]').click()
data = {
'name_d': 'abc',
'mail_d': 'xyz#gmail.com',
}
FormPage().fill_form(data).submit()
driver.quit()
One more thing. Those are really simple errors and warnings, you should be able to fix them by yourself by carefully reading what the error has to say. I also reccomend you reading about Exceptions
I would like some help on how to handle an url which fails to open, currently the whole program gets interrupted when it fails to open the url ( tree = ET.parse(opener.open(input_url)) )...
If the opening of an url fails on my first function call (motgift) I would like it to wait 10 seconds and then try to open the url again, if it once again fails I would like my script to continue with next function call (observer).
def spider_xml(input_url, extract_function, input_xpath, pipeline, object_table, object_model):
opener = urllib.request.build_opener()
tree = ET.parse(opener.open(input_url))
print(object_table)
for element in tree.xpath(input_xpath):
pipeline.process_item(extract_function(element), object_model)
motgift = spider_xml(motgift_url, extract_xml_item, motgift_xpath, motgift_pipeline, motgift_table, motgift_model)
observer = spider_xml(observer_url, extract_xml_item, observer_xpath, observer_pipeline, observer_table, observer_model)
Would be very happy and appreciate an example on how to make this happen.
Would a Try Except block work?
error = 0
while error < 2:
try:
motgift = spider_xml(motgift_url, extract_xml_item, motgift_xpath, motgift_pipeline, motgift_table, motgift_model
break
except:
error += 1
sleep(10)
try:
resp = opener.open(input_url)
except Exception:
time.sleep(10)
try:
resp = opener.open(input_url)
except Exception:
pass
Are you looking for this?
I have the following code that grabs images using urlretrieve working..... too a point.
def Opt3():
global conn
curs = conn.cursor()
results = curs.execute("SELECT stock_code FROM COMPANY")
for row in results:
#for image_name in list_of_image_names:
page = requests.get('url?prodid=' + row[0])
tree = html.fromstring(page.text)
pic = tree.xpath('//*[#id="bigImg0"]')
#print pic[0].attrib['src']
print 'URL'+pic[0].attrib['src']
try:
urllib.urlretrieve('URL'+pic[0].attrib['src'],'images\\'+row[0]+'.jpg')
except:
pass
I am reading a CSV to input the image names. It works except when it hits an error/corrupt url (where there is no image I think). I was wondering if I could simply skip any corrupt urls and get the code to continue grabbing images? Thanks
urllib has a very bad support for error catching. urllib2 is a much better choice. The urlretrieve equivalent in urllib2 is:
resp = urllib2.urlopen(im_url)
with open(sav_name, 'wb') as f:
f.write(resp.read())
And the errors to catch are:
urllib2.URLError, urllib2.HTTPError, httplib.HTTPException
And you can also catch socket.error in case that the network is down.
Simply using except Exception is a very stupid idea. It'll catch every error in the above block even your typos.
Just use a try/except and continue if it fails
try:
page = requests.get('url?prodid=' + row[0])
except Exception,e:
print e
continue # continue to next row
Instead of pass why don't you try continue when an error occurs.
try:
urllib.urlretrieve('URL'+pic[0].attrib['src'],'images\\'+row[0]+'.jpg')
except Exception e:
continue
Is there any resume() function in python. I need to apply it on my program. need proper explanation I searched a lot but didn't get it.
Here is my code where I need to place the resume function.
try:
soup = BeautifulSoup(urllib2.urlopen(url))
abc = soup.find('div', attrs={})
link = abc.find('a')['href']
#result is dictionary
results['Link'] = "http://{0}".format(link)
print results
#pause.minute(1)
#time.sleep(10)
except Exception:
print "socket error continuing the process"
time.sleep(4)
#pause.minute(1)
#break
I apply pause, time.stamp and break but not getting the required result. If any error appears in try then I want to pause the program. try block is already inside loop.
To resume the code in case of an exception, put it inside a loop:
import time
import urllib2
from bs4 import BeautifulSoup # $ pip install beautifulsoup4
for _ in range(max_retries):
try:
r = urllib2.urlopen(url)
encoding = r.info().getparam('charset')
html = r.read()
except Exception as e:
last_error = e
time.sleep(retry_timeout)
else:
break
else: # all max_retries attempts failed
raise last_error
soup = BeautifulSoup(html, from_encoding=encoding)
# ...