I am trying to download all of the spreadsheets from this website and the site requires me to click on the item and then click on a download button. The spreadsheets are all saved with the same id with an increasing number after each. I am trying to use a while loop to solve this problem but I am unsure if this is the best way to do this.
i = 0
while i < 13:
device = 'ctl00_LeftBarContent_lvLeftPanel_lbDevices_i' + str(i)
deviceElement = WebDriverWait(driver, 10).until(lambda driver: driver.find_element_by_id(device))
deviceElement.click()
time.sleep(2)
downloadXpath = "ctl00_LeftBarContent_lvLeftPanel_btnDownload"
WebDriverWait(driver, 10).until(lambda driver: driver.find_element_by_id(downloadXpath)).click()
time.sleep(2)
i += 1
Currently, the code runs up to clicking the download button, the file downloads and then it seems to hang at the button and eventually times out. I am using the chrome driver.
Thanks!
This comment is just regarding your indentation. Either indent i, or de-indent all your code after i.
Incorrect Example 'Hello World':
i = 0
while i < 13:
print 'hello world'
i+=1
'error unexpected indent'
Correct Example 'Hello World':
i = 0
while i < 13:
print 'hello world'
i+=1
'Hello world'
'Hello world'
'Hello world'
'Hello world'
etc...
So for your problem, indent the problem correctly and retry. If you're still having issues, throw in some print statements before and after every command to see where you're getting stuck. Just a guess, but you may have to switch frames and then close the frame after the first download has completed.
Related
def Item_Finder():
item_finder = driver.find_element(By.XPATH, "XPATH HERE").text
item_finder = re.sub('[%+]', '', item_finder)
item_finder = float(item_finder)
return item_finder
while Item_Finder() <= 5:
driver.find_element(By.XPATH, "XPATH HERE").click()
else:
Item_Finder()
print("No item found. Retrying...")
Cant seem to get this while loop working it only runs once. Code just looks at item markup on peoples listings . First post on here aswell not to sure how to get the indentations to show but they are there. Any help appreciated only recently started learning.
I'm trying to make a little Facebook group auto post script using python and selenium
First of all I'm uploading 3/4 photos using the following code:
l = driver.find_elements_by_tag_name('input')
for g in l:
print(g)
try:
if g == driver.find_element_by_xpath("//input[#type='file']"):
print("Found")
logging.debug("Found input for image uploading")
g.send_keys(
'/var/www/html/v1/insta-post/AutoPostFB/images/0.jpg \n/var/www/html/v1/insta-post/AutoPostFB/images/1.jpg \n/var/www/html/v1/insta-post/AutoPostFB/images/2.jpg')
print("File/s Uploaded")
logging.debug("Images uploaded")
time.sleep(5)
# break
except:
print("Element not found after upload")
logging.debug("Element for upload not found")
Then I push the post text, since the textbox is already on focus
actions = ActionChains(driver)
actions.send_keys(info)
actions.perform()
time.sleep(1)
The issue is that every space in the info variable is abnormally converted into a return key
so if the text is like
info = "HEI HOW'S IS GOING?"
what I get in the textbox is
IS
GOING
HEY
HOW'S
Someone can help me out? I've tried like everything
I have a little strange question, but I hope somebody can solve it anyway, because I've already tried so much and I just couldn't get any further. Thanks in advance
I have a problem with my Python script. To be precise, it's not just one problem. The aim of my script is to automatically search for keywords from a list in a search engine (www.startpage.com). Then it should count how often there is a search word on the search engine's results-page. If the searchword occurs more than 14 times there, it will be saved in a list. If it is less than 14 times, in another.
My problem now is that there are always errors. The program runs about 20 times (with success), but then,t there is just some error. The problem here is that the mistake is simply not correct. For example, an error occurs that says that a variable is undefined even though it is. Or another error looks like this:
File "webcrawler.py", line 42, in <module>
email_count=get_results(liste[x])
File "webcrawler.py", line 28, in get_results
search_box.send_keys(search_term)
AttributeError: 'str' object has no attribute 'send_keys'
But this mistake doesn't really make sense, since the program ran 20 times without errors.
My code looks like this:
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
import time
#opens textfile with list of all search terms
with open("list1.txt") as infile:
list1 = [list1.strip() for list1 in infile]
#this function searches for "search_term" on the website www.startpage.com
def get_results(search_term):
url="https://www.startpage.com"
options = webdriver.ChromeOptions()
options.add_argument('headless')
browser = webdriver.Chrome(chrome_options=options)
browser.get(url)
try:
search_box = browser.find_element_by_id("q")
except NoSuchElementException:
print("An error occured!")
search_box.send_keys(search_term)
search_box.submit()
time.sleep(3)
source_code = (browser.page_source).strip().lower()
browser.close()
time.sleep(1)
email=search_term.lower()
return source_code.count(email)
#textfiles for the results
f = open("works.txt", "a")
g = open("works_not.txt", "a")
x=0
while x < len(list1):
email_count=get_results(list1[x])
#saves listitem to result file (sorted after how often the search term appeared on the results page)
if email_count < 15:
g.write(list1[x])
g.write("\n")
g.flush()
time.sleep(1)
else:
f.write(list1[x])
f.write("\n")
f.flush()
time.sleep(1)
x=x+1
Is there something wrong with the code or should I add "sleep ()" somewhere? I'm sorry that this is such a strange question, but I hope someone sees the problem.
A thousand thanks in advance.
I've seen a few instances of this question, but I was not sure how to apply the changes to my particular situation. I have code that monitors a webpage for changes and refreshes every 30 seconds, as follows:
import sys
import ctypes
from time import sleep
from Checker import Checker
USERNAME = sys.argv[1]
PASSWORD = sys.argv[2]
def main():
crawler = Checker()
crawler.login(USERNAME, PASSWORD)
crawler.click_data()
crawler.view_page()
while crawler.check_page():
crawler.wait_for_table()
crawler.refresh()
ctypes.windll.user32.MessageBoxW(0, "A change has been made!", "Attention", 1)
if __name__ == "__main__":
main()
The problem is that Selenium will always show an error stating it is unable to locate the element after the first refresh has been made. The element in question, I suspect, is a table from which I retrieve data using the following function:
def get_data_cells(self):
contents = []
table_id = "table.datadisplaytable:nth-child(4)"
table = self.driver.find_element(By.CSS_SELECTOR, table_id)
cells = table.find_elements_by_tag_name('td')
for cell in cells:
contents.append(cell.text)
return contents
I can't tell if the issue is in the above function or in the main(). What's an easy way to get Selenium to refresh the page without returning such an error?
Update:
I've added a wait function and adjusted the main() function accordinly:
def wait_for_table(self):
table_selector = "table.datadisplaytable:nth-child(4)"
delay = 60
try:
wait = ui.WebDriverWait(self.driver, delay)
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, table_selector)))
except TimeoutError:
print("Operation timeout! The requested element never loaded.")
Since the same error is still occurring, either my timing function is not working properly or it is not a timing issue.
I've run into the same issue while doing web scraping before and found that re-sending the GET request (instead of refreshing) seemed to eliminate it.
It's not very elegant, but it worked for me.
I appear to have fixed my own problem.
My refresh() function was written as follows:
def refresh():
self.driver.refresh()
All I did was switch frames right after the refresh() call. That is:
def refresh():
self.driver.refresh()
self.driver.switch_to.frame("content")
This took care of it. I can see that the page is now refreshing without issues.
I'm using Raspberry Pi with Selenium and PhantomJS. I'm trying to verify the captcha from Google reCaptcha 2.0 but it said that the captcha is not correct (solve more). It always happens to 100% and I've never got a passed captcha from Selenium.
See, as example on this picture:
Then when I click on "Verify"-Button, then this happend:
My python code:
self.driver.switch_to.frame(capcthaframe)
picturetable = self.driver.find_element_by_css_selector('.rc-imageselect-table-3')
images = []
for row in picturetable.find_elements_by_tag_name('tr'):
for col in row.find_elements_by_tag_name('td'):
images.append(col.find_element_by_tag_name('img'))
if images == []:
self.fail("Found no captcha images")
return -1
print "[*] Got answer : " + str(numbers)
for number in numbers:
index = int(number)-1
images[index].click()
print '[+] clicked on image '+str(index+1)
self.driver.save_screenshot('res.png')
time.sleep(1)
verifybutton = self.driver.find_element_by_id('recaptcha-verify-button')
verifybutton.click()
print "[*] Clicked verify button"
time.sleep(10)
I don't think that my code is wrong... So, have anybody a idea what's wrong with this reCaptcha error?