Develop Reference

pyvirtualdisplay - downloads are stuck as .part file

I have a python-selenium script running on a debian 11 server without any Desktop - thats why i use pyvirtualdisplay. The script has to login to a website and then go to a page on this site that, on open, instantly downloads a xlsx file (no dialogue, direct download)
#!/usr/bin/env python
from pyvirtualdisplay import Display
from selenium import webdriver
display = Display(visible=0, size=(800, 600))
display.start()
browser = webdriver.Firefox()
profile = webdriver.FirefoxProfile()
profile.set_preference("browser.download.folderList", 2)
profile.set_preference("browser.download.manager.showWhenStarting", False)
profile.set_preference("browser.download.dir", '/updatescript')
profile.set_preference("browser.helperApps.neverAsk.saveToDisk", "application/x-gzip")
browser.get("https://example.com/login")
browser.find_element_by_class_name('name').send_keys("mymail#mysite.com")
browser.find_element_by_class_name('passwd').send_keys("password")
browser.find_element_by_class_name('btn-primary').click()
browser.get("https://example.com/download")
Note this isnt site specific but works that way on all similiar sites (only .part files are downloaded).
The second problem is that my download folder is ignored, the script always downloads in /tmp/mozilla_root0
What is the reason? Dialogue cant be as the downloads are always instantly

How to change the target directory for a screenshot using Selenium webdriver in Firefox or Chrome

I want to make a screenshot of a webpage and save it in a custom location using Selenium webdriver with Python. I tried saving the screenshot to a custom location using both Firefox and Chrome but it always saves the screenshot in the project dir. Here is my Firefox version:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
profile = webdriver.FirefoxProfile()
profile.set_preference("browser.download.folderList", 2)
profile.set_preference("browser.download.dir",
'C:\\Users\\User\\WebstormProjects')
binary = FirefoxBinary("C:\\Program Files\\Mozilla Firefox\\firefox.exe")
def foxScreen():
driver = webdriver.Firefox(firefox_binary=binary,
firefox_profile=profile)
driver.get("http://google.com")
driver.save_screenshot("foxScreen.png")
driver.quit()
if __name__ == '__main__':
foxScreen()
And here is my Chrome version:
from selenium import webdriver
options = webdriver.ChromeOptions()
prefs = {"download.default_directory": r'C:\\Users\\User\\WebstormProjects',
"directory_upgrade": True}
options.add_experimental_option("prefs", prefs)
chromedriver =
"C:\\Users\\User\\Downloads\\chromedriver_win32\\chromedriver.exe"
def chromeScreen():
driver = webdriver.Chrome(chrome_options=options,
executable_path=chromedriver)
driver.get("http://google.com")
driver.save_screenshot("chromeScreen.png")
driver.quit()
if __name__ == '__main__':
chromeScreen()
I have tried different notations for the location I want the screenshot saved to but that does not seem to help. What should I change so it does not save the screenshot to the project directory but to a given custom location?

You need to consider a couple of facts as follows:
profile.set_preference('key', 'value')
set_preference(key, value) sets the preference that we want in the firefox_profile. This preference is in effect when a specific Firefox Profile is invoked.
save_screenshot(filename)
As per the documentation save_screenshot(filename) saves a screenshot of the current window to a PNG image file. This method returns False if there is any IOError, else returns True. Use full paths in your filename.
Args:
filename: The full path you wish to save your screenshot to. This should end with a .png extension.
Usage:
driver.save_screenshot(‘/Screenshots/foo.png’)
So, save_screenshot(filename) expects the full path you wish to save your screenshot to. As you were using:
driver.save_screenshot("foxScreen.png")
Hence the screenshot was always saved within the project directory.
Solution
To save the screenshot in a different directory you need to pass the absolute path as follows:
driver.save_screenshot("./my_directory/foo.png")
Reference
You can find a detailed discussion in How to take screenshot with Selenium WebDriver

Could try adding a few more options. This worked for me:
prefs = {"download.default_directory": r"\download\directory",
"download.prompt_for_download": False,
"download.directory_upgrade": True,
"safebrowsing.enabled": True}

What is wrong with this selenium firefox profile to download file into customized folder?

I am using selenium and python v3.6 to automate firefox to download file into a customized folder. The location of the folder is C:/Users/username/Dropbox/Inv/.
Below is my firefox profile.
profile = webdriver.FirefoxProfile()
profile.set_preference('browser.download.folderList', 2) # custom location
profile.set_preference('browser.download.manager.showWhenStarting', False)
profile.set_preference('browser.download.dir', 'C:/Users/username/Dropbox/Inv/')
profile.set_preference('browser.helperApps.neverAsk.saveToDisk', 'text/plain')
profile.set_preference('browser.helperApps.neverAsk.openFile', 'text/plain')
Currently, the file is always downloaded in the default folder C:\Users\username\Downloads. How do I get the downloaded folder location to be C:/Users/username/Dropbox/Inv/?

You need to use profile while launching Firefox:
driver = webdriver.Firefox(firefox_profile = profile)
Check 8.4. How to auto save files using custom Firefox profile ? in Selenium Docs FAQ.
This is the example in the link:
import os
from selenium import webdriver
fp = webdriver.FirefoxProfile()
fp.set_preference("browser.download.folderList",2)
fp.set_preference("browser.download.manager.showWhenStarting",False)
fp.set_preference("browser.download.dir", os.getcwd())
fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "application/octet-stream")
browser = webdriver.Firefox(firefox_profile=fp)
browser.get("http://pypi.python.org/pypi/selenium")
browser.find_element_by_partial_link_text("selenium-2").click()

I will answer my own question. The problem lies with the string specifying the download directory. I should use \\ and not /.
profile.set_preference('browser.download.dir', 'C:\\Users\\username\\Dropbox\\Inv')
The code has been verified to be working now.

Why does the selenium download not work?

With selenium I try to download something (in order to verify its content), using the following code as a proof-of-concept:
from selenium import webdriver
profile = webdriver.FirefoxProfile()
#Set Location to store files after downloading.
profile.set_preference("browser.download.dir", "/tmp")
profile.set_preference("browser.download.folderList", 2)
#Set Preference to not show file download confirmation dialogue using MIME types Of different file extension types.
#profile.set_preference("browser.helperApps.neverAsk.saveToDisk",
# "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet;")
profile.set_preference("browser.download.manager.showWhenStarting", False )
profile.set_preference("pdfjs.disabled", True )
profile.set_preference("browser.helperApps.neverAsk.saveToDisk","application/zip")
profile.set_preference("plugin.disable_full_page_plugin_for_types", "application/zip")
browser = webdriver.Firefox(profile)
browser.implicitly_wait(10)
browser.get('https://www.thinkbroadband.com/download')
time.sleep(15)
elem = browser.find_element_by_xpath('//a[#href="http://ipv4.download.thinkbroadband.com/5MB.zip"]')
elem.click()
time.sleep(15)
However, nothing 'happens' (i.e. the download is not performed), and also no error message is shown. When I click on that download link manually, the test-file is being downloaded into /tmp.
Is there anything I am missing?

The issue could be because the click in this case needs to go through a child element
elem = browser.find_element_by_xpath('//a[#href="http://ipv4.download.thinkbroadband.com/5MB.zip"]/img')
elem.click()
But otherwise when you click on a link browser checks stuff in background for that link, and the site seems to have a problem, when I open the target link in browser I get an empty response

Selenium Python Chromedriver Change File Download Path

I'm looking for a way to save different files to different locations in python using chromedriver. The code below sets chrome to download to folder_path without pop the download location dialogue first.
After clicking and downloading one file into folder_path (I skipped pasting this part of code cause I have no issue), I want to download another file into new_folder_path. But the code below gives me AttributeError: 'WebDriver' object has no attribute 'Chrome'. Any ideas if I can change download location for Chrome under the same webdriver?
folder_path = "C:\\Document"
def give_chrome_option(folder_path):
chromeOptions = webdriver.ChromeOptions() #setup chrome option
prefs = {"download.default_directory" : folder_path,
"download.prompt_for_download": False,
"download.directory_upgrade": True} #set path
chromeOptions.add_experimental_option("prefs", prefs) #set option
return chromeOptions
driver = webdriver.Chrome(chrome_options = give_chrome_option(folder_path)
driver.get(sample_url)
driver.Chrome(chrome_options = give_chrome_option(new_folder_path))

No, you have to re-instantiate WebDriver if you want to download to a different directory. Depending on what exactly do you need to do, a workaround described in the first answer here might be suitable for you (download to a temporary directory then move file using os.rename())