I'm using Selenium, Chrome, and Python 3.
Here's what I'm doing to set everything up
import os
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time
chrome_options = Options()
options = webdriver.ChromeOptions()
prefs = {
"download.default_directory": r"C:\Download\Dir",
"download.directory_upgrade": "true",
"download.prompt_for_download": "false",
"disable-popup-blocking": "true",
'download.neverAsk.saveToDisk': 'application/octet-stream, application/json, jar, ' +
'text/comma-separated-values, text/csv, application/csv, ' +
'application/excel, application/vnd.ms-excel, ' +
'application/vnd.msexcel, text/anytext, text/plaintext, ' +
'image/png, image/pjpeg, image/jpeg, application/zip',
"safebrowsing.enabled": True
}
options.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome(options=options)
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--disable-software-rasterizer')
chrome_options.add_argument('--safebrowsing-disable-download-protection')
Then I have some credential secrets imported, and some website navigation to get to the file to download.
Afterwards, I click on the link to download the file:
# Click Upgrade Jar File
driver.find_element_by_xpath('//a[#id="accordionPanel:j_id_3a:1:j_id_3j"]').click()
The problem I have is here. Chrome asks me if I'd like to keep the jar file after I've downloaded it.
I've been reading through a lot of documentation and I don't understand how I can circumvent this. I believe that it should be added to prefs or as an chrome_options.add_argument but so far I've had no luck with the options I've found.
Update 01:
A little update, I was able to get it to work "silently", which allows you to bypass the "keep" button, but I haven't yet found a solution that will bypass the "keep" button when you're looking at the GUI.
options = webdriver.ChromeOptions()
prefs = {
"download.default_directory": r"C:\Download\Dir",
"download.directory_upgrade": "true",
"download.prompt_for_download": "false",
"disable-popup-blocking": "true",
"safebrowsing.enabled": False,
"default_content_settings": "contentSettings",
"download": "download"
}
options.add_experimental_option("prefs", prefs)
options.add_argument("--headless")
options.add_argument("--disable-notifications")
options.add_argument('--disable-gpu')
options.add_argument('--disable-software-rasterizer')
driver = webdriver.Chrome(options=options)
options.add_argument('--safebrowsing-disable-download-protection')
Update 02
Adapted code from a comment:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
import time
import os
# Secrets
user_id = [grabs from a function that queries secrets]
user_password = [grabs from a function that queries secrets]
download_dir = "C:\Download\Dir"
options = Options()
options.add_argument("--disable-infobars")
options.add_argument("start-maximized")
options.add_argument("--disable-extensions")
options.add_argument("--disable-popup-blocking")
# disable the banner "Chrome is being controlled by automated test software"
options.add_experimental_option("useAutomationExtension", False)
options.add_experimental_option("excludeSwitches", ['enable-automation'])
prefs = {
"download.default_directory": r"C:\Download\Dir",
'download.prompt_for_download': False,
'download.extensions_to_open': 'jar',
'safebrowsing.enabled': True
}
capabilities = DesiredCapabilities().CHROME
options.add_experimental_option('prefs', prefs)
capabilities.update(options.to_capabilities())
driver = webdriver.Chrome('.\chromedriver.exe', options=options)
url = 'URL GOES HERE'
driver.get(url)
print("Navigating to", url)
driver.get(url)
# Define Text Boxes For Login
print('Logging into GoAnywhere Website')
username = driver.find_element_by_id('email')
password = driver.find_element_by_name('secret')
# Enter Username/Password
username.send_keys(user_id)
password.send_keys(user_password)
password.send_keys('\n')
# Wait for Page to load
....[Navigation Logic Here]....
# Click Upgrades
print("Selecting Upgrades")
time.sleep(1)
driver.find_element_by_xpath('//div[#id="accordionPanel"]/div[5]').click()
# CLick Upgrade Jar File
time.sleep(1)
print("Downloading upgrade jar to", download_dir)
driver.find_element_by_xpath('//a[#id="accordionPanel:j_id_3a:1:j_id_3j"]').click()
....[Logic to do stuff with file after downloaded]....
UPDATE 05-28-2021
After doing some more testing I have determined the code below works flawlessly on a Microsoft Windows platform that does not have any type of Chrome Browser policies enabled.
The code will likely fail when the Microsoft Windows platform is under corporate IT control. Google has policy templates that allow corporate IT to set Chrome politics that can override the selenium code within this answer.
For example this is one of the politics that can be set by corporate IT that will negate the selenium code within this answer.
If either SafeBrowsingProtectionLevel_StandardProtection or SafeBrowsingProtectionLevel_EnhancedProtection are enabled Google Chrome will Gives you warnings about potentially risky sites, downloads, and extensions.
CATEGORY !!SafeBrowsing_Category
POLICY !!SafeBrowsingExtendedReportingEnabled_Policy
#if version >= 4
SUPPORTED !!SUPPORTED_WIN7
#endif
EXPLAIN !!SafeBrowsingExtendedReportingEnabled_Explain
VALUENAME "SafeBrowsingExtendedReportingEnabled"
VALUEON NUMERIC 1
VALUEOFF NUMERIC 0
END POLICY
POLICY !!SafeBrowsingProtectionLevel_Policy
#if version >= 4
SUPPORTED !!SUPPORTED_WIN7
#endif
EXPLAIN !!SafeBrowsingProtectionLevel_Explain
PART !!SafeBrowsingProtectionLevel_Part DROPDOWNLIST
VALUENAME "SafeBrowsingProtectionLevel"
ITEMLIST
NAME !!SafeBrowsingProtectionLevel_NoProtection_DropDown VALUE NUMERIC 0
NAME !!SafeBrowsingProtectionLevel_StandardProtection_DropDown VALUE NUMERIC 1
NAME !!SafeBrowsingProtectionLevel_EnhancedProtection_DropDown VALUE NUMERIC 2
END ITEMLIST
END PART
END POLICY
I also noted this within Chrome's source code. This is why the message "This type of file can harm your computer" is thrown when downloading a .jar file
namespace download_util
static const struct Executables {
const char* extension;
DownloadDangerLevel level;
} g_executables[] = {
// Some files are dangerous on all platforms.
truncated...
// Java.
{ "class", DANGEROUS },
{ "jar", DANGEROUS },
{ "jnlp", DANGEROUS },
truncated...
Jar files are considered dangerous by Google Chrome. The verbiage below is from Chrome's GitHub repository.
Note the text below is taking about displaying warning messages in the User interface (UI) of the Chrome browser.
platform_settings.danger_level: (required) Controls how files should be handled by the UI in the absence of a better signal from the Safe Browsing ping. This applies to all file types where ping_setting is either SAMPLED_PING or NO_PING, and downloads where the Safe Browsing ping either fails, is disabled, or returns an UNKNOWN verdict. Exceptions are noted below.
The warning controlled here is a generic "This file may harm your computer." If the Safe Browsing verdict is UNCOMMON, POTENTIALLY_UNWANTED, DANGEROUS_HOST, or DANGEROUS, Chrome will show that more severe warning regardless of this setting.
This policy also affects also how subresources are handled for "Save As ..." downloads of complete web pages. If any subresource ends up with a file type that is considered DANGEROUS or ALLOW_ON_USER_GESTURE, then the filename will be changed to end in .download. This is done to prevent the file from being opened accidentally.
This policy also affects also how subresources are handled for "Save As ..." downloads of complete web pages. If any subresource ends up with a file type that is considered DANGEROUS or ALLOW_ON_USER_GESTURE, then the filename will be changed to end in .download. This is done to prevent the file from being opened accidentally.
NOT_DANGEROUS: Safe to download and open, even if the download was accidental. No additional warnings are necessary.
DANGEROUS: Always warn the user that this file may harm their computer. We let them continue or discard the file. If Safe Browsing returns a SAFE verdict, we still warn the user.
CONCLUSION
The OP of this question stated in the comments that he was using a corporate IT controlled computer. It is HIGHLY LIKELY that SafeBrowsingProtection was enabled on the OP's system. With this extra level of security protection enforced by corporate IT the selenium code within this answer could not suppress the warning message being displayed in the OP's Chrome browser UI.
The OP stated that he was able to bypass the "This type of file can harm your computer" message when using selenium and Chrome in headless mode. The reason the message was suppressed is because headless mode does not use a UI thus the warning message will not be raised. Chrome's own documentation/source validates that this warning message is only displayed in the UI. Ref: platform_settings.danger_level above.
ORIGINAL POST 05-12-2021
The code below allows me to click on a href link for a .jar file without receiving "This type of file can harm your computer" message.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
chrome_options = Options()
chrome_options.add_argument("--disable-infobars")
chrome_options.add_argument("start-maximized")
chrome_options.add_argument("--disable-extensions")
chrome_options.add_argument("--disable-popup-blocking")
# disable the banner "Chrome is being controlled by automated test software"
chrome_options.add_experimental_option("useAutomationExtension", False)
chrome_options.add_experimental_option("excludeSwitches", ['enable-automation'])
prefs = {
'download.default_directory': 'download_directory',
'download.prompt_for_download': False,
'download.extensions_to_open': 'jar',
'safebrowsing.enabled': True
}
capabilities = DesiredCapabilities().CHROME
chrome_options.add_experimental_option('prefs', prefs)
capabilities.update(chrome_options.to_capabilities())
driver = webdriver.Chrome('/usr/local/bin/chromedriver', options=chrome_options)
# I used this site in my testing, because it had JAR files
url_main = 'http://www.jgoodies.com/downloads/demos/'
driver.get(url_main)
driver.implicitly_wait(20)
# download a jar file
download_jar_file = driver.find_element_by_xpath('//*[#id="post-70"]/div/table/tbody/tr[2]/td[5]/a')
download_jar_file.click()
the preference download.extensions_to_open is linked to this policy in the Chrome's source code.
"AutoOpenFileTypes" : {
"os": ["win", "mac", "linux", "chromeos"],
"policy_pref_mapping_tests": [
{
"policies": { "AutoOpenFileTypes": ["exe", ".txt", "pdf"] },
"prefs": {
"download.extensions_to_open_by_policy": {"value" : ["exe", "pdf"] }
}
}
]
},
----------------------------------------
My system information
----------------------------------------
Platform: macOS
Python: 3.8.0
Selenium: 3.141.0
Chromedriver: 90.0.4430.24
----------------------------------------
Try adding chrome_options.add_argument('--safebrowsing-disable-download-protection') in your chrome options setup.
Edit:
Wait a second. You've defined options = webdriver.ChromeOptions(). Try setting the arguments like this?
options.add_argument('--disable-gpu')
options.add_argument('--disable-software-rasterizer')
options.add_argument('--safebrowsing-disable-download-protection')
I want to change web driver preferences so I can bypass the save/open prompt that pops up when the Firefox webdriver clicks on the download button for a pdf I want.
I am setting the preferences for the Firefox web driver and passing it as a parameter "options" when I initialize the webdriver. It shows that the preferences I enter save into options.preferences but when I have selenium click on the download button on the website, the download prompt for the pdf still pops up.
def __init__(self):
options = webdriver.FirefoxOptions()
options.headless = False
print(options.preferences)
options.set_preference("browser.download.folderList", 2)
options.set_preference("browser.download.manager.showWhenStarting", False)
options.set_preference("browser.download.dir", 'myDir')
options.set_preference("browser.helperApps.neverAsk.saveToDisk", "application/pdf")
print(options.preferences)
self.driver = webdriver.Firefox(options = options)
I plan to have this headless, but have it set to False for now. What would be causing the webdriver not to take in the preferences I passed into Options?
Fixed the issue. It seems that the firefox pdf previewer was bypassing the settings I was passing through. The line below blocked the preview of the pdf and allowed the automatic download to take effect.
options.set_preference("pdfjs.disabled", True)
Using the following selenium/chrome preferences, I can download a file to the same folder that I run the original python file from, and it works properly (file saves as download.xls)
DOWNLOAD_DIR = r'/mnt/ssd/rl-scrape/files/dev/dl/'
options = Options()
options.headless = True
options.add_argument("--incognito")
options.add_argument("--window-size=1920,1200")
options.add_argument("--disable-extensions")
options.add_experimental_option("prefs", {
"download.default.directory": DOWNLOAD_DIR,
"download.prompt_for_download": False,
"download.directory_upgrade": True,
"safebrowsing.enabled": True})
I understand that I've typo'd download.default_directory above. If I correct it, the file gets downloaded to the desired directory, but as download.xls.crdownload and not download.xls
Chrome adds the .crdownload extension while the download is in progress. Once the file is complete, it will rename the file to its proper name. As long as you see .crdownload, the download is not complete.
Add a time delay for the download process to finish before it quits the window. Worked for me.
I'm using Selenium to download an embedded pdf accessed through many complex layers of logins and other browser actions. I've set up my chromedriver with the following options per instruction from various other posts:
chromedriver = r'C:\Users\cj9250\AppData\Local\Continuum\anaconda3\chromedriver.exe'
download_dir = "C:\\Users\\CJ9250\\Downloads\\" # for linux/*nix, download_dir="/usr/Public"
options = webdriver.ChromeOptions()
profile = {
"plugins.plugins_list": [{"enabled": False, "name": "Chrome PDF Viewer"}],
"download.default_directory": download_dir ,
"download.extensions_to_open": "applications/pdf",
"plugins.always_open_pdf_externally": True,
"download.prompt_for_download": False,
"safebrowsing.enabled": True
}
options.add_experimental_option("prefs", profile)
browser = webdriver.Chrome(chromedriver, chrome_options=options)
However, I get this box that I have to click before it downloads to my specified directory:
The 'Open' element doesn't have an xpath that I can find through the inspector. I'm guessing that this is some kind of internal security setting for the ChromeDriver but I can't find a way past it.
My end goal is just to download a embedded PDF in an open Selenium Test page, this seemed the only suggested course of action.
reportSho.do OPEN
I'm not sure why it worked but, I modified my profile variable to:
profile = {
"plugins.plugins_list": [{"enabled": False, "name": "Chrome PDF Viewer"}], # Disable Chrome's PDF Viewer
"download.default_directory": download_dir ,
"download.extensions_to_open": "applications/pdf",
"safebrowsing.enabled": False
}
From there I was able to get a frame element on the page with the open button. One of the element attributes had a URL. When I instructed the browser to go to the URL, it downloaded the file to my specified directory!
I want to make a screenshot of a webpage and save it in a custom location using Selenium webdriver with Python. I tried saving the screenshot to a custom location using both Firefox and Chrome but it always saves the screenshot in the project dir. Here is my Firefox version:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
profile = webdriver.FirefoxProfile()
profile.set_preference("browser.download.folderList", 2)
profile.set_preference("browser.download.dir",
'C:\\Users\\User\\WebstormProjects')
binary = FirefoxBinary("C:\\Program Files\\Mozilla Firefox\\firefox.exe")
def foxScreen():
driver = webdriver.Firefox(firefox_binary=binary,
firefox_profile=profile)
driver.get("http://google.com")
driver.save_screenshot("foxScreen.png")
driver.quit()
if __name__ == '__main__':
foxScreen()
And here is my Chrome version:
from selenium import webdriver
options = webdriver.ChromeOptions()
prefs = {"download.default_directory": r'C:\\Users\\User\\WebstormProjects',
"directory_upgrade": True}
options.add_experimental_option("prefs", prefs)
chromedriver =
"C:\\Users\\User\\Downloads\\chromedriver_win32\\chromedriver.exe"
def chromeScreen():
driver = webdriver.Chrome(chrome_options=options,
executable_path=chromedriver)
driver.get("http://google.com")
driver.save_screenshot("chromeScreen.png")
driver.quit()
if __name__ == '__main__':
chromeScreen()
I have tried different notations for the location I want the screenshot saved to but that does not seem to help. What should I change so it does not save the screenshot to the project directory but to a given custom location?
You need to consider a couple of facts as follows:
profile.set_preference('key', 'value')
set_preference(key, value) sets the preference that we want in the firefox_profile. This preference is in effect when a specific Firefox Profile is invoked.
save_screenshot(filename)
As per the documentation save_screenshot(filename) saves a screenshot of the current window to a PNG image file. This method returns False if there is any IOError, else returns True. Use full paths in your filename.
Args:
filename: The full path you wish to save your screenshot to. This should end with a .png extension.
Usage:
driver.save_screenshot(‘/Screenshots/foo.png’)
So, save_screenshot(filename) expects the full path you wish to save your screenshot to. As you were using:
driver.save_screenshot("foxScreen.png")
Hence the screenshot was always saved within the project directory.
Solution
To save the screenshot in a different directory you need to pass the absolute path as follows:
driver.save_screenshot("./my_directory/foo.png")
Reference
You can find a detailed discussion in How to take screenshot with Selenium WebDriver
Could try adding a few more options. This worked for me:
prefs = {"download.default_directory": r"\download\directory",
"download.prompt_for_download": False,
"download.directory_upgrade": True,
"safebrowsing.enabled": True}