Develop Reference

python selenium Access to script at https://sitesA.com from origin https://sitesB.com has been blocked by CORS policy only in headless mode

I'm building a python program that using selenium and chrome driver to download .xml and .pdf files from a website, it's running fine, but when i turn the driver into headless mode then "CORS policy occur", I already try adding "disable-web-security" and "--disable-site-isolation-trials" after spend alot of time searching on internet but still no luck,So anyone please tell me what am I missing? What am I doing wrong? this is how I implement chrome driver:
options = webdriver.ChromeOptions()
options.add_argument('no-sandbox')
options.add_argument('--disable-gpu')
options.add_argument('--disable-extensions')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--disable-blink-features=AutomationControlled')
options.add_argument('--safebrowsing-disable-download-protection')
options.add_argument('--start-maximized')
options.headless = True
extset = ['enable-automation', 'ignore-certificate-errors']
options.add_experimental_option('excludeSwitches', extset)
options.add_experimental_option('useAutomationExtension', False)
options.add_argument('--ignore-certificate-errors')
options.add_argument('--allow-insecure-localhost')
options.add_argument('--safebrowsing-disable-download-protection')
options.add_argument('--disable-web-security')
options.add_argument('--disable-site-isolation-trials')
options.add_argument('safebrowsing-disable-extension-blacklist')
prefs = {
'download.default_directory' : DOWNLOAD_DIRECTORY, # set up download directory
'safebrowsing.enabled': True, # disable xml download asking
'profile.default_content_setting_values.automatic_downloads': 1, # allow multiple files download
'download.prompt_for_download': False
}
if rpa.get('options') == 'AUTO_DOWNLOAD_PDF':
prefs.update({ 'plugins.plugins_list': [{ 'enabled': False, 'name': 'Chrome PDF Viewer' }] })
prefs.update({ 'plugins.always_open_pdf_externally': True })
prefs.update({ 'browser.helperApps.neverAsk.saveToDisk': 'application/pdf,application/vnd.adobe.xfdf,application/vnd.fdf,application/vnd.adobe.xdp+xml' })
options.add_experimental_option('prefs', prefs)
driver = webdriver.Chrome(os.getcwd()+'\\webdriver\\chromedriver.exe', options=options)

Robocorp selenium fails to open pdf in chrome browser

I am trying to automate clicking through a website and currently have the following code
def open_site(self, url, headless=True, **kwargs):
options = webdriver.ChromeOptions()
options.add_argument("--incognito")
options.add_argument("--disable-popup-blocking")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
options.add_argument("--start-maximized")
options.add_argument("--disable-extensions")
options.add_argument('--disable-infobars')
options.add_argument("--disable-web-security")
options.add_argument("--allow-running-insecure-content")
options.add_experimental_option('prefs', {
"download.default_directory": str(Path(self.download_dir).resolve()), # Change default directory for downloads
"download.prompt_for_download": False, # To auto download the file
"download.directory_upgrade": True,
"plugins.always_open_pdf_externally": True, # It will not show PDF directly in chrome
"download.extensions_to_open": "applications/pdf",
"profile.default_content_settings.popups": 1
})
self.create_webdriver('Chrome', desired_capabilities=options.to_capabilities(), executable_path="/path/to/chrome/driver")
self.go_to(url=url)
However, when the website opens I get stuck at the point of loading the pdf. It seems as though chrome fails on the aspx part. The website does load the pdf in Safari but this can only be used locally. Firefox isint supported at all by the website. Chrome is the only option. pdf image and click through image. Any assistance would be appreciated.

Selenium webdriver - prefs ignored when using specific Chrome profile

I read a lot of answers about setting a download directory for Chrome, but couldn't find anything when you want to save a file to a specific directory, different from the default one, and you are using a specific profile.
The below code works fine, and when I download a file, it gets saved to the intended downloadFolderPath:
downloadFolderPath = 'P:\\reports\\attachments'
options = webdriver.ChromeOptions()
preferences = {"download.default_directory": downloadFolderPath,
"directory_upgrade": True,
"safebrowsing.enabled": True }
options.add_experimental_option("prefs", preferences)
options.headless = False
browser = webdriver.Chrome(executable_path='chromedriver.exe', options=options)
# browse to the page, click download, etc.
But when I try to do the same, this time using a specific Chrome profile, by adding user-data-dir and profile-directory arguments:
downloadFolderPath = 'P:\\reports\\attachments'
options = webdriver.ChromeOptions()
options.add_argument(r'--user-data-dir=C:\\Users\\my.name\\AppData\\Local\\Google\\Chrome\\User Data')
options.add_argument(r'--profile-directory=Profile 5')
preferences = {
"download.default_directory": downloadFolderPath,
"directory_upgrade": True,
"safebrowsing.enabled": True
}
options.add_experimental_option("prefs", preferences)
options.headless = False
browser = webdriver.Chrome(executable_path='chromedriver.exe', options=options)
# browse the page and download the file
Chrome indeed gets opened with the chosen profile, but the file that I download gets saved under the user profile default directory, C:\Users\my.name\Downloads instead of the one that I defined (P:\reports\attachments). It seems therefore that the preferences passed with options.add_experimental_option("prefs", preferences) are ignored when a profile different from the default one is used.
Or is there any different argument that needs to be used when working with specific profiles?
For info, I am on Windows 10, Python 3.9, using Chrome 95.0.4638.54 and chromedriver.exe is 95.0.4638.17
Thanks

Disable download pop up Firefox Selenium

I am very new to Selenium and I am typing up my first script to download a .csv file from a webpage.
Problem is, when I click the button to download the .csv file a download window pops up. How do I automatically save the file to a folder?
I tried a variety of profiles but I can't seem to get them to work.

I am using the below configuration in one of my project
You can possibly have this firefox profile, In Python you could this :
profile = FirefoxProfile()
profile.set_preference("browser.download.panel.shown", False)
profile.set_preference("browser.helperApps.neverAsk.openFile","text/csv,application/vnd.ms-excel")
profile.set_preference("browser.helperApps.neverAsk.saveToDisk", "application/msword, application/csv, application/ris, text/csv, image/png, application/pdf, text/html, text/plain, application/zip, application/x-zip, application/x-zip-compressed, application/download, application/octet-stream");
profile.set_preference("browser.download.manager.showWhenStarting", False);
profile.set_preference("browser.download.manager.alertOnEXEOpen", False);
profile.set_preference("browser.download.manager.focusWhenStarting", False);
profile.set_preference("browser.download.folderList", 2);
profile.set_preference("browser.download.useDownloadDir", True);
profile.set_preference("browser.helperApps.alwaysAsk.force", False);
profile.set_preference("browser.download.manager.alertOnEXEOpen", False);
profile.set_preference("browser.download.manager.closeWhenDone", True);
profile.set_preference("browser.download.manager.showAlertOnComplete", False);
profile.set_preference("browser.download.manager.useWindow", False);
profile.set_preference("services.sync.prefs.sync.browser.download.manager.showWhenStarting", False);
profile.set_preference("pdfjs.disabled", True);
profile.set_preference("browser.download.dir", "C:\\Users\\***\\****\\Desktop\\Automation")
driver = webdriver.Firefox(firefox_profile = profile, executable_path = "Full file path to gecko driver.exe")

Downloading a file at a specified location through python and selenium using Chrome driver

I am trying to automatically download some links through selenium's click functionality and I am using a chrome webdriver and python as the programming language. How can I select the download directory through the python program so that it does not get downloaded in the default Downloads directory. I found a solution for firefox but there the download dialog keeps popping up every time it clicks on the link which does not happen in Chrome.

I found the accepted solution didn't work, however this slight change did:
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
prefs = {'download.default_directory' : '/path/to/dir'}
chrome_options.add_experimental_option('prefs', prefs)
driver = webdriver.Chrome(chrome_options=chrome_options)

Update 2018:
Its not valid Chrome command line switch, see the source code use hoju answer below to set the Preferences.
Original:
You can create a profile for chrome and define the download location for the tests. Here is an example:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("download.default_directory=C:/Downloads")
driver = webdriver.Chrome(chrome_options=options)

the exact problem I also have faced while trying to do exactly same what you want to :)
For chrome:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--start-maximized")
prefs = {"profile.default_content_settings.popups": 0,
"download.default_directory":
r"C:\Users\user_dir\Desktop\\",#IMPORTANT - ENDING SLASH V IMPORTANT
"directory_upgrade": True}
options.add_experimental_option("prefs", prefs)
browser=webdriver.Chrome(<chromdriver.exe path>, options=options)
For Firefox:
follow this blog for the answer:
https://srirajeshsahoo.wordpress.com/2018/07/26/how-to-bypass-pop-up-during-download-in-firefox-using-selenium-and-python/
The blog says all about the pop-up and downloads dir and how to do

Using prefs solved my problem
path = os.path.dirname(os.path.abspath(__file__))
prefs = {"download.default_directory":path}
options = Options()
options.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome('../harveston/chromedriver.exe',options = options)

This worked for me on Chrome v81.0.4044.138
preferences = {
"profile.default_content_settings.popups": 0,
"download.default_directory": os.getcwd() + os.path.sep,
"directory_upgrade": True
}
chrome_options.add_experimental_option('prefs', preferences)
browser = webdriver.Chrome(executable_path="/usr/bin/chromedriver", options=chrome_options)

I see that many people have the same problem, just add the backslash at the end
op = webdriver.ChromeOptions()
prefs = {'download.default_directory' : 'C:\\Users\\SAJComputer\\PycharmProjects\\robot-test\\'}
op.add_experimental_option('prefs', prefs)
driver = webdriver.Chrome(executable_path=driver_path , options=op)

Update 2022:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
prefs = {"download.default_directory" : "C:\YourDirectory\Folder"}
options.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)

To provide download directory and chrome's diver executable path use the following code.
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("download.default_directory=C:/Your_Directory")
driver = webdriver.Chrome(options=options ,executable_path='C:/chromedriver')
change the path in your code accordingly.

If you are using linux distribution
Use this code
prefs = {'download.prompt_for_download': False,
'download.directory_upgrade': True,
'safebrowsing.enabled': False,
'safebrowsing.disable_download_protection': True}
options.add_argument('--headless')
options.add_experimental_option('prefs', prefs)
driver = webdriver.Chrome('chromedriver.exe', chrome_options=options)
driver.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')
driver.desired_capabilities['browserName'] = 'ur mum'
params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': r'C:\chickenbutt'}}
driver.execute("send_command", params)

Below code snippet holds good for Windows/linux/MacOs distro:
downloadDir = f"{os.getcwd()}//downloads//"
# Make sure path exists.
Path(downloadDir).mkdir(parents=True, exist_ok=True)
# Set Preferences.
preferences = {"download.default_directory": downloadDir,
"download.prompt_for_download": False,
"directory_upgrade": True,
"safebrowsing.enabled": True}
chromeOptions = webdriver.ChromeOptions()
chromeOptions.add_argument("--window-size=1480x560")
chromeOptions.add_experimental_option("prefs", preferences)
driver = webdriver.Chrome(DRIVER_PATH, options=chromeOptions)
driver.get(url)
time.sleep(10)
driver.close()

This is non code level solution with no chrome profiling/options settings.
If you are using script only on your local machine then use this solution
Click on Menu -> Setting -> Show advanced settings... -> Downloads
Now uncheck
Ask where to save each file before downloading
Hope it will help you :)