OS: Win 10
Chrome: 81.0.4044.129
ChromeDriver: 81.0.4044.69
Goal:
Load an existing profile with extensions and preference configured AND specify default download locations.
Purpose:
I want to save images into their respective folders name.
Challenges
If I specify a Chrome profile to be loaded, then I cannot change the default download folder.
Code Snippets:
# Loading profile works!
options = webdriver.ChromeOptions()
options.add_argument(f'user-data-dir={profile_path}')
options.add_argument(f'--profile-directory={profile_name}')
driver = webdriver.Chrome(chrome_options=options)
# Changing default download location works!
options = webdriver.ChromeOptions()
prefs = {"download.default_directory" : "C:/Downloads/Folder A"}
options.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome(chrome_options=options)
# This DOESN'T work! Default download location is not changed.
options = webdriver.ChromeOptions()
options.add_argument(f'user-data-dir={profile_path}')
options.add_argument(f'--profile-directory={profile_name}')
prefs = {"download.default_directory" : "C:/Downloads/Folder A"}
options.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome(chrome_options=options)
Is it possible to BOTH load the profile and change the default download location before the driver is created?
I believe there isn't a way to both load an existing profile and also to change the default_directory option.
So instead, I used json.loads and json.dump to modify the 'Preference' file before loading the profile.
import json
from selenium import webdriver
# helper to edit 'Preferences' file inside Chrome profile directory.
def set_download_directory(profile_path, profile_name, download_path):
prefs_path = os.path.join(profile_path, profile_name, 'Preferences')
with open(prefs_path, 'r') as f:
prefs_dict = json.loads(f.read())
prefs_dict['download']['default_directory'] = download_path
prefs_dict['savefile']['directory_upgrade'] = True
prefs_dict['download']['directory_upgrade'] = True
with open(prefs_path, 'w') as f:
json.dump(prefs_dict, f)
options = webdriver.ChromeOptions()
set_download_directory(profile_path, profile_name, download_path) # Edit the Preferences first before loading the profile.
options.add_argument(f'user-data-dir={profile_path}')
options.add_argument(f'--profile-directory={profile_name}')
driver = webdriver.Chrome(chrome_options=options)
Related
I am trying to download images using selenium but I don't know how to direct those files at a desired location. Can anyone tell me how to do this?
Use this code to set desire download location in Selenium-Python bindings :
executable_path = r"C:\\Selenium+Python\\chromedriver.exe"
options = webdriver.ChromeOptions()
prefs = {'download.default_directory' : '/path/to/dir'}
options.add_experimental_option('prefs', prefs)
driver = webdriver.Chrome(executable_path, options=options)
You need to change /path/to/dir to your desired location.
In case you are using a Chrome webdriver you can use these settings:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("download.default_directory=C:/Downloads")
driver = webdriver.Chrome(chrome_options=options)
Here I set it to "C:/Downloads" but you can change it to any other destination.
For the Firefox you can use this:
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_preference("browser.download.folderList", 2)
profile.set_preference("browser.download.manager.showWhenStarting", False)
profile.set_preference("browser.download.dir", 'PATH TO DESKTOP')
profile.set_preference("browser.helperApps.neverAsk.saveToDisk", "application/x-gzip")
driver = webdriver.Firefox(firefox_profile=profile)
Where 'PATH TO DESKTOP' is a path on your disk where you want to download your files
as per this post to download using firefox
from selenium import webdriver
profile = webdriver.FirefoxProfile()
path = 'C:\\downloads'
profile.set_preference('browser.download.folderList', 2) # custom location
profile.set_preference('browser.download.manager.showWhenStarting', False)
profile.set_preference('browser.download.dir', path)
profile.set_preference('browser.helperApps.neverAsk.saveToDisk', 'image/png', 'image/jpeg')
then select the button to download and click it.
if you only have the link of the image, i'd recommend using something like mechanize or urllib to download the contents
img = driver.find_element_by_xpath(xpath)
src = img.get_attribute('src')
# download the image
req = urllib.urlopen(src)
f = open(filename,'wb')
f.write(req.read())
f.close()
For some reason the program does not load the profile. The location of the profile is fine.
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\\Users\\Aron\\AppData\\Local\\Google\\Chrome\\User Data\\Profile 2")
capabilities = DesiredCapabilities.CHROME.copy()
chromedriver = 'C:\\test\\chromedriver.exe'
# Opening the browser
driver = webdriver.Chrome(executable_path = chromedriver, chrome_options=options, desired_capabilities=capabilities)
Two steps:
1 - When using an empty folder for the profile Chrome creates a new profile in that folder. If it creates the profile you can suppose that it also loads it next time.
options = webdriver.ChromeOptions()
options.add_argument(r"user-data-dir=C:\test\Profile 2")
chromedriver = r'C:\test\chromedriver.exe'
driver = webdriver.Chrome(executable_path=chromedriver, chrome_options=options)
2 - But to be sure you can show the used profile in Chrome it self with chrome://version/.
options = webdriver.ChromeOptions()
options.add_argument(r"user-data-dir=C:\test\Profile 2")
chromedriver = r'C:\test\chromedriver.exe'
driver = webdriver.Chrome(executable_path=chromedriver, chrome_options=options)
driver.get('chrome://version/')
input('Enter')
I have this script that successfully downloads a file from a webpage, even headlessly thanks to this
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import os
# function to take care of downloading file
def enable_download_headless(browser,download_dir):
browser.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')
params = {'cmd':'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': download_dir}}
browser.execute("send_command", params)
# instantiate a chrome options object so you can set the size and headless preference
# some of these chrome options might be uncessary but I just used a boilerplate
# change the <path_to_download_default_directory> to whatever your default download folder is located
chrome_options = Options()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--window-size=1920x1080")
chrome_options.add_argument("--disable-notifications")
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--verbose')
chrome_options.add_experimental_option("prefs", {
"download.default_directory": ".",
"download.prompt_for_download": False,
"download.directory_upgrade": True,
"safebrowsing_for_trusted_sources_enabled": False,
"safebrowsing.enabled": False
})
chrome_options.add_argument('--disable-gpu')
#chrome_options.add_argument('--disable-software-rasterizer')
# initialize driver object and change the <path_to_chrome_driver> depending on your directory where your chromedriver should be
driver = webdriver.Chrome(options=chrome_options)
# change the <path_to_place_downloaded_file> to your directory where you would like to place the downloaded file
download_dir = "."
# function to handle setting up headless download
enable_download_headless(driver, download_dir)
# get request to target the site selenium is active on
driver.get("https://www.thinkbroadband.com/download")
# initialize an object to the location on the html page and click on it to download
search_input = driver.find_element_by_css_selector('#main-col > div > div > div:nth-child(8) > p:nth-child(1) > a > img')
search_input.click()
#this won't work because the file isn't here yet
with zipfile.ZipFile('my_downloaded_file.zip', 'r') as zip_ref:
zip_ref.extractall('.')
file = open('my_downloaded_file')
for line in file:
print(line)
file.close()
I want to use the file I have just downloaded. Everything at the bottom after the button.click() is just an example - printing out the contents for instance. It would be even better if I didn't have to find the file by name - if I could just pass selenium an empty object or something along with a function to call when the download is complete where I can use the object. How can I do something like this?
This will check if "Show in Folder" is in the pages source, which will be there if a file has been downloaded. Useful for one file, could be modified to see how many times it exists and thus check for multiple files.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.headless = True
with webdriver.Chrome(options=options) as b:
# your code here
b.execute_script("window.open('');")
b.switch_to.window(b.window_handles[1])
b.get('chrome://downloads/')
x = b.page_source
if "Show in folder" in x:
print("Yes")
else:
print("No")
I'd like to launch Chrome with its default profile using Python's webdriver so that cookies and site preferences persist across sessions.
How can I do that?
This is what finally got it working for me.
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\\Path") #Path to your chrome profile
w = webdriver.Chrome(executable_path="C:\\Users\\chromedriver.exe", chrome_options=options)
To find path to your chrome profile data you need to type chrome://version/ into address bar . For ex. mine is displayed as C:\Users\pc\AppData\Local\Google\Chrome\User Data\Default, to use it in the script I had to exclude \Default\ so we end up with only C:\Users\pc\AppData\Local\Google\Chrome\User Data.
Also if you want to have separate profile just for selenium: replace the path with any other path and if it doesn't exist on start up chrome will create new profile and directory for it.
This solved my problem. (remove Default at the end)
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--user-data-dir=/home/username/.config/google-chrome")
cls.driver = webdriver.Chrome(options=options,
executable_path="./../ext/chromedriver")
Chrome_Options ist deprecated. Use options instead
I solved my problem with answer of "Yoannes Geissler".
In my case My profile was named "Profile 2"
My Code :
options = webdriver.ChromeOptions()
options.add_argument('--user-data-dir=C:/Users/GOD/AppData/Local/Google/Chrome/User Data')
options.add_argument('--profile-directory=Profile 2')
wd = webdriver.Chrome(options=options)
The below line solved my problem:
options.add_argument('--profile-directory=Profile 2')
Just to share what worked for me. Using default's profile was complicated, chrome keeps crashing.
from pathlib import Path
from selenium import webdriver
driver_path = Path("{}/driver/chromedriver75.exe".format(PATH_TO_FOLDER))
user_data_dir = Path("{}/driver/User Data".format(PATH_TO_FOLDER))
options = webdriver.ChromeOptions()
# TELL WHERE IS THE DATA DIR
options.add_argument("--user-data-dir={}".format(user_data_dir))
# USE THIS IF YOU NEED TO HAVE MULTIPLE PROFILES
options.add_argument('--profile-directory=Default')
driver = webdriver.Chrome(executable_path=driver_path, options=options)
driver.get("https://google.com/")
By doing this Chrome will create the folder User Data and keep all the data in it where I want and it's easy to just move your project to another machine.
This answer is pretty simple and self-explained.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
exec_path_chrome = "path/to/Google Chrome" #Do not use this path that is extracted from "chrome://version/"
exec_path_driver = "path/to/chromedriver"
ch_options = Options() #Chrome Options
ch_options.add_argument("user-data-dir = /path/to/Chrome Profile") #Extract this path from "chrome://version/"
driver = webdriver.Chrome(executable_path = exec_path_driver, options = ch_options) #Chrome_Options is deprecated. So we use options instead.
driver.get("https://stackoverflow.com/a/57894065/4061346")
As #MadRabbit said type chrome://version/ into the address bar to find the path to your chrome profile data.
It appears like this in Windows C:\Users\pc\AppData\Local\Google\Chrome\User Data\Default
It appears like this in Mac /Users/user/Library/Application Support/Google/Chrome/Default
So all you have to do is to erase the last portion Default from the profile path.
Note: Make sure you don't run more than one session at the same time to avoid problems.
This is what I did.
import chromedriver_binary
chrome_options = Options()
chrome_options.add_experimental_option("detach", True)
chrome_options.add_argument(r"--user-data-dir=User Data Directory")
chrome_options.add_argument(r"--profile-directory=Profile name")
chrome_options.add_experimental_option("useAutomationExtension", False)
chrome_options.add_experimental_option("excludeSwitches",["enable
logging"])
chrome_options.add_experimental_option("excludeSwitches", ["enable
automation"])
chrome_options.add_argument("start-maximized")
self.driver = webdriver.Chrome(chrome_options=chrome_options)
self.wait = WebDriverWait(self.driver, 20)
Here we do not need to give the chrome driver path.
I'd like to launch Chrome with its default profile using Python's webdriver so that cookies and site preferences persist across sessions.
How can I do that?
This is what finally got it working for me.
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\\Path") #Path to your chrome profile
w = webdriver.Chrome(executable_path="C:\\Users\\chromedriver.exe", chrome_options=options)
To find path to your chrome profile data you need to type chrome://version/ into address bar . For ex. mine is displayed as C:\Users\pc\AppData\Local\Google\Chrome\User Data\Default, to use it in the script I had to exclude \Default\ so we end up with only C:\Users\pc\AppData\Local\Google\Chrome\User Data.
Also if you want to have separate profile just for selenium: replace the path with any other path and if it doesn't exist on start up chrome will create new profile and directory for it.
This solved my problem. (remove Default at the end)
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--user-data-dir=/home/username/.config/google-chrome")
cls.driver = webdriver.Chrome(options=options,
executable_path="./../ext/chromedriver")
Chrome_Options ist deprecated. Use options instead
I solved my problem with answer of "Yoannes Geissler".
In my case My profile was named "Profile 2"
My Code :
options = webdriver.ChromeOptions()
options.add_argument('--user-data-dir=C:/Users/GOD/AppData/Local/Google/Chrome/User Data')
options.add_argument('--profile-directory=Profile 2')
wd = webdriver.Chrome(options=options)
The below line solved my problem:
options.add_argument('--profile-directory=Profile 2')
Just to share what worked for me. Using default's profile was complicated, chrome keeps crashing.
from pathlib import Path
from selenium import webdriver
driver_path = Path("{}/driver/chromedriver75.exe".format(PATH_TO_FOLDER))
user_data_dir = Path("{}/driver/User Data".format(PATH_TO_FOLDER))
options = webdriver.ChromeOptions()
# TELL WHERE IS THE DATA DIR
options.add_argument("--user-data-dir={}".format(user_data_dir))
# USE THIS IF YOU NEED TO HAVE MULTIPLE PROFILES
options.add_argument('--profile-directory=Default')
driver = webdriver.Chrome(executable_path=driver_path, options=options)
driver.get("https://google.com/")
By doing this Chrome will create the folder User Data and keep all the data in it where I want and it's easy to just move your project to another machine.
This answer is pretty simple and self-explained.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
exec_path_chrome = "path/to/Google Chrome" #Do not use this path that is extracted from "chrome://version/"
exec_path_driver = "path/to/chromedriver"
ch_options = Options() #Chrome Options
ch_options.add_argument("user-data-dir = /path/to/Chrome Profile") #Extract this path from "chrome://version/"
driver = webdriver.Chrome(executable_path = exec_path_driver, options = ch_options) #Chrome_Options is deprecated. So we use options instead.
driver.get("https://stackoverflow.com/a/57894065/4061346")
As #MadRabbit said type chrome://version/ into the address bar to find the path to your chrome profile data.
It appears like this in Windows C:\Users\pc\AppData\Local\Google\Chrome\User Data\Default
It appears like this in Mac /Users/user/Library/Application Support/Google/Chrome/Default
So all you have to do is to erase the last portion Default from the profile path.
Note: Make sure you don't run more than one session at the same time to avoid problems.
This is what I did.
import chromedriver_binary
chrome_options = Options()
chrome_options.add_experimental_option("detach", True)
chrome_options.add_argument(r"--user-data-dir=User Data Directory")
chrome_options.add_argument(r"--profile-directory=Profile name")
chrome_options.add_experimental_option("useAutomationExtension", False)
chrome_options.add_experimental_option("excludeSwitches",["enable
logging"])
chrome_options.add_experimental_option("excludeSwitches", ["enable
automation"])
chrome_options.add_argument("start-maximized")
self.driver = webdriver.Chrome(chrome_options=chrome_options)
self.wait = WebDriverWait(self.driver, 20)
Here we do not need to give the chrome driver path.