Develop Reference

Selenium either Custom Profile or get() method not working

I was trying to control an already open instance of Chrome, but when that didn't work, I started to try to make a new instance of Chrome on my profile. I found two stackoverflow posts detailing the problem (How to use Chrome Profile in Selenium Webdriver Python 3 and Selenium: get() not working with custom google profile) but neither solution worked. I have two code blocks, both of which has its own problem.
The first thing I tried was this:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_argument(r"--user-data-dir=C:\\Users\\EliBa\\AppData\\Local\\Google\\Chrome\\User Data")
options.add_argument(r'--profile-directory=Default')
driver = webdriver.Chrome(executable_path=r'C:\\Selenium Drivers\\chromedriver.exe', chrome_options=options)
driver.get("https://google.com")
While this opened my profile, it did not open google like I asked it to. The second thing I tried was this:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("user-data-dir=C:\\Users\\EliBa\\AppData\\Local\\Google\\Chrome\\User Data\\Default")
driver = webdriver.Chrome(executable_path='C:\\Selenium Drivers\\chromedriver.exe', options=options)
driver.get("https://www.google.com")
While this opened Google, it did not open my custom profile. Does anyone know what I should be putting so that the custom profile opens AND it loads the site I ask it to?

Selenium closes after it opens

So I'm using Python 3.9.5
Everytime I use the driver.get function, chrome appears for a second and then immediately closes the window. How do I make it like it stays open? It used to work, but when I closed VSCode and re-opened it, I did something that made it so that chrome doesn't appear to be opened.

I think you can solve this by adding the detatch option.
For example:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
options = webdriver.ChromeOptions()
options.add_experimental_option("detach", True)
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)

Check the logs for more information. Sometimes your web browser has been updated and the chromedriver does not match with the version, so, you have to download the specific version of your browser.

How to initiate a Chromium based Vivaldi browser session using Selenium and Python

I am trying to use the vivaldi browser with Selenium. It is a chromium browser that runs very similar to chrome. I have Selenium working with Firefox (geckodriver), and Google Chrome(chromedriver), but I can't seem to find a way with Vivaldi. Any help would be appreciated.

If the vivaldi binary by default is located at C:\Users\levir\AppData\Local\Vivaldi\Application\vivaldi.exe you can use the following solution:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("start-maximized")
options.binary_location=r'C:\Users\levir\AppData\Local\Vivaldi\Application\vivaldi.exe'
driver = webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe', options=options)
driver.get('http://google.com/')

For future reference:
to make Vivaldi work with selenium you need to make sure of three things :
The correct version of ChromeDriver
Set selenium's driver to use Vivaldi's binary via webdriver.ChromeOptions()
Make sure you're getting the correct url (don't forget "https://")
All of the above is explained step by step with screenshots in this blog post

The key executable_path will be deprecated in the upcoming releases of Selenium.
This post has the solution. I'm posting a copy of said solution with the path to Vivaldi, where the username is fetched by the script so you don't have to hard code it.
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
import os
current_user = os.getlogin()
s = Service(rf"C:\Users\{current_user}\AppData\Local\Vivaldi\Application\vivaldi.exe")
driver = webdriver.Chrome(service=s)
driver.get("http://duckduckgo.com") # or your website of choice

You can use ChromeOptions and supply binary.
from selenium.webdriver.chrome.options import Options
opt = Options()
opt.binary_location = chromium_path//path to chromium binary
driver = webdriver.Chrome(options=opt, executable_path="path_to_chromedriver")

Chrome Webdriver doesn't load the default profile

The Selenium Chrome Webdriver does not load my default Chrome profile.
I already tried many other Stack Overflow solutions including changing the path (and using the local Chrome App, which gives a 'permission denied' Error).
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_argument("--user-data-dir=~/Library/Application Support/Google/Chrome/Default")
options.add_argument("--profile-directory=Default")
driver = webdriver.Chrome()
driver.get("https://tagmanager.google.com/#/home")
assert "Google Analytics" in driver.title
account_element = driver.find_element_by_css_selector("div.account.card div#149385038 table")
accountPublicId = driver.find_element_by_css_selector("div#149385038 table td.account__public-id")
The result is still the same; it loads only a 'naked' Chrome Webdriver instead of loading the local default profile (I use everyday for work).
Update:
I don't know how or why but now, when I quit Chrome and start Chrome through the Python Script, Google Chrome starts with my profile but does not use the cookies I have in that profile.
I will see if I can add the cookies "manually" with an options.add_arguments.

Try this:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\\ProfilePath") ##profile path
w = webdriver.Chrome(executable_path="C:\\chromedriver.exe", chrome_options=options)

you may try this
options.add_argument("user-data-dir=C:\\Test")
did not add anything to the
options arguments (looking via debugger it was an empty list)
options.arguments.append("user-data-dir=C:/test/") managed to update the list of arguments
and it worked.

Python disable Chrome Extension

I try to use Python to log in a web, type in use id and password, and download some files. however, before logging the page, there is a error msg says
"Failed to load extension from
C:\User\xyz\AppData\Local\Temp\scoped_dir12188_17478\internal. Loading
of unpacked extensions is disabled by administrator"
. After I hit "Enter", the rest goes well.
I have searched here, and find one post is related to my question, Here. But the code is still not working for me. the pop up error windows is still showing up. Any ideas or suggestion is highly welcome. Thank you!
Here is my code:
import time
import urllib3
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
driverpath = 'C:\\Users\\xyz\\AppData\\Local\\Programs\\Python\\Python36\\chromedriver.exe'
chrome_options = Options()
chrome_options.add_argument("--disable-extensions")
browser = webdriver.Chrome(executable_path=driverpath, chrome_options=chrome_options)
url ='https://userid:password#weblink'
browser.get(url)