Develop Reference

Chrome Driver crashes in headless

I am making a bot to scrap me some information from the web via chromedriver. But because this information is sometimes limited to an account (like instagram insights) i need to use several UserData-Folders to save all the logins etc for the chromedriver. To save some memory I made a function "init" which initializes a chromedriver window with the desired UserData-Folder.
the function:
def init(userdata):
global driver
warnings.filterwarnings("ignore", category=DeprecationWarning)
option = Options()
option.add_argument("--disable-infobars")
option.add_argument("window-size=fullscreen")
option.add_argument("--disable-extensions")
option.add_argument("--log-level=3")
option.add_argument("--headless")
option.add_argument(r'--user-data-dir=dir\to\UserData' + str(userdata))
option.add_experimental_option("prefs", {"profile.default_content_setting_values.notifications": 1})
driver = webdriver.Chrome(chrome_options=option, executable_path='C:\Windows\chromedriver.exe')
when running the code without headless, it works like a charm. However when using headless it gives the following error:
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: was killed.
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location C:\Program Files\Google\Chrome\Application\chrome.exe is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

chrome_options have been deprecated for quite some time now. Instead you have to use options as follows:
driver = webdriver.Chrome(options=option, executable_path='C:\Windows\chromedriver.exe')

Getting error while opening a page using selenium webdriver [duplicate]

When I'm executing this code with Selenium using Python:
from selenium import webdriver
from selenium.webdriver.common.by import By
import time
driver = webdriver.Chrome(executable_path=r'/Users/qa/Documents/Python/chromedriver')
The error occurred:
Traceback (most recent call last):
File "/Users/qa/Documents/Python/try.py", line 4, in <module>
driver = webdriver.Chrome(executable_path=r'/Users/qa/Documents/Python/chromedriver')
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
desired_capabilities=desired_capabilities)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created
from disconnected: unable to connect to renderer
(Session info: chrome=71.0.3578.98)
(Driver info: chromedriver=2.44.609545 (c2f88692e98ce7233d2df7c724465ecacfe74df5),platform=Mac OS X 10.13.6 x86_64)
Can someone help me? Thanks.

I had a similar error, first getting the error message:
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited normally.
(unknown error: DevToolsActivePort file doesn't exist)
This was solved by adding options.add_argument("--remote-debugging-port=9230") to the ChromeOptions(). And the program runs once and I gained the same error message as above:
selenium.common.exceptions.SessionNotCreatedException: Message: session not created
from disconnected: unable to connect to renderer
(Session info: headless chrome=89.0.4389.114)
The problem here was that the chrome process does not close properly in the program so the process is still active on the debugging-port. To solve this problem close the active port sudo kill -9 $(sudo lsof -t -i:9230) and simply add the following lines to the end of the code:
driver.stop_client()
driver.close()
driver.quit()
Since I didn't find this answer anywhere, I hope it helps someone.

If you have options.add_argument("--remote-debugging-port=9222") change this to options.add_argument("--remote-debugging-port=9230")
or just simply Adding options.add_argument("--remote-debugging-port=9230") fixed in my case.

This error message...
selenium.common.exceptions.SessionNotCreatedException: Message: session not created
from disconnected: unable to connect to renderer
...implies that the ChromeDriver was unable to initiate/spawn a new WebBrowser i.e. Chrome Browser session.
You need to consider a fact:
As you are using Mac OS X the Key executable_path must be supported with a Value as :
'/Users/qa/Documents/Python/chromedriver'
So line will be:
driver = webdriver.Chrome(executable_path='/Users/qa/Documents/Python/chromedriver')
Note: The path itself is a raw path so you don't need to add the switch r and drop it.
Additionally, ensure that /etc/hosts on your system contains the following entry :
127.0.0.1 localhost.localdomain localhost
#or
127.0.0.1 localhost loopback

I ran into this same issue on a Windows 10 machine. What I had to do to resolve the issue was to open the Task Manager and exit all Python.exe processes, along with all Chrome.exe processes. After doing this,

I am facing the same error and add the close code solve the problem, the code finnaly block look like this:
#staticmethod
def fetch_music_download_url(music_name: str):
chrome_driver_service = Service(ChromeDriverManager(chrome_type=ChromeType.GOOGLE).install())
chrome_options = Options()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--disable-gpu")
chrome_options.add_argument("--remote-debugging-port=9230")
driver = webdriver.Chrome(service=chrome_driver_service, options=chrome_options)
try:
driver.maximize_window()
driver.get('http://example.cn/music/?page=audioPage&type=migu&name=' + music_name)
driver.implicitly_wait(5)
// do some logic
except Exception as e:
logger.error(e)
finally:
// add the close logic
driver.stop_client()
driver.close()
driver.quit()
chrome_driver_service.stop()
the key is you should close the chrome service after using it by add chrome_driver_service.stop().hope this code could help other people facing the same issue.

This worked for me on WINDOWS OS
from selenium import webdriver
import time
from bs4 import BeautifulSoup
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu')
driver = webdriver.Chrome(options=options)
driver.get(url)
time.sleep(3)
page = driver.page_source
driver.quit()
soup = BeautifulSoup(page, 'html.parser')
Hope you'd find this useful. I also used it more comprehensively here.

WebDriverException: 'chromedriver.exe' executable may have wrong permissions

I know there are already a couple threads about this, but I've went over them all, and this is my 3rd day trying to break through and haven't made any improvements. Here is the situation:
I'm trying to use this https://github.com/tejavoo/GooglePlayReviewScraper/blob/master/scraper.py to create a google play review scraper for my research, but continue recieving the following error:
Conda3/Python38/Windows10/Spyder 4.0.1
I've made sure that the chromedriver is in the PATH, and I've made sure it as all permissions, along with spyder and python. I also made sure it was the correct version that matches my google chrome.
I've downloaded the chromedriver, unzipped it into the folder I'm working in.
I also didn't have the Permission error (Winerror 5) Until I recently updated to spyder 4.
Please, what am I missing?
chromedriver = "C:\\Users\\james\\Downloads\\ML\\New folder\\chromedriver.exe"
options = webdriver.ChromeOptions()
options.add_argument('headless')
options.add_argument('window-size=1200x600') # optional
driver = webdriver.Chrome(executable_path = chromedriver, chrome_options = options)
The Error
Traceback (most recent call last):
File "C:\Users\james\.conda\envs\spyder-4.0.0_1\lib\site-packages\selenium\webdriver\common\service.py", line 76, in start
stdin=PIPE)
File "C:\Users\james\.conda\envs\spyder-4.0.0_1\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 104, in __init__
super(SubprocessPopen, self).__init__(*args, **kwargs)
File "C:\Users\james\.conda\envs\spyder-4.0.0_1\lib\subprocess.py", line 800, in __init__
restore_signals, start_new_session)
File "C:\Users\james\.conda\envs\spyder-4.0.0_1\lib\subprocess.py", line 1207, in _execute_child
startupinfo)
*PermissionError: [WinError 5] Access is denied*
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\james\Downloads\ML\New folder\PlayReviews.py", line 37, in <module>
driver = webdriver.Chrome(executable_path = chromedriver, chrome_options = options)
File "C:\Users\james\.conda\envs\spyder-4.0.0_1\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 73, in __init__
self.service.start()
File "C:\Users\james\.conda\envs\spyder-4.0.0_1\lib\site-packages\selenium\webdriver\common\service.py", line 88, in start
os.path.basename(self.path), self.start_error_message)
WebDriverException: 'chromedriver.exe' executable may have wrong permissions. Please see https://sites.google.com/a/chromium.org/chromedriver/home
And if I change the code and place '' around chromedriver like this
chromedriver = ("C:\\Users\\james\\Downloads\\ML\\New folder\\chromedriver.exe")
options = webdriver.ChromeOptions()
options.add_argument('headless')
options.add_argument('window-size=1200x600') # optional
driver = webdriver.Chrome(executable_path= 'chromedriver', chrome_options = 'options')
I get this
runcell(0, 'C:/Users/james/Downloads/ML/New folder/PlayReviews.py')
C:\Users\james\Downloads\ML\New folder\PlayReviews.py:37: DeprecationWarning: use options instead of chrome_options
driver = webdriver.Chrome(executable_path= 'chromedriver', chrome_options = 'options')
Traceback (most recent call last):
File "C:\Users\james\Downloads\ML\New folder\PlayReviews.py", line 37, in <module>
driver = webdriver.Chrome(executable_path= 'chromedriver', chrome_options = 'options')
File "C:\Users\james\.conda\envs\spyder-4.0.0_1\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 64, in __init__
desired_capabilities = options.to_capabilities()
AttributeError: 'str' object has no attribute 'to_capabilities'
And a webdriver.py page opens
Any help is much appreciated!
Cheers :)
Update
I tried the raw string idea as mentioned
chromedriver = "C:/Users/james/Downloads/ML/New folder/PlayReviews.py"
options = webdriver.ChromeOptions()
options.add_argument('headless')
options.add_argument('window-size=1200x600') # optional
driver = webdriver.Chrome(executable_path = r'C:\\Users\\james\\Downloads\\ML\\New folder\\chromedriver.exe', chrome_options='options')
And it seems to maybe pass it? As now I am on to this error :
runcell(0, 'C:/Users/james/Downloads/ML/New folder/PlayReviews.py')
C:\Users\james\Downloads\ML\New folder\PlayReviews.py:39: DeprecationWarning: use options instead of chrome_options
driver = webdriver.Chrome(executable_path = r'C:\\Users\\james\\Downloads\\ML\\New folder\\chromedriver.exe', chrome_options='options')
Traceback (most recent call last):
File "C:\Users\james\Downloads\ML\New folder\PlayReviews.py", line 39, in <module>
driver = webdriver.Chrome(executable_path = r'C:\\Users\\james\\Downloads\\ML\\New folder\\chromedriver.exe', chrome_options='options')
File "C:\Users\james\.conda\envs\spyder-4.0.0_1\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 64, in __init__
desired_capabilities = options.to_capabilities()
AttributeError: 'str' object has no attribute 'to_capabilities'
However, this error, actually comes from line 64 of the webdriver.py which opens as a result of running the former
if chrome_options:
warnings.warn('use options instead of chrome_options',
DeprecationWarning, stacklevel=2)
options = chrome_options
if options is None:
# desired_capabilities stays as passed in
if desired_capabilities is None:
desired_capabilities = self.create_options().to_capabilities()
else:
if desired_capabilities is None:
desired_capabilities = options.to_capabilities()
else:
desired_capabilities.update(options.to_capabilities())

I have seen similar permission errors when the executable path was not a raw string. Have you tried formatting it like this?
driver = webdriver.Chrome(executable_path = r'C:\\Users\\james\\Downloads\\ML\\New folder\\chromedriver.exe', chrome_options = options)

I've finally managed to figure out where I went wrong.
While I had the PATH and all correct - the Chromdriver.exe file was not in my Python/scripts folder.
After setting it there, re-placing the PATH, all works well!! :)

Here is part of a little script I made to login and check for new questions on here. It is in a different library in C# but maybe it is still relevant and will help you discover the issue. Anyways the code is below hope it helps.
The only thing you aren't doing that I am is setting the BinaryLocation on the ChromeOptions and I am discovering the driver directory at run-time.
Also, sometimes the error messages like those are caused by exactly what they say is wrong. Maybe in order to run the selenium driver it may require more elevated windows permissions like when you right click and say run as administrator on programs sometimes. Have you tried running the terminal/command line that you are trying to run the python script on as an admin?
ChromeOptions options = new ChromeOptions();
options.BinaryLocation = #"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe";
using (IWebDriver driver = new ChromeDriver(GetChromeDriverDirectory(), options))
{
driver.NavigateToUrl("https://stackoverflow.com/questions/tagged?sort=Newest&filters=NoAnswers&tagMode=Watched");
}
// Here is the method I use to get the Chrome driver directory.
private static string GetChromeDriverDirectory() => Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);

You need to take care of a few things:
While using Selenium's python client, to pass the absolute path of the ChromeDriver you need to use the single forward slash along with the raw i.e. r switch.
The argument chrome_options is now deprecated, instead you need to use options.
Your effective code block will be:
options = webdriver.ChromeOptions()
options.headless = True
# previously: options.set_headless(headless=True)
# long ago: options.add_argument('--headless')
options.add_argument('--window-size=1200x600')
driver = webdriver.Chrome(executable_path = r'C:\Users\james\Downloads\ML\New folder\chromedriver.exe', options = options)
References
You can find a couple of relevant discussions in:
'Webdrivers' executable may have wrong permissions. Please see https://sites.google.com/a/chromium.org/chromedriver/home
How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium?

InvalidArgumentException: Message: invalid argument: user data directory is already in use error using --user-data-dir to start Chrome using Selenium

When I am trying to use --user-data-dir for the current user to start Chrome using Selenium I am getting an error as:
File "C:\Program Files (x86)\Python\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Program Files (x86)\Python\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
How can I fix this error?

This error message...
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
...implies that the ChromeDriver was unable to initiate the new Chrome Browser session using the specified user data directory as it was already in use.
This error can be reproduced as follows:
Code Block:
from selenium import webdriver
import getpass
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_argument(r"--user-data-dir=C:\Users\{}\AppData\Local\Google\Chrome\User Data".format(getpass.getuser()))
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get("https://www.google.com/")
Complete relevant traceback:
[12148:21412:0204/035557.731:ERROR:cache_util_win.cc(21)] Unable to move the cache: Access is denied. (0x5)
[12148:21412:0204/035557.731:ERROR:cache_util.cc(141)] Unable to move cache folder C:\Users\Soma Bhattacharjee\AppData\Local\Google\Chrome\User Data\ShaderCache\GPUCache to C:\Users\Soma Bhattacharjee\AppData\Local\Google\Chrome\User Data\ShaderCache\old_GPUCache_000
[12148:21412:0204/035557.731:ERROR:disk_cache.cc(178)] Unable to create cache
[12148:21412:0204/035557.731:ERROR:shader_disk_cache.cc(605)] Shader Cache Creation failed: -2
Opening in existing browser session.
Traceback (most recent call last):
File "C:\Users\Soma Bhattacharjee\Desktop\Debanjan\PyPrograms\yandex_ru.py", line 18, in <module>
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
File "C:\Python\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 81, in __init__
desired_capabilities=desired_capabilities)
File "C:\Python\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "C:\Python\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "C:\Python\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Python\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
Analysis
The error stack trace clearly complains of Access is denied as the program was unable to move the cache folder ..\ShaderCache\GPUCache to ..\ShaderCache\old_GPUCache_000. hence the creation of cache failed and subsequently creation of Shader Cache Creation failed. Though these issues raises the InvalidArgumentException but forcefully able to open a new window within the existing Chrome Browser Session.
Though the error is thrown, still the new Chrome window gets initiated but remains attached with the already opened Chrome session but the new window can't be controlled by the WebDriver instance. Hence you see data:, in the url bar.
Solution
You need to take care of a couple of things:
If you are using the Default Chrome Profile to access webpages for your other work on the same Test Machine, you shouldn't set user-data-dir as the User Data as it remains locked by the other Chrome process you have initiated manually.
In the above scenario you need to create and use another Chrome Profile and you can find a detailed discussion in How to open a Chrome Profile through Python
If you are executing your tests in a isolated test system, you can set user-data-dir as ..\User Data\Default to access the Default Chrome Profile.
In the above scenario you need to create and use another Chrome Profile and you can find a detailed discussion in How to use Chrome Profile in Selenium Webdriver Python 3
However as per best practices you must always create a new Chrome Profile to execute your tests as the Default Chrome Profile may contain Extensions, Bookmarks, Browsing History, etc, and may not load properly.
You can find a detailed discussion in How to open a Chrome Profile through --user-data-dir argument of Selenium

The simplest and easiest fix is; to clear existing opened chrome driver: Here're the steps: type task manager to the Spotlight Search/in the Search Window at the task bar or using other ways accessing to task manager. When the Task Manager Wizard/window pops up, search chromedriver, right click on it and then click "End Task". That's it. It's not an eternal fix. Once you opened chrome browser multiple times, you got to do the same step avoid the issue. Hope this helps as am looking for a stable fix.

Just had the same problem trying to skip the login process and the solution was to close the already openned browser.

As Tes answer mentioned, I opened up the Windows Task Manager and I closed all the chrome.exe and chromedriver.exe processes and it worked!!!
I used these configurations to open Google Chrome using my Chrome Profile:
options = webdriver.ChromeOptions()
options.add_argument('user-data-dir=C:\\Users\\my_user\\AppData\\Local\\Google\\Chrome\\User Data')
driver = webdriver.Chrome(executable_path='./chromedriver.exe', options=options)

System.setProperty("webdriver.chrome.driver", "C://Users//Mhamed//Desktop//chromedriver.exe");
ChromeOptions options = new ChromeOptions();
options.setExperimentalOption("useAutomationExtension", false);
WebDriver driver = new ChromeDriver(options);
driver.manage().window().fullscreen();
driver.get("");

Getting "unknown command" Exception calling the "minimize_window" method

Example code:
from selenium import webdriver
browser = webdriver.Chrome()
browser.minimize_window()
Returns the following exception:
File "myScript.py", line 4, in <module>
browser.minimize_window()
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 738, in minimize_window
self.execute(Command.MINIMIZE_WINDOW)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 312, in execute
self.error_handler.check_response(response)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 208, in check_response
raise exception_class(value)
selenium.common.exceptions.WebDriverException: Message: unknown command: session/8252be05ea571a2c623450db8ba097c0/window/minimize
Adding the line
print dir(browser)
Shows that minimize_window() is a listed function of browser. So what gives? Is this functionality just not compatible with Chrome?
Python 2.7

This error message...
selenium.common.exceptions.WebDriverException: Message: unknown command: session/8252be05ea571a2c623450db8ba097c0/window/minimize
...implies that the call to minimize_window() function wasn't recoznized.
You spotted it correct. As now WebDriver Specification is a W3C Recommendation the function defination for maximizing the window was adjusted as per the W3C Recommended Specification as follows:
def maximize_window(self):
"""
Maximizes the current window that webdriver is using
"""
params = None
command = Command.W3C_MAXIMIZE_WINDOW
if not self.w3c:
command = Command.MAXIMIZE_WINDOW
params = {'windowHandle': 'current'}
self.execute(command, params)
But, the function defination for minimizing the window is still pending to be W3C compliant within the Python Client as is still defined as:
def minimize_window(self):
"""
Invokes the window manager-specific 'minimize' operation
"""
self.execute(Command.MINIMIZE_WINDOW)
Hence you see the error:
unknown command: session/8252be05ea571a2c623450db8ba097c0/window/minimize
Solution
As the default/common practice is to open the browser in maximized mode, while Test Execution is In Progress minimizing the browser would be against the best practices as Selenium may loose the focus over the Browsing Context and an exception may raise during the Test Execution. However, Selenium's python client does have a minimize_window() method which eventually pushes the Chrome Browsing Context effectively to the background.
Python:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get('https://www.google.co.in')
driver.minimize_window()
Java Solution:
driver.navigate().to("https://www.google.com/");
Point p = driver.manage().window().getPosition();
Dimension d = driver.manage().window().getSize();
driver.manage().window().setPosition(new Point((d.getHeight()-p.getX()), (d.getWidth()-p.getY())));

Just tried to downgrade my chromedriver version to 2.25 and got...
selenium.common.exceptions.WebDriverException: Message: unknown command: session/f35727d2129895c35b24deeb7090eb26/window/minimize
with the same code.
But if to use the last (2.43) it work just fine
So just upgrade to up-to-date chromedriver version to be able to use minimize_window method