Develop Reference

Got permission denied for chromedriver with Jenkins

I got tests who run on a raspberry with jenkins and Selenium and it work fine.
I got an industrial computer with Ubuntu on it. I have tests who work with command line. But when I launch it with jenkins either by being the jenkins user or by the Jenkins page it does not work.
To simplify in maximum to know where the issue came from, I made a minimum Selenium test which is the following:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
options = Options()
options.add_argument('--disable-gpu')
options.add_argument("disable-infobars")
options.add_argument("--disable-extensions")
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage")
options.add_argument("--headless")
options.add_argument("--ignore-certificate-errors")
options.add_argument("--silent")
options.add_argument("--allow-running-insecure-content")
chromedriver_service = Service("/lib/chromium-browser/chromedriver")
driver = webdriver.Chrome(options=options)
driver.get('https://www.google.com')
driver.close()
And I Got this error:
Traceback (most recent call last):
File "/home/swof/DPASS_Automation/test.py", line 18, in <module>
driver = webdriver.Chrome(options=options)
File "/home/swof/DPASS_Automation/.venv/lib/python3.9/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
self.service.start()
File "/home/swof/DPASS_Automation/.venv/lib/python3.9/site-packages/selenium/webdriver/common/service.py", line 98, in start
self.assert_process_still_running()
File "/home/swof/DPASS_Automation/.venv/lib/python3.9/site-packages/selenium/webdriver/common/service.py", line 109, in assert_process_still_running
raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: Service chromedriver unexpectedly exited. Status code was: 1
When I launch this test by being the main user no error.
I use python3.9 and a venv to have less issue between different user and that works well on the RPi.
I try different thing:
I add Jenkins to the sudo user by editing /etc/sudoers and add jenkins ALL= (ALL) NOPASSWD: ALL at the end of the file
I gave all right to the folder where my project by using chmod -R 777 /folder
Even if I use a venv I install Selenium with sudo pip install Selenium
When I use Python shell I can import Selenium without issue, it crash only here: driver = webdriver.Chrome (options=options)
I try to uninstall and install jenkins
I think it must be with chromium-browser because I got this error when I try to launch it by being the Jenkins user with chromium-browser:
WARNING: cannot start document portal: dial unix /run/user/1000/bus: connect permission denied
/user.slice/user-1000.slice/user#1000.service/app.slice/app-org.gnome.Terminal.slice/vte-spawn-99fb6268-be1d-42e8-8b6b-5267c428ebf4.scope is not a snap cgroup

A colleague of mine found a solution.
To sum up he change the home directory of Jenkins.
He creates new directory in /home. Copy all the file from /var/lib/Jenkins to this new directory. He gives access to this directory to Jenkins. Finally, he changes the home location in /etc/default/Jenkins.
It is this tutorial he uses: https://dzone.com/articles/jenkins-02-changing-home-directory

How to resolve 'Process unexpectedly closed with status 11' for Selenium in IPython?

I run the following code in IPython (from official Python extension VSCode):
# %%
from selenium import webdriver
driver = webdriver.Firefox(
executable_path='.driver/geckodriver')
It gives me the following errors:
For GeckoDriver 0.28.0
WebDriverException: Message: Process unexpectedly closed with status 11
For GeckkoDriver 0.27.0:
WebDriverException: Message: invalid argument: can't kill an exited process
My setup:
Firefox: 82.0
GeckoDriver: 0.28.0
Selenium: 3.141.0
IPython: 7.19.0
Ubuntu: 20.10
Python3 venv
Note: This script works in terminal but not IPython
Thank you.

I had the same issue with Firefox driver and I change it to Chrome driver it works well now
# install chromium, its driver, and selenium
!apt update
!apt install chromium-chromedriver
!pip install selenium
# set options to be headless, ..
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
# open it, go to a website, and get results
wd = webdriver.Chrome(options=options)
wd.get("https://www.website.com")
print(wd.page_source) # results

According to your description, I installed the same configuration as yours on my computer. When I run the code in the .py file and .ipynb file in VSCode, the result can be displayed:
It is recommended that you check the following aspects:
Make sure that the kernel "ipykernel" required by Jupyter has been installed in your currently selected VSCode environment.
Please check the execution path of "geckodriver" in the code.
Please try to reload VSCode or restart VSCode or even your computer if conditions permit.

Facing chromedriver unexpectedly exited. Status code was: 127 while using Headless chrome browser with docker & python 3.7 on AWS ec2

This is not a duplicate of other questions & I have tried the existing answers as well. On AWS ec2 I have created a docker container in which I am trying to open headless chrome. The docker file is
FROM python:3
# Install Chrome for Selenium
RUN curl https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb -o /chrome.deb
RUN dpkg -i /chrome.deb || apt-get install -yf
RUN rm /chrome.deb
RUN wget https://chromedriver.storage.googleapis.com/2.38/chromedriver_linux64.zip
RUN unzip chromedriver_linux64.zip -d chromedriver
RUN chmod +x /chromedriver/chromedriver
ENV PATH="/chromedriver:${PATH}"
RUN echo 'logging path'
RUN echo "${PATH}"
The code which I am using to open browser is:
options = Options()
options.add_argument("--window-size=1920,1080")
options.add_argument("--disable-gpu")
options.add_argument("--disable-extensions")
options.add_argument("--proxy-server='direct://'")
options.add_argument("--proxy-bypass-list=*")
options.add_argument("--start-maximized")
options.add_argument("--headless")
options.add_argument('--no-sandbox')
options.headless = True
driver = webdriver.Chrome(options=options, executable_path='/chromedriver/chromedriver')
driver.get(url)
The error which on ec2 is:
File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
self.service.start()
File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 98, in start
self.assert_process_still_running()
File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 111, in assert_process_still_running
% (self.path, return_code)
selenium.common.exceptions.WebDriverException: Message: Service /chromedriver/chromedriver unexpectedly exited. Status code was: 127
I have tried Installing Chromium on Amazon Linux and other stackoverflow answers as well but unable to solve it.
Google-chrome version - Google Chrome 75.0.3770.100
driver version - 2.38
selenium - 3.141.0
The OS of AWS AMI is linux and not ubuntu thuscommand ldd chrome | grep not would help. Even though I have fixed that issue as well, still getting error. When logged the path of chromedriver it's logged as below
/usr/local/lib/python3.7/site-packages/chromedriver_binary:/chromedriver:

I had a similar error which was fixed by installing xvfb, this is required because you are running a headless session of chrome. Try doing apt-get install xvfb

Selenium crashing with selenium.common.exceptions.WebDriverException: Message: newSession

OS: Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-1066-aws x86_64)
Selenium Version: selenium==3.6.0
Browser: Mozilla Firefox 63.0
Geckodriver version : geckodriver-v0.19.0-linux64
Expected Behavior -
Create a new firefox browser and do some steps - parsing the website.
Actual Behavior -
Crashing with a log :-
self.driver = webdriver.Firefox()
File "/home/ubuntu/env/local/lib/python2.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 154, in __init__
keep_alive=True)
File "/home/ubuntu/env/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 151, in __init__
self.start_session(desired_capabilities, browser_profile)
File "/home/ubuntu/env/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 240, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/home/ubuntu/env/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 308, in execute
self.error_handler.check_response(response)
File "/home/ubuntu/env/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: newSession
Has anyone faced this issue before and has a possible solution ?
UPDATE :
Ran the following command : geckodriver --log trace & curl -d '{}' 127.0.0.1:4444/session
Log :-
https://pastebin.com/TirTNKrG

I fixed the issue by updating the selenium python package and using the latest geckodriver.
Thanks #fzbd for all your help.
Additional context -
Refer to the compatibility chart here - https://firefox-source-docs.mozilla.org/testing/geckodriver/Support.html#supported-platforms
update selenium :
pip install -U selenium
update geckodriver (choose the correct version for your needs based on the above compatibility chart)
Check the versions for all components -
firefox -v
geckodriver -V
pip freeze | grep selenium

Thanks #skyfail.
Your answer helped me.
The following sequence was performed and resolved the issue.
To upgrade selenium run: sudo pip3 install selenium --upgrade
To upgrade geckodriver follow steps 1-3 from this
sudo mv geckodriver /usr/local/bin/geckodriver , /usr/local/bin is usually in your PATH so no need to edit it.

In your log is the following line:
1540501901605 geckodriver ERROR Address in use (os error 98)
Which indicates that the driver is trying to use a port which is already used by some other process. Since the log doesn't show which port it is, you can run the driver with strace:
strace geckodriver 2>&1 | grep -iE 'bind|getsockname'
In my case I get these lines:
bind(3, {sa_family=AF_INET, sin_port=htons(4444), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
getsockname(3, {sa_family=AF_INET, sin_port=htons(4444), sin_addr=inet_addr("127.0.0.1")}, [128->16]) = 0
You can then check which process is using the port (for example):
netstat -tulpn | grep -i 4444
In my case returning:
tcp 0 0 127.0.0.1:4444 0.0.0.0:* LISTEN 31471/geckodriver
According to a issue about geckodriver port logging, you can let the os allocate a free port:
geckodriver --port 0
If all this doesn't work out, there may an incompatibility between your version of geckodriver and selenium, as this error in the log seems to suggest:
geckodriver::marionette TRACE <- [1,1,{"error":"unknown command","message":"newSession","stacktrace":"WebDriverError#chrome://marionette/content/error.js:178:5
I am using the following versions:
firefox 62.0.3
geckodriver 0.23.0
selenium 3.14.1

How to fix Selenium WebDriverException: The browser appears to have exited before we could connect?

I have installed firefox and Xvfb on my centos6.4 server to use selenium webdriver.
But, when I run the code, I got an error.
from selenium import webdriver
browser = webdriver.Firefox()
Error
selenium.common.exceptions.WebDriverException: Message:
'The browser appears to have exited before we could connect. The output was: None'
I read some related pages on stackoverflow and someone suggested to remove all files in tmp folder, so I did it. But, it still doesn't work.
Could anyone please give me a help?
Thank you in advance!
Edit
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.4/site-packages/selenium/webdriver/firefox/webdriver.py", line 59, in __init__
self.binary, timeout),
File "/usr/local/lib/python3.4/site-packages/selenium/webdriver/firefox/extension_connection.py", line 47, in __init__
self.binary.launch_browser(self.profile)
File "/usr/local/lib/python3.4/site-packages/selenium/webdriver/firefox/firefox_binary.py", line 64, in launch_browser
self._wait_until_connectable()
File "/usr/local/lib/python3.4/site-packages/selenium/webdriver/firefox/firefox_binary.py", line 103, in _wait_until_connectable
self._get_firefox_output())
selenium.common.exceptions.WebDriverException: Message: 'The browser appears to have exited before we could connect. The output was: None'

for Googlers, this answer didn't work for me, and I had to use this answer instead. I am using AWS Ubuntu.
Basically, I needed to install Xvfb and then pyvirtualdisplay:
sudo apt-get install xvfb
sudo pip install pyvirtualdisplay
Once I had done that, this python code worked:
#!/usr/bin/env python
from pyvirtualdisplay import Display
from selenium import webdriver
display = Display(visible=0, size=(1024, 768))
display.start()
browser = webdriver.Firefox()
browser.get('http://www.ubuntu.com/')
print browser.page_source
browser.close()
display.stop()
Thanks to #That1Guy for the first answer

I was running into this on an (headless) Ubuntu 14.04 server with Jenkins and xvfb installed. I had installed the latest stable Firefox (47) which started a build failing that ran a python script which used the Firefox driver for selenium (version 2.53).
Apparently Firefox 47+ is not compatible with the driver used in Selenium 2.53, and Selenium 3+ will be using a new driver called "Marionette" or "Gecko Driver" (which isn't officially released yet).
This page explains how to use the new driver pretty well, in several languages: https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette/WebDriver
Basically:
get/build the executable from the project on github: https://github.com/mozilla/geckodriver/releases (and make sure it's perms are set to be executable, IE chmod a+x /path/to/geckdriver-executable)
rename/copy binary to "wires"
make sure the binary's location is added to the PATH that the build uses when executing the selenium test
update the selenium test to use the new driver
For Python, step 4 looked something like the following for me:
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
firefox_capabilities = DesiredCapabilities.FIREFOX
firefox_capabilities['marionette'] = True
firefox_capabilities['binary'] = '/usr/bin/firefox'
driver = webdriver.Firefox(capabilities=firefox_capabilities)

I too had faced same problem. I was on Firefox 47 and Selenium 2.53; I downgraded Firefox to 45. This worked.
Remove Firefox 47 first :
sudo apt-get purge firefox
Check for available versions:
apt-cache show firefox | grep Version
It will show available firefox versions like:
Version: 47.0+build3-0ubuntu0.16.04.1
Version: 45.0.2+build1-0ubuntu1
Install a specific version
sudo apt-get install firefox=45.0.2+build1-0ubuntu1
Next you have to not upgrade to the newer version again.
sudo apt-mark hold firefox
If you want to upgrade later
sudo apt-mark unhold firefox
sudo apt-get upgrade

Check your DISPLAY environment variable. Run echo $DISPLAY in the command line.
If nothing is printed, then you are running FireFox without any DISPLAY assigned. You should assign one! Run export DISPLAY=:1 in the command line before running your python script.
Check this thread for more information: http://hashcat.net/forum/thread-1973.html

I think the simplest solution here is just run Python with xvfb-run:
sudo apt-get install xvfb
xvfb-run python <your_file_or_args>

Rollback your Firefox to the previous working version. I suggest 2 versions back. Disable Firefox Maintenance Service.
I was working on a solution and the Firefox Maintenance Service updated Firefox to the latest build in the background. This broke my code and it was giving me this error.
Now it is fixed!
Thank you everyone!

This error is due to your Xvfb is not running. So restart your xvfb:
Xvfb :99 -ac
then check.
This works for me.

Instead of downgrading firefox from 47 version to 45 or something I'll suggest to upgrade to 47.0.1 or above since they seem to fix an issue.
But if your OS doesn't have new packages in repo (for example Ubuntu 14.04 in time of this answer), you can use debs from ubuntuzilla project:
wget sourceforge.net/projects/ubuntuzilla/files/mozilla/apt/pool/main/f/firefox-mozilla-build/firefox-mozilla-build_47.0.1-0ubuntu1_amd64.deb
sudo dpkg -i firefox-mozilla-build_47.0.1-0ubuntu1_amd64.deb
For x86 use _i386.deb postfix.
That sold problem for me

I fixed this by running a recursive chown against not only the python script using selenium, but against the entire virtualenv that script was running in. I changed the ownership to the user running the file. After that, this error went away.

I also faced the same issue, what I did was:
Upgrade selenium package
sudo pip install -U selenium
Instead of rolling back to older version(like suggested) I rolled up to newer version(48.0, I was previously using V47.0).
(for upgrading follow the instructions given by Toby Speight but instead of choosing older version choose newer version)

I found this solution on Windows 10 Build 18363. I had to call out specifically the Firefoxbinary and the geckdriver executable path.
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
caps = DesiredCapabilities.FIREFOX.copy()
caps['marionette'] = True
# Path to Firefox binary
binary = FirefoxBinary(r'C:\Program Files\Mozilla Firefox\firefox.exe')
# Browser (driver) binary assigned, capabilities, and executable path for the geckodriver
driver = webdriver.Firefox(firefox_binary=binary, capabilities=caps,
executable_path=r'C:\Users\<name>\python\python-projects\geckodriver-v0.28.0-win64\geckodriver.exe')
# get google.co.in
driver.get("https://google.com")

update your selenuim version ---> pip install -U selenium

It can be solved by changing the file permission of the output file ( or related files to the program).
I used Firefox's webdriver.
Try:
chmod -R 777 output_file
This solved me the same trouble you have.