I got tests who run on a raspberry with jenkins and Selenium and it work fine.
I got an industrial computer with Ubuntu on it. I have tests who work with command line. But when I launch it with jenkins either by being the jenkins user or by the Jenkins page it does not work.
To simplify in maximum to know where the issue came from, I made a minimum Selenium test which is the following:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
options = Options()
options.add_argument('--disable-gpu')
options.add_argument("disable-infobars")
options.add_argument("--disable-extensions")
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage")
options.add_argument("--headless")
options.add_argument("--ignore-certificate-errors")
options.add_argument("--silent")
options.add_argument("--allow-running-insecure-content")
chromedriver_service = Service("/lib/chromium-browser/chromedriver")
driver = webdriver.Chrome(options=options)
driver.get('https://www.google.com')
driver.close()
And I Got this error:
Traceback (most recent call last):
File "/home/swof/DPASS_Automation/test.py", line 18, in <module>
driver = webdriver.Chrome(options=options)
File "/home/swof/DPASS_Automation/.venv/lib/python3.9/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
self.service.start()
File "/home/swof/DPASS_Automation/.venv/lib/python3.9/site-packages/selenium/webdriver/common/service.py", line 98, in start
self.assert_process_still_running()
File "/home/swof/DPASS_Automation/.venv/lib/python3.9/site-packages/selenium/webdriver/common/service.py", line 109, in assert_process_still_running
raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: Service chromedriver unexpectedly exited. Status code was: 1
When I launch this test by being the main user no error.
I use python3.9 and a venv to have less issue between different user and that works well on the RPi.
I try different thing:
I add Jenkins to the sudo user by editing /etc/sudoers and add jenkins ALL= (ALL) NOPASSWD: ALL at the end of the file
I gave all right to the folder where my project by using chmod -R 777 /folder
Even if I use a venv I install Selenium with sudo pip install Selenium
When I use Python shell I can import Selenium without issue, it crash only here: driver = webdriver.Chrome (options=options)
I try to uninstall and install jenkins
I think it must be with chromium-browser because I got this error when I try to launch it by being the Jenkins user with chromium-browser:
WARNING: cannot start document portal: dial unix /run/user/1000/bus: connect permission denied
/user.slice/user-1000.slice/user#1000.service/app.slice/app-org.gnome.Terminal.slice/vte-spawn-99fb6268-be1d-42e8-8b6b-5267c428ebf4.scope is not a snap cgroup
A colleague of mine found a solution.
To sum up he change the home directory of Jenkins.
He creates new directory in /home. Copy all the file from /var/lib/Jenkins to this new directory. He gives access to this directory to Jenkins. Finally, he changes the home location in /etc/default/Jenkins.
It is this tutorial he uses: https://dzone.com/articles/jenkins-02-changing-home-directory
Related
I run the following code in IPython (from official Python extension VSCode):
# %%
from selenium import webdriver
driver = webdriver.Firefox(
executable_path='.driver/geckodriver')
It gives me the following errors:
For GeckoDriver 0.28.0
WebDriverException: Message: Process unexpectedly closed with status 11
For GeckkoDriver 0.27.0:
WebDriverException: Message: invalid argument: can't kill an exited process
My setup:
Firefox: 82.0
GeckoDriver: 0.28.0
Selenium: 3.141.0
IPython: 7.19.0
Ubuntu: 20.10
Python3 venv
Note: This script works in terminal but not IPython
Thank you.
I had the same issue with Firefox driver and I change it to Chrome driver it works well now
# install chromium, its driver, and selenium
!apt update
!apt install chromium-chromedriver
!pip install selenium
# set options to be headless, ..
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
# open it, go to a website, and get results
wd = webdriver.Chrome(options=options)
wd.get("https://www.website.com")
print(wd.page_source) # results
According to your description, I installed the same configuration as yours on my computer. When I run the code in the .py file and .ipynb file in VSCode, the result can be displayed:
It is recommended that you check the following aspects:
Make sure that the kernel "ipykernel" required by Jupyter has been installed in your currently selected VSCode environment.
Please check the execution path of "geckodriver" in the code.
Please try to reload VSCode or restart VSCode or even your computer if conditions permit.
I am facing this issue; I know this question is already present and I have tried the solutions mentioned hence asking this with my configurations and code. I am getting the following error when Celery tasks try to use Selenium;
[2020-06-12 23:23:04,228: WARNING/ForkPoolWorker-1] Message: unknown error: Chrome failed to start: exited abnormally.
(chrome not reachable)
(The process started from chrome location /opt/python/run/venv/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
[2020-06-12 23:23:04,230: ERROR/ForkPoolWorker-1] Task Update All Results[73583ec6-474a-4f20-8d88-d7be0a150642] raised unexpected: NameError("name 'sleep' is not defined",)
Traceback (most recent call last):
File "/opt/python/bundle/42/app/fixtures/models.py", line 1035, in get_flash_result
driver = webdriver.Chrome(options=opts)
File "/opt/python/run/venv/local/lib/python3.6/dist-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
desired_capabilities=desired_capabilities)
File "/opt/python/run/venv/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "/opt/python/run/venv/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/opt/python/run/venv/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/opt/python/run/venv/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally.
(chrome not reachable)
(The process started from chrome location /opt/python/run/venv/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
I have a Django app on an EC2 server via Elastic Beanstalk which runs periodic tasks using Celery however when webdriver is set I get the error.
The Celery task calls a class method on my model which contains the selenium code. The error doesn't occur if I run the same code via Django shell or call the method from the admin dashboard - just when the Celery task is running.
the error occurs after the following code;
from selenium import webdriver
from selenium.common.exceptions import TimeoutException, NoSuchElementException, ElementClickInterceptedException, WebDriverException
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.chrome.options import Options
...
opts = Options()
opts.headless = True
opts.add_argument("--no-sandbox")
opts.add_argument("--disable-dev-shm-usage")
opts.add_argument("--remote-debugging-port=9222")
assert opts.headless # Operating in headless mode
driver = webdriver.Chrome(options=opts)
My Django app runs on a virtual environment - /opt/python/run/venv - and chromedriver and chrome are available to it;
/opt/python/run/venv/bin/chromedriver &
/opt/python/run/venv/bin/google-chrome --> /opt/google/chrome/google-chrome
versions;
(venv) [ec2-user#ip-172-31-12-230 ~]$ google-chrome --version && which google-chrome
Google Chrome 83.0.4103.97
/opt/python/run/venv/bin/google-chrome
(venv) [ec2-user#ip-172-31-12-230 ~]$ chromedriver --version && which chromedriver
ChromeDriver 83.0.4103.39 (ccbf011cb2d2b19b506d844400483861342c20cd-refs/branch-heads/4103#{#416})
/opt/python/run/venv/bin/chromedriver
(venv) [ec2-user#ip-172-31-12-230 ~]$ pip list
Package Version
--------------------- ----------
celery 4.4.2
Django 3.0.4
django-celery-beat 2.0.0
selenium 3.141.0
I'm asking this again as my previous question was closed without an answer. It was linked to another question here by moderator and closed, however the link doesn't answer my question as I'm still getting the error. I tried making the following adjustments and validations based on the linked question;
Google Chrome and Chromedriver are up to date
executable path (/opt/python/run/venv/bin/chromedriver) to chromedriver was added to webdriver()
.add_argument('--headless') was moved to be the first option set
google-chrome in venv and /usr/bin are symlinks
binary location option added
tried deleting chrome and chromedriver and reinstalling (a few times)
tried changing options to just '--no-sandbox' and '--disable-dev-shm-usage'
I've tried putting it in a loop and adding a sleep
Celery is not using root user or sudo
I only encounter this error when Celery worker executes tasks, I can run the same code being run by the worker in the django shell or via the admin dashboard and don't receive the error so it feels like this is a celery error and not my selenium options.
I had a similar problem to this. Celery, selenium and Chromedriver all worked fine on my local machine. Selenium and Chromedriver worked fine on my live server.. but when I added celery into the mix things went wrong.
I tried everything in the end I set the users/groups of celery to root and everything worked fine.. so it's a permissions issue.
After much fiddling I gave celery ownership of the google-chrome and google-chrome-stable binaries with:
sudo chown celery:celery /usr/bin/google-chrome
sudo chown celery:celery /usr/bin/google-chrome-stable
I had also previous set the chromedirve file to 755 with:
sudo chmod 755 /your/location/chromdriver
Restarted everything... and it worked.
Server: Raspberry Pi 3
OS: Dietpi - version 159
Geckodriver version: 0.22 for arm
Firefox version: 52.9.0
Python version: 3.5
Selenium version: 3.14.1
Gecko is executable, and is located in /usr/local/bin/
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.firefox.options import Options
import time
options = Options()
options.set_headless(headless=True)
driver = webdriver.Firefox(firefox_options=options)
print('Need your login credential')
username = input('What is your username?:\n')
password = input('What is your password?:\n')
...
...
Output:
root#RPi3:~# python3.5 ITE-bot.py
Traceback (most recent call last):
File "ITE-bot.py", line 12, in <module>
driver = webdriver.Firefox(firefox_options=options)
File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/firefox/webdriver.py", line 174, in __init__
keep_alive=True)
File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: invalid argument: can't kill an exited process
Any idea what is wrong? I've tried google without luck.
If you are running Firefox on a system with no display, make sure you use headless mode.
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
options = Options()
options.headless = True
driver = webdriver.Firefox(options=options)
Also, make sure you have compatible versions of Firefox, Selenium, and Geckodriver:
https://firefox-source-docs.mozilla.org/testing/geckodriver/Support.html
Thumb rule
A common cause for Browsers to crash during startup is running WebDriver initiated Browsers as root user (administrator) on Linux. While it is possible to work around this issue by passing --no-sandbox flag when creating your WebDriver session, such a configuration is unsupported and highly discouraged. You need to configure your environment to run Browser as a regular user instead.
This error message...
selenium.common.exceptions.WebDriverException: Message: invalid argument: can't kill an exited process
...implies that the GeckoDriver was unable to initiate/spawn a new WebBrowsing Session i.e. Firefox Browser session.
Your main issue is the incompatibility between the version of the binaries you are using as follows:
Your GeckoDriver version is 0.22.0.
Release Notes of GeckoDriver v0.21.0 (2018-06-15) clearly mentions the following:
Firefox 57 (and greater)
Selenium 3.11 (and greater)
Your Firefox version is 52.9.0.
So there is a clear mismatch between GeckoDriver v0.22.0 and the Firefox Browser v57
Solution
Upgrade GeckoDriver to GeckoDriver v0.22.0 level.
GeckoDriver is present in the specified location.
GeckoDriver is having executable permission for non-root users.
Upgrade Firefox version to Firefox v62.0.2 levels.
Clean your Project Workspace through your IDE and Rebuild your project with required dependencies only.
If your base Web Client version is too old, then uninstall it through Revo Uninstaller and install a recent GA and released version of Web Client.
Execute your Selenium Test as a non-root user.
GeckoDriver, Selenium and Firefox Browser compatibility chart
I was on headless mode, using correct versions of everything, and the only way to get out of this error message was not to execute the selenium test as root
Yes checked Start Xvfb before the build can fix the problem, but if you have a job like a pipeline or multibranch pipeline this option is not visible. In the node of your Selenium grid that you go to execute the test you need:
1- Install Xvfb: apt install xvfb
2- Execute Xvfb: /usr/bin/Xvfb :99 -ac -screen 0 1024x768x8 & export DISPLAY=":99"
3- Rerun your node, for example: java -jar selenium.jar -role node -hub http://#.#.#.#:4444/grid/register -capabilities browserName=firefox,plataform=linux -host #.#.#.# -port 1991
This solution worked for me
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
options = Options()
options.headless = True
driver = webdriver.Firefox(options=options)
As there can be many different underlying causes for this error it is best to find the root cause setting selenium to use debug level logging. In my case, for Ruby with capybara I needed to set: Selenium::WebDriver.logger.level = :debug. And voilà, running the same spec I could see in the logs that a dependency was missing, in my case:
libdbus-glib-1.so.2: cannot open shared object file: No such file or directory
Couldn't load XPCOM.
After installing it all worked fine.
I used:
VS Code
Linunx/Ubuntu:18.10
Nightwatch.js
My problem was that I tried to run Nightwatch (which automatically starts GeckoDriver) from the VS Code terminal.
I had the same problem, and realized that the real problem was some firefox dependencies not being installed inside the docker container I was testing in.
Try to initiate firefox and check if it returns an error.
As Nico and jay have stated you need to check the logs to see the details of the error. As you might use different systems, you can specify the path where the log is stored (i.e. "/tmp/geckodriver.log").
from selenium import webdriver
firefox_options = webdriver.firefox.webdriver.Options()
driver = webdriver.Firefox(log_path="/tmp/geckodriver.log",
options=firefox_options)
In my particular case, what the log said was:
Error: no DISPLAY environment variable specified
That was resolved adding in the options the headless mode before starting the driver. With the line:
firefox_options.set_headless()
I was able to fix this by running my tests with Xvfb. I was running them on a remote server.
I was using Jenkins so I checked the box that looked like this:
Credit to https://www.obeythetestinggoat.com/book/chapter_CI.html
in my case, I was running test cases as root
geckodriver.log
1576076416677 mozrunner::runner INFO Running command: "/usr/bin/firefox" "-marionette" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofilenCbl2e"
Running Firefox as root in a regular user's session is not supported. ($HOME is /home/seluser which is owned by seluser.)
1576077143004 mozrunner::runner INFO Running command: "/usr/bin/firefox" "-marionette" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofile7wpSQ7"
1576077143689 addons.webextension.screenshots#mozilla.org WARN Loading extension 'screenshots#mozilla.org': Reading manifest: Invalid extension permission: mozillaAddons
1576077143689 addons.webextension.screenshots#mozilla.org WARN Loading extension 'screenshots#mozilla.org': Reading manifest: Invalid extension permission: telemetry
1576077143689 addons.webextension.screenshots#mozilla.org WARN Loading extension 'screenshots#mozilla.org': Reading manifest: Invalid extension permission: resource://pdf.js/
1576077143689 addons.webextension.screenshots#mozilla.org WARN Loading extension 'screenshots#mozilla.org': Reading manifest: Invalid extension permission: about:reader*
1576077145372 Marionette INFO Listening on port 35571
1576077145423 Marionette WARN TLS certificate errors will be ignored for this session
1576077200207 mozrunner::runner INFO Running command: "/usr/bin/firefox" "-marionette" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofilenhoHlr"
Running Firefox as root in a regular user's session is not supported. ($HOME is /home/seluser which is owned by seluser.)
i could get around by
cd /home
chown -R root seluser
i woundnt say its correct but it got my job done
This is not a duplicate of other questions & I have tried the existing answers as well. On AWS ec2 I have created a docker container in which I am trying to open headless chrome. The docker file is
FROM python:3
# Install Chrome for Selenium
RUN curl https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb -o /chrome.deb
RUN dpkg -i /chrome.deb || apt-get install -yf
RUN rm /chrome.deb
RUN wget https://chromedriver.storage.googleapis.com/2.38/chromedriver_linux64.zip
RUN unzip chromedriver_linux64.zip -d chromedriver
RUN chmod +x /chromedriver/chromedriver
ENV PATH="/chromedriver:${PATH}"
RUN echo 'logging path'
RUN echo "${PATH}"
The code which I am using to open browser is:
options = Options()
options.add_argument("--window-size=1920,1080")
options.add_argument("--disable-gpu")
options.add_argument("--disable-extensions")
options.add_argument("--proxy-server='direct://'")
options.add_argument("--proxy-bypass-list=*")
options.add_argument("--start-maximized")
options.add_argument("--headless")
options.add_argument('--no-sandbox')
options.headless = True
driver = webdriver.Chrome(options=options, executable_path='/chromedriver/chromedriver')
driver.get(url)
The error which on ec2 is:
File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
self.service.start()
File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 98, in start
self.assert_process_still_running()
File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 111, in assert_process_still_running
% (self.path, return_code)
selenium.common.exceptions.WebDriverException: Message: Service /chromedriver/chromedriver unexpectedly exited. Status code was: 127
I have tried Installing Chromium on Amazon Linux and other stackoverflow answers as well but unable to solve it.
Google-chrome version - Google Chrome 75.0.3770.100
driver version - 2.38
selenium - 3.141.0
The OS of AWS AMI is linux and not ubuntu thuscommand ldd chrome | grep not would help. Even though I have fixed that issue as well, still getting error. When logged the path of chromedriver it's logged as below
/usr/local/lib/python3.7/site-packages/chromedriver_binary:/chromedriver:
I had a similar error which was fixed by installing xvfb, this is required because you are running a headless session of chrome. Try doing apt-get install xvfb
I am running python 2.7.12 with selenium version 2.53.6 and firefox 46.0.1. My goal is to run headless tests on an AWS Ubuntu Server 16.04 LTS. This is how I have set it up.
# install anaconda
cd ~
wget https://repo.continuum.io/archive/Anaconda2-4.2.0-Linux-x86_64.sh
bash Anaconda2-4.2.0-Linux-x86_64.sh
rm Anaconda2-4.2.0-Linux-x86_64.sh
echo export 'PATH="/home/ubuntu/anaconda2/bin:$PATH"' >> ~/.bashrc
source .bashrc
conda update conda
# install firefox version 46.0.1
wget https://ftp.mozilla.org/pub/firefox/releases/46.0.1/linux-x86_64/en-US/firefox-46.0.1.tar.bz2
tar jxf firefox-46.0.1.tar.bz2
rm firefox-46.0.1.tar.bz2
# install pyvirtualdisplay
sudo apt-get install xvfb xserver-xephyr
pip install pyvirtualdisplay
# install selenium version 2.53.6
pip install selenium==2.53.6
When I run this code
from pyvirtualdisplay import Display
from selenium import webdriver
display = Display(visible=0, size=(1024, 768))
display.start()
browser = webdriver.Firefox()
driver.get("http://www.python.org")
driver.quit()
display.stop()
I get this error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 55, in __init__
self.binary = firefox_binary or capabilities.get("binary", FirefoxBinary())
File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/selenium/webdriver/firefox/firefox_binary.py", line 47, in __init__
self._start_cmd = self._get_firefox_start_cmd()
File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/selenium/webdriver/firefox/firefox_binary.py", line 163, in _get_firefox_start_cmd
" Please specify the firefox binary location or install firefox")
RuntimeError: Could not find firefox in your system PATH. Please specify the firefox binary location or install firefox
I've tried to add a link to firefox in the bash profile by running echo export 'PATH="$PATH:/home/ubuntu/firefox"' >> ~/.bashrc but that did not fix the issue.
UPDATE
I was able to successfully add the path to my bash profile. I am now getting this error
"The browser appears to have exited "
selenium.common.exceptions.WebDriverException: Message: The browser appears to have exited before we could connect. If you specified a log_file in the FirefoxBinary constructor, check it for details.
Given that I have am already using pyvirtualdisplay, does anyone know of a solution?
As per my understanding, Firefox is not build for Headless mechanism [Correct me if I am wrong].
You can use PhantomJs for better handling.