Selenium - WebDriver not working with profile options - python

I was trying to run my code after a few months. It worked fine back then when I wrote but now -after some version updates- it keeps giving errors. Without those option arguments, it works fine.
Here it is simplified;
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument(r"--user-data-dir=C:\Users\hp\AppData\Local\Google\Chrome\User Data")
options.add_argument("--profile-directory=Default")
browser = webdriver.Chrome(options=options)
browser.get("https://google.com")
browser.quit()
It said Devtoolsactiveport file doesn't exist then I solved version conflicting errors and after tried adding --headless and this helped a little but somewhere ahead it crashed again.
Now I added options.add_argument("--remote-debugging-port=5040") and it says Chrome not reachable this time.
--disable-dev-shm-usage and --no-sandbox options did'nt help also.

Related

Chromedriver failed to start

I'm working with the most recent version of Chromedriver, but it seems to be failing to start. When I try to start tests with it, it simply loads to a blank screen with "data;," in the address bar, and prints this in the debug console:
====== WebDriver manager ======
Get LATEST chromedriver version for google-chrome 109.0.5414
Driver [C:\Users\Peter.[redacted]\.wdm\drivers\chromedriver\win32\109.0.5414\chromedriver.exe] found in cache
[ FAIL ] WebDriverException: Message: unknown error: Chrome failed to start: exited normally.
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location C:\Program Files\Google\Chrome\Application\chrome.exe is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
[ FAIL ] AttributeError: 'NoneType' object has no attribute 'close'
I have tried to delete the versions of chromedriver, manually download and replace the executable, uninstal and reinstall webdrever-manager, and have tried different versions of webdriver-manager. Nothing has worked except for swtiching to other webdrivers like firefox or remote-chrome, but both of those have conflicts with other libraries we use so they aren't options. Any help would be must appreciated.
This error usually occurs when there is an incompatibility between versions of Google Chrome, Chromedriver and/or Selenium. Here are some suggestions that may help resolve the issue:
Make sure you are using the latest version of Google Chrome and that it is compatible with the Chromedriver version you are using.
Make sure the Chromedriver version you are using is compatible with the Selenium version you are using. Make sure they are updated to the latest versions.
Try setting the 'disable-dev-shm-usage' property to true in Chrome's configuration. This can help avoid issues with shared memory usage. Example:
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome(options=chrome_options)
Try specifying the path to the Chrome executable explicitly instead of using the default path. For example:
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.binary_location = '/path/to/chrome'
driver = webdriver.Chrome(options=chrome_options)

How to restore default options for webdriver in selenium (using Python)

I tried to make edge in selenium work headless by settings --headless and --disable-gpu options, but now nothing works. Neither edge, nor chrome or firefox. Every time I encounter an error stating something like "Message: 'MicrosoftWebDriver.exe' executable needs to be in PATH".
How do I fix this?
I tried adding the driver to my workspace but it did nothing. Also, the terminal gives a link to download the driver, but it does not work.

Selenium Opens Browser But Refuses to Execute the Next Line of Code

It looks like after opening the browser, Selenium cannot move on for some reasons I can't figure out. No error was ever displayed.
Here is my simple code:
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
# Using Chrome to access web
browser = webdriver.Chrome(executable_path = "C:\Program Files\Google\Chrome\Application\chrome.exe")
print ("done")
# Open YouTube website
browser.get("https://www.youtube.com/")
The browser opens just fine, but the print("done") statement is never executed. (In the terminal the word "done" was never printed. So, it infers that the selenium has never finished executing the command to open the browser even though the browser has opened, and I have waited for several minutes.
Thanks in advance to our wonderful StackOverflow community!
The reason is because you are using chrome.exe which is for browser. Instead you should download chromdriver.exe, Please download from here. You should download Latest stable release: ChromeDriver 94.0.4606.61 (as on 3rd-oct-2021). Keep that in your automation directory and any directory of your preference.
driver_path = r'C:\\Users\\username\\Desktop\\Automation\\chromedriver.exe'
driver = webdriver.Chrome(executable_path = driver_path)
Note that, in place of driver_path, you should give the path where you've kept the chromdriver.exe
You should use chromedriver.exe instead of the path to your chrome.exe.
Download the chromedriver suitable for your chrome version, from here:
donwload Chromedriver.exe
Afterwards, do something like this:
browser = webdriver.Chrome("E:\YourPathToChromeDriver\chromedriver.exe")
Try these steps:
Check your Google Chrome version here "chrome://settings/help"
Download chromedriver.exe from "https://chromedriver.chromium.org/downloads"
Change executable path to newly downloaded file.
As mentioned above, this issue is the chrome.exe. You need to use a chromedriver instead. You can download one manually here https://chromedriver.chromium.org/downloads and then set the path to it like you did to the chrome.exe.
However, instead of downloading the chromedriver manually, I recommend using a library on GitHub which does that for you and loads it from cache if there is already one installed. (I'm not the owner nor the maintainer of this repository, but I do find it rather simple to use.)
https://github.com/SergeyPirogov/webdriver_manager
First you'd need to pip install webdriver-manager and then you can use it as following:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(ChromeDriverManager().install())

Selenium: WebDriverException:Chrome failed to start: crashed as google-chrome is no longer running so ChromeDriver is assuming that Chrome has crashed

Recently I switched computers and since then I can't launch chrome with selenium. I've also tried Firefox but the browser instance just doesn't launch.
from selenium import webdriver
d = webdriver.Chrome('/home/PycharmProjects/chromedriver')
d.get('https://www.google.nl/')
i get the following error:
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /opt/google/chrome/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
(Driver info: chromedriver=2.43.600233, platform=Linux 4.15.0-38-generic x86_64)
i have the latest chrome version and chromedriver installed
EDIT:
After trying #b0sss solution i am getting the following error.
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed
(chrome not reachable)
(The process started from chrome location /opt/google/chrome/google-chrome is no longer running, so chromedriver is assuming that Chrome has crashed.)
(Driver info: chromedriver=2.43.600233 (523efee95e3d68b8719b3a1c83051aa63aa6b10d),platform=Linux 4.15.0-38-generic x86_64)
Try to download HERE and use this latest chrome driver version:
https://sites.google.com/chromium.org/driver/
Try this:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
d = webdriver.Chrome('/home/<user>/chromedriver',chrome_options=chrome_options)
d.get('https://www.google.nl/')
This error message...
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /opt/google/chrome/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
...implies that the ChromeDriver was unable to initiate/spawn a new WebBrowser i.e. Chrome Browser session.
Your main issue is the Chrome browser is not installed at the default location within your system.
The server i.e. ChromeDriver expects you to have Chrome installed in the default location for each system as per the image below:
1For Linux systems, the ChromeDriver expects /usr/bin/google-chrome to be a symlink to the actual Chrome binary.
Solution
In case you are using a Chrome executable in a non-standard location you have to override the Chrome binary location as follows:
Python Solution:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.binary_location = "C:\\path\\to\\chrome.exe" #chrome binary location specified here
options.add_argument("--start-maximized") #open Browser in maximized mode
options.add_argument("--no-sandbox") #bypass OS security model
options.add_argument("--disable-dev-shm-usage") #overcome limited resource problems
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\path\to\chromedriver.exe')
driver.get('http://google.com/')
Java Solution:
System.setProperty("webdriver.chrome.driver", "C:\\Utility\\BrowserDrivers\\chromedriver.exe");
ChromeOptions opt = new ChromeOptions();
opt.setBinary("C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe"); //chrome binary location specified here
options.addArguments("start-maximized");
options.setExperimentalOption("excludeSwitches", Collections.singletonList("enable-automation"));
options.setExperimentalOption("useAutomationExtension", false);
WebDriver driver = new ChromeDriver(opt);
driver.get("https://www.google.com/");
hope this helps someone. this worked for me on Ubuntu 18.10
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
chrome_options.add_argument('--no-sandbox')
driver = webdriver.Chrome('/usr/lib/chromium-browser/chromedriver', options=chrome_options)
driver.get('http://www.google.com')
print('test')
driver.close()
I encountered the exact problem running on docker container (in build environment). After ssh into the container, I tried running the test manually and still encountered
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/bin/google-chrome-stable is
no longer running, so ChromeDriver is assuming that Chrome has crashed.)
When I tried running chrome locally /usr/bin/google-chrome-stable, error message
Running as root without --no-sandbox is not supported
I checked my ChromeOptions and it was missing --no-sandbox, which is why it couldn't spawn chrome.
capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(
chromeOptions: { args: %w(headless --no-sandbox disable-gpu window-size=1920,1080) }
)
I had a similar issue, and discovered that option arguments must be in a certain order. I am only aware of the two arguments that were required to get this working on my Ubuntu 18 machine. This sample code worked on my end:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
d = webdriver.Chrome(executable_path=r'/home/PycharmProjects/chromedriver', chrome_options=options)
d.get('https://www.google.nl/')
For RobotFramework
I solved it! using --no-sandbox
${chrome_options}= Evaluate sys.modules['selenium.webdriver'].ChromeOptions() sys, selenium.webdriver
Call Method ${chrome_options} add_argument test-type
Call Method ${chrome_options} add_argument --disable-extensions
Call Method ${chrome_options} add_argument --headless
Call Method ${chrome_options} add_argument --disable-gpu
Call Method ${chrome_options} add_argument --no-sandbox
Create Webdriver Chrome chrome_options=${chrome_options}
Instead of
Open Browser about:blank headlesschrome
Open Browser about:blank chrome
Assuming that you already downloaded chromeDriver, this error is also occurs when already multiple chrome tabs are open.
If you close all tabs and run again, the error should clear up.
in my case, the error was with www-data user but not with normal user on development. The error was a problem to initialize an x display for this user. So, the problem was resolved running my selenium test without opening a browser window, headless:
opts.set_headless(True)
A simple solution that no one else has said but worked for me was not running without sudo or not as root.
The solutions that every body provide here is good for Clear the face of the issue but
All you need to solve this problem is that You have to run The App on non-root user
on linux.
According to this post
https://github.com/paralelo14/google_explorer/issues/2#issuecomment-246476321
I had the same problem but it was solved just by reinstalling chrome again with the commands below:
$ wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
$ sudo apt install ./google-chrome-stable_current_amd64.deb
This error has been happening randomly during my test runs over the last six months (still happens with Chrome 76 and Chromedriver 76) and only on Linux. On average one of every few hundred tests would fail, then the next test would run fine.
Unable to resolve the issue, in Python I wrapped the driver = webdriver.Chrome() in a try..except block in setUp() in my test case class that all my tests are derived from. If it hits the Webdriver exception it waits ten seconds and tries again.
It solved the issue I was having; not elegantly but it works.
from selenium import webdriver
from selenium.common.exceptions import WebDriverException
try:
self.driver = webdriver.Chrome(chrome_options=chrome_options, desired_capabilities=capabilities)
except WebDriverException as e:
print("\nChrome crashed on launch:")
print(e)
print("Trying again in 10 seconds..")
sleep(10)
self.driver = webdriver.Chrome(chrome_options=chrome_options, desired_capabilities=capabilities)
print("Success!\n")
except Exception as e:
raise Exception(e)
I came across this error on linux environment. If not using headless then you will need
from sys import platform
if platform != 'win32':
from pyvirtualdisplay import Display
display = Display(visible=0, size=(800, 600))
display.start()
Faced with this issue trying to run/debug Python Selenium script inside WSL2 using Pycharm debugger.
First solution was to use --headless mode, but I prefer to have Chrome GUI during the debug process.
In the system terminal outside Pycharm debugger Chrome GUI worked nice with DISPLAY env variable set this way (followed guide here):
export DISPLAY=$(cat /etc/resolv.conf | grep nameserver | awk '{print $2; exit;}'):0.0
Unfortunately ~/.bashrc is not running in Pycharm during the debug, export is not working.
The way I've got Chrome GUI worked from Pycharm debugger: run echo $DISPLAY in WSL2, paste ip (you've got something similar to this) 172.18.144.1:0 into Pycharm Debug Configuration > Environment Variables:
Just do not run the script as the root user (in my case).
i had same problem. I was run it on terminal with "sudo geany", you should run it without "sudo" just type on terminal "geany" and it is solved for me.
i faced the same problem but i solved it by moving the chromedriver to this path
'/opt/google/chrome/'
and this code works correctly
from selenium.webdriver import Chrome
driver = Chrome('/opt/google/chrome/chromedrive')
driver.get('https://google.com')
In my case, chrome was broken. following two lines fixed the issue,
apt -y update; apt -y upgrade; apt -y dist-upgrade
apt --fix-broken install
Fixed it buy killing all the chrome processeses running in the remote server before running my script.
That may explain why some answers that recommend you run your script as root works.
$ pkill -9 chrome
$ ./my_script.py
Maybe when you where developing in local, you used options.headless=False in order to see what is the browser doing but you forgot to change it to True in the vm.
For me, the root issue was that the google-chrome/chromedriver version were not compatible with the Selenium version.
Seleniumn and Chrome were working fine until a few days ago and I started getting this missing DevToolsActivePort issue. After trying all sorts of solutions on this thread, it finally occurred to me that the Chrome version might not be compatible with the Selenium version.
Versions at the time of the initial error:
# Below combo does NOT work
Python 3.7.3
selenium==3.141.0
Google Chrome 110.0.5481.77
ChromeDriver 110.0.5481.77
I then downgraded Chrome and ChromeDriver to 109.0.5414.74 but still faced the same error. I checked the versions on a different machine and saw that this combo worked:
# Below combo works
Python 3.7.6
selenium==3.141.0
Google Chrome 80.0.3987.100
ChromeDriver 80.0.3987.16
However, I wasn't able to find a download for Google Chrome V80. This comment had a download to V97 so that's the version I went with. There might be higher versions of Google Chrome that do work but after spending so many days fixing this, I was eager to move onto something else.
sudo apt-get purge google-chrome-stable
sudo wget http://dl.google.com/linux/chrome/deb/pool/main/g/google-chrome-stable/google-chrome-stable_97.0.4692.71-1_amd64.deb && \
sudo dpkg -i google-chrome-stable_97.0.4692.71-1_amd64.deb && \
sudo apt-mark hold google-chrome-stable
wget https://chromedriver.storage.googleapis.com/97.0.4692.71/chromedriver_linux64.zip
sudo unzip chromedriver_linux64.zip chromedriver -d /usr/local/bin
After that, my Selenium calls were able to work again. The final version combo:
# Below combo works
Python 3.7.3
selenium==3.141.0
ChromeDriver 97.0.4692.71
Google Chrome 97.0.4692.71
Make sure that both the chromedriver and google-chrome executable have execute permissions
sudo chmod -x "/usr/bin/chromedriver"
sudo chmod -x "/usr/bin/google-chrome"

Python test using selenium cannot perform simple test

I am trying to learn about Selenium but I am not able to get even a simple program to test. Selenium webdriver seems to not be cooperating with Firefox and I am very frustrated, so I come to Stack Overflow for help.
For background, I use Python, can install with pip, and know command line.
I am on windows 10, firefox 48, and selenium webdriver 3 with python 3.5.2
Whenever I run the selenium test, (it opens a Firefox windows and the selenium website)
from selenium import webdriver
browser = webdriver.Firefox()
browser.get('http://www.seleniumhq.org')
I always get an error:
selenium.common.exceptions.WebDriverException: Message: Can't load the profile. Profile Dir: C:\ ... \AppData\Local\Temp\tmp68m5rtwt If you specified a log_file in the FirefoxBinary constructor, check it for details
It also opens a firefox window and has the link of about:blank&utm_content=firstrun(not a valid url)
I have looked across the internet for a similar situation, but nothing really close. I also tried many tutorials and made sure that I installed selenium the right way. I noticed that firefox was recently updated, but I am not sure if this has any effect.
I would appreciate any help for this, and instructions for what I should do.
Firefox 48+ doesn't support webdriver.Firefox().
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
caps = DesiredCapabilities.FIREFOX
caps["marionette"] = True
caps["binary"] = "path/to/your/firefox"
browser = webdriver.Firefox(capabilities=caps)
browser.get('http://www.seleniumhq.org')
This is what I was trying
1. download geckodriver.https://github.com/mozilla/geckodriver/releases. v.0.10.0is for selenium 3(beta).
2. add PATH your geckodriver.
4. rename it to wires
5. restart shell
6. check the version
$ wires --version
7. and run above the code.

Categories

Resources