Develop Reference

Selenium script crashed and will no longer work; session not created from tab crashed

When dealing with some memory issues, my script was interrupted and now it will not restart. I am getting an error "selenium.common.exceptions.SessionNotCreatedException: Message: session not created from tab crashed (Session info: headless chrome=84.0.4147.135)"
I read a few other similar questions (this one is almost identical), but the solutions do not resolve my problem.
Here is my code:
from selenium import webdriver
chromeOptions = webdriver.ChromeOptions()
chromeOptions.add_argument("--headless")
chromeOptions.add_argument("--remote-debugging-port=9222")
chromeOptions.add_argument('--no-sandbox')
chromeOptions.add_experimental_option('prefs', {'intl.accept_languages': 'en,en_US',
"download.default_directory": download_location,
"download.directory_upgrade": True})
webdriver_location = '/usr/bin/chromedriver'
driver = webdriver.Chrome(webdriver_location,options=chromeOptions)
I attempted to add the chromeOptions.add_argument('--disable-dev-shm-usage') and received the same error.
Because the setup was running perfectly moments before and no updates have been done, I do not believe any versioning is the problem. I'm running on a Centos 7 server. I originally received the error with Google Chrome 84 and Chromedriver 84. Same error occurs when upgrading both to 85.
Here is the full error:
Traceback (most recent call last):
File "server_scraper_javascript_v2.py", line 323, in <module>
driver = webdriver.Chrome(webdriver_location,options=chromeOptions)
File "/home/education01/scraper/env/lib64/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 76, in __init__
RemoteWebDriver.__init__(
File "/home/education01/scraper/env/lib64/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "/home/education01/scraper/env/lib64/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/home/education01/scraper/env/lib64/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/home/education01/scraper/env/lib64/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created
from tab crashed
(Session info: headless chrome=84.0.4147.135)
UPDATE:
I do not have experience using Chromium without Selenium and a webdriver, but if I run chrome --headless --disable-gpu --dump-dom https://www.chromestatus.com/, it fails and I receive this:
#
# Fatal error in , line 0
# Fatal process out of memory: Failed to reserve memory for new V8 Isolate
#
#
#
#FailureMessage Object: 0x7ffc0fc7ed50#0 0x5619c34ebdd9 <unknown>
--2020-08-27 10:20:27-- https://clients2.google.com/cr/report
Resolving clients2.google.com (clients2.google.com)... 172.217.165.142, 2607:f8b0:4006:808::200e
Connecting to clients2.google.com (clients2.google.com)|172.217.165.142|:443... connected.
HTTP request sent, awaiting response... [0827/102028.181052:ERROR:headless_shell.cc(391)] Abnormal renderer termination.
200 OK
Length: unspecified [text/plain]
Saving to: ‘/dev/fd/4’
0K Crash dump id: 80939f7cd5646339

I recently ran into this issue where a script I call regularly was working, and then failed with the same "session not created from tab" error. I hadn't updated chrome or the chromedriver so the other solutions I found didn't work. I found that the issue was that I wasn't quitting the browsers at the end of my script so they were all still active.
I checked the active pid's for chrome with pgrep chrome and saw that I had a bunch of chrome processes running. I killed them all with pkill chrome and that fixed the issue. I added driver.quit() to the end of the script to kill the browser when it's done.
I'm not sure if that will solve your issue, but the memory error you got when trying to run google-chrome --headless made me think it might be the same issue I had.

Python, Variables in 2 files and selenium

I’ve created the main.py and the login.py, then I tried to link the 2 files together. I don’t know how to correctly link these 2 files.
I’m using selenium and the program works, but it opens 2 chrome windows (When I want to open just one) then, the second one goes ahead and works perfectly but when it stops to do the login file suddently this error comes:
Traceback (most recent call last):
File "C:/Users/alebu/PycharmProjects/selenium/Main.py", line 9, in <module>
Login.login()
File "C:\Users\alebu\PycharmProjects\selenium\Login.py", line 6, in login
Main.browser.find_element_by_link_text('Log in').click()
File "C:\Users\alebu\PycharmProjects\selenium\venv\lib\site-
packages\selenium\webdriver\remote\webdriver.py", line 419, in
find_element_by_link_text
return self.find_element(by=By.LINK_TEXT, value=link_text)
File "C:\Users\alebu\PycharmProjects\selenium\venv\lib\site-
packages\selenium\webdriver\remote\webdriver.py", line 955, in find_element
'value': value})['value']
File "C:\Users\alebu\PycharmProjects\selenium\venv\lib\site-
packages\selenium\webdriver\remote\webdriver.py", line 312, in execute
self.error_handler.check_response(response)
File "C:\Users\alebu\PycharmProjects\selenium\venv\lib\site-
packages\selenium\webdriver\remote\errorhandler.py", line 242, in
check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element:
Unable to locate element: {"method":"link text","selector":"Log in"}
(Session info: chrome=64.0.3282.167)
(Driver info: chromedriver=2.35.528161
(5b82f2d2aae0ca24b877009200ced9065a772e73),platform=Windows NT 10.0.16299
x86_64)
Process finished with exit code 1
probably I’ve done something wrong with the variables
the main.py
from selenium import webdriver
import Login
driver_location = "C:\webDrivers\chromedriver.exe"
options = webdriver.ChromeOptions()
options.add_argument('--lang=en')
browser = webdriver.Chrome(executable_path=driver_location,
chrome_options=options)
Login.login()
the login.py
def login():
import Main
from time import sleep
Main.browser.get('https://www.instagram.com')
Main.browser.find_element_by_link_text('Log in').click()
Main.browser.find_element_by_name('username').send_keys('*******')
Main.browser.find_element_by_name('password').send_keys(********')
Main.browser.find_element_by_xpath('//form/span/button[text()="Log
in"]').click()
sleep(3)
Main.browser.find_element_by_link_text('Not Now').click()
sleep(2)
print("Logged In")
The weird thing is: before the program was in one unique file and it worked perfectly.

I would suggest you read some material on how import works in python here.
to get what you have working properly quickly, you should not import Main in your login function. Try passing the driver as an argument in your login function as so (Note: after passing the browser as an argument and not importing Main, you will no longer need to use Main.browser, just browser):
from time import sleep
def login(browser):
browser.get('https://www.instagram.com')
Then when you go to call login from your main, you will want to pass browser as an argument:
Login.login(browser)
This should fix the problem you were having with two browsers opening. If the issue is still occurring with the Log in link not found please read this and ask a new question if you have a clear understanding of how it works and require more help.

Python script runs in console but errors out as a script

I'm working on a script to pull some information from a site that I must login to use. I'm using Python 2.7.12 and Selenium 3.4.3.
#!/usr/bin/python
from selenium import webdriver
browser = webdriver.Firefox(firefox_binary='/usr/bin/firefox', executable_path="./geckodriver")
# Get to the login page
browser.get('https://example.com')
browser.find_element_by_link_text('Application').click()
# Login
browser.find_element_by_id('username').send_keys('notmyusername')
browser.find_element_by_id('password').send_keys('notmypassword')
browser.find_element_by_css_selector('.btn').click()
# Open the application
browser.find_element_by_id('ctl00_Banner1_ModuleList_ctl01_lnkModule').click()
If I copy this code and paste it into the python console, it runs just fine and goes to the page I want. However, when I run the script from the terminal (bash on Linux Mint 18), it errors out. Here's the output with the try and catch statements removed:
Traceback (most recent call last):
File "./script.py", line 14, in <module>
browser.find_element_by_id('ctl00_Banner1_ModuleList_ctl01_lnkModule').click()
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 289, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 791, in find_element
'value': value})['value']
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 256, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: [id="ctl00_Banner1_ModuleList_ctl01_lnkModule"]
I don't even know how to go about troubleshooting this. Any help?

What is most probably happening is that when you run the script from bash, the script runs too quickly and the get_by_id operation is started before the browser has finished loading the page, which results in this error.
As #murali-selenium suggested, you should probably add some wait time before starting to look for stuff in the document.
That can be achived this way:
#!/usr/bin/python
from selenium import webdriver
import time
wait_time_sec = 1
browser = webdriver.Firefox(firefox_binary='/usr/bin/firefox', executable_path="./geckodriver")
# Get to the login page
browser.get('https://example.com')
time.sleep(wait_time_sec)
browser.find_element_by_link_text('Application').click()
time.sleep(wait_time_sec)
# Login
browser.find_element_by_id('username').send_keys('notmyusername')
browser.find_element_by_id('password').send_keys('notmypassword')
browser.find_element_by_css_selector('.btn').click()
time.sleep(wait_time_sec)
# Open the application
try:
browser.find_element_by_id('ctl00_Banner1_ModuleList_ctl01_lnkModule').click()
except:
print('failed')
#browser.stop()

Per selenium docs, setting browser.implicity_wait(10) # seconds tells the browser to poll the page for 10 seconds before deciding the element isn't there. By default it is set to 0 seconds. Once it is set, the setting persists for the life of the webdriver object.
There are a lot of other nifty tools to wait for elements to load, documented at readthedocs.io.

python selenium does not scale with firefox - Message: Can't load the profile. Profile Dir

Why does this happen using selenium with python and firefox? Selenium cant scale to a simple loop. Why cant firefox scale? Over time it decides to thow an error and quit.
Mozilla Firefox 45.0
selenium.version
'2.53.2'
from pyvirtualdisplay import Display
display = Display(visible=0, size=(800, 600))
display.start()
mylist = ['page1','page1',...,'pageN']
while True:
for data in mylist:
browser = webdriver.Firefox()
try:
myfunc(data) # Code that uses selenum to fetch pages
except:
pass
browser.quit()
time.sleep(60)
No handlers could be found for logger "sentry.errors.serializer"
Traceback (most recent call last):
File "platrieveerp.py", line 231, in <module>
browser = webdriver.Firefox()
File "/usr/local/lib/python2.7/dist-packages/selenium-2.53.2-py2.7.egg/selenium/webdriver/firefox/webdriver.py", line 103, in __init__
self.binary, timeout)
File "/usr/local/lib/python2.7/dist-packages/selenium-2.53.2-py2.7.egg/selenium/webdriver/firefox/extension_connection.py", line 51, in __init__
self.binary.launch_browser(self.profile, timeout=timeout)
File "/usr/local/lib/python2.7/dist-packages/selenium-2.53.2-py2.7.egg/selenium/webdriver/firefox/firefox_binary.py", line 68, in launch_browser
self._wait_until_connectable(timeout=timeout)
File "/usr/local/lib/python2.7/dist-packages/selenium-2.53.2-py2.7.egg/selenium/webdriver/firefox/firefox_binary.py", line 106, in _wait_until_connectable
% (self.profile.path))
selenium.common.exceptions.WebDriverException: Message: Can't load the profile. Profile Dir: /tmp/tmpOgsKOx If you specified a log_file in the FirefoxBinary constructor, check it for details.

Selenium is not failing while running the loop, it is failing at webdriver.Firefox(). The error is recognized when the connection times out at a certain time after line 231 when the script still cant recognize an open firefox application.
It is just a bug in selenium when running with a headless browser. To get a working version of selenium, run
sudo pip install -U selenium
This should be a similar issue:
Selenium: FirefoxProfile exception Can't load the profile

"Permission Denied" error ruins Selenium scraping

I've been scraping a website using Selenium (Python Webdriver). When I try to have it click() an option, I get a permission denied error. Full stack trace:
Traceback (most recent call last):
File "scrape.py", line 19, in <module>
subjectOptions[1].click()
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webelement.py", line 45, in click
self._execute(Command.CLICK_ELEMENT)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webelement.py", line 194, in _execute
return self._parent.execute(command, params)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 153, in execute
self.error_handler.check_response(response)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 147, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: u"'Error: Permission denied for <http://localhost/scrape_test> to get property HTMLDocument.compatMode' when calling method: [wdIMouse::move]"
Here is the code that causes the problem. I know for a fact that the option I'm trying to click exists (based on print):
from selenium import webdriver
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait #available since 2.4.0
import time
# Create a new instance of the FireFox driver
driver = webdriver.Firefox()
# go to the local version of the page for testing
driver.get("http://localhost/scrape_test")
# Find the select by ID, get its options
selectElement = driver.find_element_by_id("CLASS_SRCH_WRK2_SUBJECT$65$")
subjectOptions = selectElement.find_elements_by_tag_name("option")
# Click the desired option
subjectOptions[1].click()
I'm using Firefox 8.0.1 on Mac OS X 10.7.2

Looks like it's a webdriver bug. The latest log entry from the programmer who last modified one of the selenium source code files says:
This leads to permissions errors, which I've still been unable to
reduce:
Error: Permission denied for http://www.finn.no to get property
HTMLDocument.compatMode' when calling method: [wdIMouse::move]
There is some discussion about the issue here, here and here.
According to the discussion it should work fine with Firefox 7. Also, this related issue implies that the link is still clicked in spite of the error, so it might work inside a try/except.

For time being you can overcome with work around given here