Using selenium 4.0.0 for python, I am getting an error
selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
(Session info: headless chrome=94.0.4606.81)
when the code tries to make a screenshot. The traceback is
...
File "/usr/local/lib/python3.8/site-packages/check_pages/page_dom_check.py", line 57, in make_full_screenshot
img = Image.open(BytesIO((driver.get_screenshot_as_png())))
File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 1064, in get_screenshot_as_png
return base64.b64decode(self.get_screenshot_as_base64().encode('ascii'))
File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 1074, in get_screenshot_as_base64
return self.execute(Command.SCREENSHOT)['value']
File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
I cannot post a small reproducible code, as it only happens when I run this within a gitlab CI. But it happens always when the screenshot is being made.
Maybe there is another way to make a screenshot?
I have no answer to my question, but I created a workaround that seems to work.
I was creating a selenium driver, and that single driver was opening different pages. Here I got the error.
# pseudocode
driver = ...
for check in checks:
driver.get(url)
# do other stuff with driver
driver.quit()
To work around this problem, create a new driver for each page individually.
# pseudocode
for check in checks:
driver = ...
driver.get(url)
# do other stuff with driver
driver.quit()
That way you can even put all the stuff around the driver in a try-except loop to try again
# pseudocode
for check in checks:
while ...
try:
driver = ...
driver.get(url)
# do other stuff with driver
driver.quit()
except UselessException:
pass
Related
I have a problem, every, single, TIME
Basically this is my code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.binary_location = "C:\Program Files (x86)\Google\Chrome Beta\Application\chrome.exe"
options.add_argument("--no-sandbox")
options.add_argument("--no-default-browser-check")
options.add_argument("--no-first-run")
options.add_argument("--disable-default-apps")
driver = webdriver.Chrome(options=options, executable_path="C:\Program Files (x86)\Google\Chrome Beta\Application\chromedriver.exe")
driver.get('https://www.youtube.com/')
and the error is
selenium.common.exceptions.SessionNotCreatedException: Message: session not created
I used Pycharm and tried to use VS code with python 3.4 and 3.7 and 3.8.3
plz help me I'm getting tired of this.
Full error Log:
Traceback (most recent call last):
File "C:/Users/um/PycharmProjects/Selenium/main.py", line 10, in <module>
driver = webdriver.Chrome(options=options, executable_path="C:\Program Files (x86)\Google\Chrome Beta\Application\chromedriver.exe")
File "C:\Users\um\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 76, in __init__
RemoteWebDriver.__init__(
File "C:\Users\um\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "C:\Users\um\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "C:\Users\um\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\um\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created
from disconnected: unable to send message to renderer
(Session info: chrome=81.0.4044.83)
Your error is not
unable to send message to renderer
as you mentioned. According to the stack trace you posted the problem in the line 6
driver = webdriver.Chrome(options=options, executable_path="chromedriver.exe", )
and the problem is:
selenium.common.exceptions.SessionNotCreatedException: Message: session not created
It seems like you specified an invalid path to the chromedriver and Selenium is unable to initialize the driver and create a session.
You should also check the version of the chromedriver with the guide.
Here you can download the valid version (I have taken your Chrome version from the stacktrace).
yes it did start and then close with the error log above, and also i have the correct version
I'm writing some code using Selenium, and at one point I make 7 requests, all to different websites. For the first one, this works fine. However, for others, I get a session ID error. I think that my browser is configured correctly, as I do get results from the first website. I have tried to put a WebDriverWait in between the requests, but to no avail. I think the websites might be blocking my requests. Does anyone have any idea how to solve this problem?
I'm sorry if this is something stupid or if I'm doing anything wrong, I'm quite new ^^
Thanks in advance!
Traceback (most recent call last):
File "/home/cena/PycharmProjects/Frikandelbroodje/main.py", line 56, in <module>
dirk_price = get_price(dirk_url, dirk_classname)
File "/home/cena/PycharmProjects/Frikandelbroodje/main.py", line 44, in get_price
browser.get(url)
File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
self.execute(Command.GET, {'url': url})
File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: invalid session id
(Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729#{#29}),platform=Linux 4.15.0-50-generic x86_64)
invalid session id
The invalid session ID error is a WebDriver error that occurs when the server does not recognize the unique session identifier. This happens if the session has been deleted or if the session ID is invalid.
A WebDriver session can be deleted through either of the following ways:
Explicit session deletion: A WebDriver session is explicitly deleted when explicitly invoking the quit() method as follows:
Code Block:
from selenium import webdriver
from selenium.common.exceptions import InvalidSessionIdException
driver = webdriver.Chrome(executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
print("Current session is {}".format(driver.session_id))
driver.quit()
try:
driver.get("https://www.google.com/")
except Exception as e:
print(e.message)
Console Output:
Current session is a9272550-c4e5-450f-883d-553d337eed48
No active session with ID a9272550-c4e5-450f-883d-553d337eed48
Implicit session deletion: A WebDriver session is implicitly deleted when you close the last window or tab invoking close() method as follows:
Code Block:
driver = webdriver.Chrome(executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
print("Current session is {}".format(driver.session_id))
# closes current window/tab
driver.close()
try:
driver.get("https://www.google.com/")
except Exception as e:
print(e.message)
Console Output:
Current session is a9272550-c4e5-450f-883d-553d337eed48
No active session with ID a9272550-c4e5-450f-883d-553d337eed48
Conclusion
As the first one request works fine but for others you get a session ID error most possibly the WebDriver controled Web Browser is getting detected and hence blocking the next requests.
There are different reasons for the WebDriver controled Web Browser to get detected and simultaneously get blocked. You can find a couple of detailed discussion in:
How does recaptcha 3 know I'm using selenium/chromedriver?
Selenium and non-headless browser keeps asking for Captcha
I got this error message because I was running Selenium in docker and I hadn't mounted enough swap memory, so it would crash after just a few pages.
To fix this, I used the same docker command, but added -v /dev/shm:/dev/shm after docker run.
If you had this
docker run -d -p 5901:5900 -p 127.0.0.1:4445:4444 selenium/standalone-chrome
then change to this
docker run -v /dev/shm:/dev/shm -d -p 5901:5900 -p 127.0.0.1:4445:4444 selenium/standalone-chrome
I found this info here, and here.
Browser page crash may leads to InvalidSessionIdException. Selenium says to us: session deleted because of page crash. Check if your browser page still exists when you got your errors.
Here an example of a traceback of this case:
[2021-06-28 15:05:43,787: ERROR/ForkPoolWorker-2] Message: invalid session id
Traceback (most recent call last):
...
File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
self.execute(Command.GET, {'url': url})
File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash
from tab crashed
(Session info: chrome=83.0.4103.61)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
...
File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 580, in find_elements_by_class_name
return self.find_elements(by=By.CLASS_NAME, value=name)
File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 1007, in find_elements
'value': value})['value'] or []
File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSessionIdException: Message: invalid session id
If you want some technical details, take a look at Chromium sources where you can find string session deleted because of page crash.
I had this problem and the reason was that I wrote url in wrong format - not like this, which is correct:
self.driver.get('https://twitter.com')
But this way:
self.driver.get('twitter.com')
Maybe you had the same issue. If not, just check all the links and make sure that all of them are in format of the correct one
in mycase the issue I executed driver.close() then tried to access current_url property of driver which is already closed
this what my wrong code leads to this error message:
url = 'http://localhost:5000/traning'
webpage = driver.get(url)
time.sleep(2)
driver.close()
return driver.current_url
this returns error:
selenium.common.exceptions.InvalidSessionIdException: Message: invalid
session id
and this is solution just save current url and all data in variable before close driver to return this data
driver = setDriver()
url = 'http://localhost:5000/traning'
webpage = driver.get(url)
current_url = driver.current_url
time.sleep(2)
driver.close()
return current_url
Try this:
driver.get('url')
#some code
current_page = driver.current_url
driver.close()
driver.quit()
time.sleep(10)
driver.get(current_page)
After I put driver.close with driver.quit it solve same issue, as you have
After closing the driver you have to reinitialize it in order to make it start again.
chrome_driver_path = "D:\chromedriver.exe"
s = Service(chrome_driver_path)
driver = webdriver.Chrome(service=s)
driver.get("https://google.com/")
driver.close()
driver = webdriver.Chrome(service=s)
driver.get("https://stackoverflow.com/")
I’ve created the main.py and the login.py, then I tried to link the 2 files together. I don’t know how to correctly link these 2 files.
I’m using selenium and the program works, but it opens 2 chrome windows (When I want to open just one) then, the second one goes ahead and works perfectly but when it stops to do the login file suddently this error comes:
Traceback (most recent call last):
File "C:/Users/alebu/PycharmProjects/selenium/Main.py", line 9, in <module>
Login.login()
File "C:\Users\alebu\PycharmProjects\selenium\Login.py", line 6, in login
Main.browser.find_element_by_link_text('Log in').click()
File "C:\Users\alebu\PycharmProjects\selenium\venv\lib\site-
packages\selenium\webdriver\remote\webdriver.py", line 419, in
find_element_by_link_text
return self.find_element(by=By.LINK_TEXT, value=link_text)
File "C:\Users\alebu\PycharmProjects\selenium\venv\lib\site-
packages\selenium\webdriver\remote\webdriver.py", line 955, in find_element
'value': value})['value']
File "C:\Users\alebu\PycharmProjects\selenium\venv\lib\site-
packages\selenium\webdriver\remote\webdriver.py", line 312, in execute
self.error_handler.check_response(response)
File "C:\Users\alebu\PycharmProjects\selenium\venv\lib\site-
packages\selenium\webdriver\remote\errorhandler.py", line 242, in
check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element:
Unable to locate element: {"method":"link text","selector":"Log in"}
(Session info: chrome=64.0.3282.167)
(Driver info: chromedriver=2.35.528161
(5b82f2d2aae0ca24b877009200ced9065a772e73),platform=Windows NT 10.0.16299
x86_64)
Process finished with exit code 1
probably I’ve done something wrong with the variables
the main.py
from selenium import webdriver
import Login
driver_location = "C:\webDrivers\chromedriver.exe"
options = webdriver.ChromeOptions()
options.add_argument('--lang=en')
browser = webdriver.Chrome(executable_path=driver_location,
chrome_options=options)
Login.login()
the login.py
def login():
import Main
from time import sleep
Main.browser.get('https://www.instagram.com')
Main.browser.find_element_by_link_text('Log in').click()
Main.browser.find_element_by_name('username').send_keys('*******')
Main.browser.find_element_by_name('password').send_keys(********')
Main.browser.find_element_by_xpath('//form/span/button[text()="Log
in"]').click()
sleep(3)
Main.browser.find_element_by_link_text('Not Now').click()
sleep(2)
print("Logged In")
The weird thing is: before the program was in one unique file and it worked perfectly.
I would suggest you read some material on how import works in python here.
to get what you have working properly quickly, you should not import Main in your login function. Try passing the driver as an argument in your login function as so (Note: after passing the browser as an argument and not importing Main, you will no longer need to use Main.browser, just browser):
from time import sleep
def login(browser):
browser.get('https://www.instagram.com')
Then when you go to call login from your main, you will want to pass browser as an argument:
Login.login(browser)
This should fix the problem you were having with two browsers opening. If the issue is still occurring with the Log in link not found please read this and ask a new question if you have a clear understanding of how it works and require more help.
This is my first web scraping project and I'm using selenium webdriver with Python in order to dynamically generate some csv files after choosing a few options on a website (though I'm not there yet).
However, I'm facing an unexpected timeout when the execution reaches a button click(). The click is performed but it gets stuck in there and does not continue the execution till the timeout.
Any clues on how to solve that?
Thanks!!
from selenium import webdriver
from selenium.webdriver.support.ui import Select
import time
driver = webdriver.Firefox()
driver.get('http://www8.receita.fazenda.gov.br/SimplesNacional/Aplicacoes/ATBHE/estatisticasSinac.app/Default.aspx')
driver.find_element_by_id('ctl00_ctl00_Conteudo_AntesTabela_lnkOptantesPorCNAE').click()
Select(driver.find_element_by_id("ctl00_ctl00_Conteudo_AntesTabela_ddlColuna")).select_by_visible_text("Município")
filtro_uf = driver.find_element_by_id('ctl00_ctl00_Conteudo_AntesTabela_btnFiltros')
for i in range (1, 28):
filtro_uf.click()
uf = Select(driver.find_element_by_id("ctl00_ctl00_Conteudo_AposTabela_ddlUf"))
uf.options[i].click()
time.sleep(2)
driver.find_element_by_id('chkTodosMunicipios').click()
time.sleep(2)
driver.find_element_by_xpath("//*[contains(text(),'Ok')]").click()
time.sleep(2)
# Here is where my code get stuck and gets a timeout
driver.find_element_by_id('ctl00_ctl00_Conteudo_AntesTabela_btnExibir').click()
The error I get:
Traceback (most recent call last):
File "/home/hissashi/Desktop/Python3/WS_SINAC/download_SINAC.py", line 22, in <module> driver.find_element_by_id('ctl00_ctl00_Conteudo_AntesTabela_btnExibir').click()
File "/home/hissashi/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "/home/hissashi/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webelement.py", line 501, in _execute
return self._parent.execute(command, params)
File "/home/hissashi/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 308, in execute
self.error_handler.check_response(response)
File "/home/hissashi/.local/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
raise exception_class(message, screen, stacktrace)
**selenium.common.exceptions.TimeoutException: Message: Timeout loading page after 300000ms**
I've found a workaround for the problem.
Apparently, the click() function blocks the code until the page is "completely" loaded. However, for some reason, the page keeps loading forever (without anything else to load) and it holds my code till it reaches the timeout limit.
Instead of using click, I've changed it to key ENTER and the page still keeps loading forever but it doesn't hold the code anymore.
#FROM CLICK
driver.find_element_by_id('ctl00_ctl00_Conteudo_AntesTabela_btnExibir').click()
#TO SENDING ENTER (ue007)
driver.find_element_by_id('ctl00_ctl00_Conteudo_AntesTabela_btnExibir').send_keys(u'\ue007')
I'm working on a script to pull some information from a site that I must login to use. I'm using Python 2.7.12 and Selenium 3.4.3.
#!/usr/bin/python
from selenium import webdriver
browser = webdriver.Firefox(firefox_binary='/usr/bin/firefox', executable_path="./geckodriver")
# Get to the login page
browser.get('https://example.com')
browser.find_element_by_link_text('Application').click()
# Login
browser.find_element_by_id('username').send_keys('notmyusername')
browser.find_element_by_id('password').send_keys('notmypassword')
browser.find_element_by_css_selector('.btn').click()
# Open the application
browser.find_element_by_id('ctl00_Banner1_ModuleList_ctl01_lnkModule').click()
If I copy this code and paste it into the python console, it runs just fine and goes to the page I want. However, when I run the script from the terminal (bash on Linux Mint 18), it errors out. Here's the output with the try and catch statements removed:
Traceback (most recent call last):
File "./script.py", line 14, in <module>
browser.find_element_by_id('ctl00_Banner1_ModuleList_ctl01_lnkModule').click()
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 289, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 791, in find_element
'value': value})['value']
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 256, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: [id="ctl00_Banner1_ModuleList_ctl01_lnkModule"]
I don't even know how to go about troubleshooting this. Any help?
What is most probably happening is that when you run the script from bash, the script runs too quickly and the get_by_id operation is started before the browser has finished loading the page, which results in this error.
As #murali-selenium suggested, you should probably add some wait time before starting to look for stuff in the document.
That can be achived this way:
#!/usr/bin/python
from selenium import webdriver
import time
wait_time_sec = 1
browser = webdriver.Firefox(firefox_binary='/usr/bin/firefox', executable_path="./geckodriver")
# Get to the login page
browser.get('https://example.com')
time.sleep(wait_time_sec)
browser.find_element_by_link_text('Application').click()
time.sleep(wait_time_sec)
# Login
browser.find_element_by_id('username').send_keys('notmyusername')
browser.find_element_by_id('password').send_keys('notmypassword')
browser.find_element_by_css_selector('.btn').click()
time.sleep(wait_time_sec)
# Open the application
try:
browser.find_element_by_id('ctl00_Banner1_ModuleList_ctl01_lnkModule').click()
except:
print('failed')
#browser.stop()
Per selenium docs, setting browser.implicity_wait(10) # seconds tells the browser to poll the page for 10 seconds before deciding the element isn't there. By default it is set to 0 seconds. Once it is set, the setting persists for the life of the webdriver object.
There are a lot of other nifty tools to wait for elements to load, documented at readthedocs.io.