Unable to run selenium on heroku with python script - python

I'm trying to run a simple python script that open firefox and search some info on https://biblioteca.aneel.gov.br/.
On my computer the script works, but when i try to run on heroku, it always crashes. I tried all tutorials that I could find, but I don't know what more to do.
I did:
I put as 'Buildpacks' on heroku: https://github.com/pyronlaboratory/heroku-integrated-firefox-geckodriver
In 'Config Vars' on heroku:
FIREFOX_BIN = /app/vendor/firefox/firefox
GECKODRIVER_PATH = /app/vendor/geckodriver/geckodriver
LD_LIBRARY_PATH = /usr/local/lib:/usr/lib:/lib:/app/vendor
PATH = /usr/local/bin:/usr/bin:/bin:/app/vendor/
Adjuste code on heroku:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from bs4 import BeautifulSoup
import re
import yagmail
import time
import os
options = webdriver.FirefoxOptions()
# enable trace level for debugging
options.log.level = "trace"
options.add_argument("-remote-debugging-port=9224")
options.add_argument("-headless")
options.add_argument("-disable-gpu")
options.add_argument("-no-sandbox")
binary = FirefoxBinary(os.environ.get('FIREFOX_BIN'))
rea_inicial='teste'
while True:
driver = webdriver.Firefox(firefox_binary=binary, executable_path=os.environ.get('GECKODRIVER_PATH'), options=options)
#driver.maximize_window()
driver.get("https://biblioteca.aneel.gov.br/Busca/Avancada")
print('entrando no site...')
#time.sleep(5)
# Busca por legislacoo
driver.find_element(By.XPATH, '/html/body/main/div/div/div[2]/div/div/button[2]').click()
wait = WebDriverWait(driver, 30)
print('chegou ate aqui')
Then, the error after deploy on heroku:
2021-11-05T17:40:55.035807+00:00 app[worker.1]: code.py:24: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
2021-11-05T17:40:55.035820+00:00 app[worker.1]: driver = webdriver.Firefox(firefox_binary=binary, executable_path=os.environ.get('GECKODRIVER_PATH'), options=options)
2021-11-05T17:40:55.035825+00:00 app[worker.1]: code.py:24: DeprecationWarning: firefox_binary has been deprecated, please pass in a Service object
2021-11-05T17:40:55.035825+00:00 app[worker.1]: driver = webdriver.Firefox(firefox_binary=binary, executable_path=os.environ.get('GECKODRIVER_PATH'), options=options)
2021-11-05T17:40:55.540760+00:00 app[worker.1]: Traceback (most recent call last):
2021-11-05T17:40:55.540774+00:00 app[worker.1]: File "code.py", line 24, in <module>
2021-11-05T17:40:55.540947+00:00 app[worker.1]: driver = webdriver.Firefox(firefox_binary=binary, executable_path=os.environ.get('GECKODRIVER_PATH'), options=options)
2021-11-05T17:40:55.540949+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.8/site-packages/selenium/webdriver/firefox/webdriver.py", line 175, in __init__
2021-11-05T17:40:55.541113+00:00 app[worker.1]: self.service.start()
2021-11-05T17:40:55.541123+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.8/site-packages/selenium/webdriver/common/service.py", line 101, in start
2021-11-05T17:40:55.541249+00:00 app[worker.1]: self.assert_process_still_running()
2021-11-05T17:40:55.541250+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.8/site-packages/selenium/webdriver/common/service.py", line 113, in assert_process_still_running
2021-11-05T17:40:55.541359+00:00 app[worker.1]: raise WebDriverException(
2021-11-05T17:40:55.541409+00:00 app[worker.1]: selenium.common.exceptions.WebDriverException: Message: Service /app/vendor/geckodriver/geckodriver unexpectedly exited. Status code was: 64
Can anyone helps me how to solve it ?

The following arguments:
-no-sandbox
-disable-gpu
perhaps are applicable to ChromeDriver / google-chrome combo but Not Applicable to GeckoDriver / firefox combo.
Remove the arguments and re-execute.

Related

AttributeError Py

New to Python Selenium.
I am trying to create an script to login to my home router and press the button restart.
Running to error, when trying to login to the router, can some on guide on my mistake here.
below is the code and also attaching the .screenshot
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
driver_service = Service(executable_path="C:\Program Files (x86)\chromedriver.exe")
driver = webdriver.Chrome(service=driver_service)
PASSWORD = 'testtes'
login_page = 'http://192.168.2.1/login.html'
driver.get(login_page)
driver.find_element_by_xpath("//input[#placeholder='Password']").send_keys(PASSWORD)
Below is the error I am getting.
Traceback (most recent call last):
File "C:\Users\admin\Desktop\pyhton\index.py", line 14, in
driver.find_element_by_xpath("//input[#placeholder='Password']").send_keys(PASSWORD)
AttributeError: 'WebDriver' object has no attribute 'find_element_by_xpath'
getting this error now.
Traceback (most recent call last):
File "C:\Users\admin\Desktop\pyhton\index.py", line 13, in
driver.find_element(By.XPATH, "//input[#placeholder='Password']").send_keys(PASSWORD)
File "C:\Python310\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 856, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Python310\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 429, in execute
self.error_handler.check_response(response)
File "C:\Python310\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//input[#placeholder='Password']"}
Probably you are using Selenium 4. if so, find_element_by_xpath and all the others find_element_by_* methods are not supported by Selenium 4, you have to use the new syntax and add an essential import, as following:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
driver_service = Service(executable_path="C:\Program Files (x86)\chromedriver.exe")
driver = webdriver.Chrome(service=driver_service)
PASSWORD = 'testtes'
login_page = 'http://192.168.2.1/login.html'
driver.get(login_page)
driver.find_element(By.XPATH, "//input[#placeholder='Password']").send_keys(PASSWORD)
Try this:
from selenium.webdriver.common.by import By
driver.find_element(By.XPATH, "//input[#placeholder='Password']").send_keys(PASSWORD)

Selenium.common.exceptions.WebDriverException: Message: unknown error: no chrome binary

I'm trying to run a python script I made a few months ago which uses selenium to scrape a web page. Here's my code:
import pandas as pd
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Chrome(executable_path="/users/aliallam/Documents/chromedriver")
Here's the full error I'm getting:
Traceback (most recent call last):
File "/Users/aliallam/Desktop/MISOS_Python_Scraper/main.py", line 16, in <module>
driver = webdriver.Chrome(executable_path="/users/aliallam/Documents/chromedriver")
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 76, in __init__
RemoteWebDriver.__init__(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot find Chrome binary
I tried the solution under this question, but still no luck: Selenium gives "selenium.common.exceptions.WebDriverException: Message: unknown error: cannot find Chrome binary" on Mac
Here's what I changed my code to:
options = webdriver.ChromeOptions()
options.binary_location = " /Applications/Google\ Chrome\ 2.app"
chrome_driver_binary = "/users/aliallam/Documents/chromedriver"
driver = webdriver.Chrome(chrome_driver_binary,chrome_options=options)
This is really frustrating, some help would be much appreciated! Thanks in advance!
EDIT: In addition to the following mistake, I think you also need to change the binary location to be /Applications/Google Chrome 2.app/Contents/MacOS/Google Chrome
Based on what you posted, I think the error is because of an extra space located after the first quotation mark in this line:
options.binary_location = " /Applications/Google\ Chrome\ 2.app"
Try changing that to:
options.binary_location = "/Applications/Google Chrome 2.app/Contents/MacOS/Google Chrome"
and rerunning the code.
Full code based on what you provided:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.binary_location = "/Applications/Google Chrome 2.app/Contents/MacOS/Google Chrome"
chrome_driver_binary = "/users/aliallam/Documents/chromedriver"
driver = webdriver.Chrome(chrome_driver_binary,chrome_options=options)
If that doesn't work, you could also try
from selenium import webdriver
options = webdriver.ChromeOptions()
options.binary_location = "/Applications/Google Chrome 2.app/Contents/MacOS/Google Chrome"
executable_path = "/users/aliallam/Documents/chromedriver"
driver = webdriver.Chrome(executable_path=executable_path, chrome_options=options)
For me on MacOS the error got solved when I moved my downloaded "Google Chrome" application from "Downloads" folder to "Applications" Folder

Selenium Python Error

I am trying to create a small test program that is able to login into gmail. So far the program opens up the website on chrome but then fails to actually type anything into the "Enter Email" form box. Also I am receiving an error within the shell which may provide insight into my question.
Driver Version: 2.40
Chrome Version: 67.0.3396.99
Below is the code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver= webdriver.Chrome('C:\chromedriver_win32\chromedriver.exe')
driver.get("http://mail.google.com")
emailid=driver.find_element_by_name("identifier")
emailid.send_keys("samplekeys")
Below is the error:
Traceback (most recent call last):
File "C:\gmail.py", line 7, in <module>
driver.get("http://mail.google.com")
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 332, in get
self.execute(Command.GET, {'url': url})
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 320, in execute
self.error_handler.check_response(response)
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: missing or invalid 'entry.level'
(Session info: chrome=67.0.3396.99)
(Driver info: chromedriver=2.27.440174 (e97a722caafc2d3a8b807ee115bfb307f7d2cfd9),platform=Windows NT 10.0.17134 x86_64)
Any help would be greatly appreciated!
try this :
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
browser = webdriver.Chrome(executable_path = r'C:/chromedriver_win32/chromedriver.exe')
browser.get("http://mail.google.com")
wait = WebDriverWait(browser, 10)
user_name = wait.until(EC.element_to_be_clickable((By.ID, 'identifierId')))
user_name.click()
user_name.send_keys("samplekeys")
You need to download the chromedriver, and put the path to your webdriver.Chrome(), for example my chromedriver are in /usr/local/bin/chromdriver
driver = webdriver.Chrome('/usr/local/bin/chromedriver')
or for widndows
driver = webdriver.Chrome(executable_path = r'C:\chromedriver_win32\chromedriver.exe')

selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH error with Headless Chrome

when i run my script , i got this error
Traceback (most recent call last):
File "C:\Users\ishaq\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\common\service.py", line 74, in start
stdout=self.log_file, stderr=self.log_file)
File "C:\Users\ishaq\AppData\Local\Programs\Python\Python36\lib\subprocess.py", line 707, in __init__
restore_signals, start_new_session)
File "C:\Users\ishaq\AppData\Local\Programs\Python\Python36\lib\subprocess.py", line 992, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/ishaq/AppData/Local/Programs/Python/Python36/headless.py", line 9, in <module>
driver = webdriver.Chrome(executable_path=os.path.abspath("chromedriver"), chrome_options=chrome_options)
File "C:\Users\ishaq\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 62, in __init__
self.service.start()
File "C:\Users\ishaq\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\common\service.py", line 81, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
here is my script
import os
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
chrome_options.binary_location =
r'C:\Users\ishaq\Desktop\chrome\chromedriver.exe'
driver = webdriver.Chrome(executable_path=os.path.abspath("chromedriver"),
chrome_options=chrome_options)
driver.get("http://www.duo.com")
magnifying_glass = driver.find_element_by_id("js-open-icon")
if magnifying_glass.is_displayed():
magnifying_glass.click()
else:
menu_button = driver.find_element_by_css_selector(".menu-trigger.local")
menu_button.click()
search_field = driver.find_element_by_id("site-search")
search_field.clear()
search_field.send_keys("Olabode")
search_field.send_keys(Keys.RETURN)
assert "Looking Back at Android Security in 2016" in driver.page_source
driver.close()
If we analyze the logs it seems the main issue is with in start os.path.basename(self.path) and subsequent error message:
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH
So it's clear from the error that the Python client was unable to locate the chromedriver executable binary.
You need to take care of a couple of things as follows:
chrome_options.binary_location : The parameter points to the chrome.exe not the chromedriver.exe
os.path.abspath("chromedriver") will pick up the file path of chromedriver but won't append chromedriver.exe at the end.
Here is the sample code on my windows-8 system to start google-chrome-headless:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
driver = webdriver.Chrome(chrome_options=chrome_options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get("http://www.duo.com")
print("Chrome Browser Initialized in Headless Mode")
driver.quit()
print("Driver Exited")

Selenium Python WebDriverException("The browser appears to have exited ") - No specific error message

I am try to run a selenium script on firefox using python. But I receive an unclear error. I have read topics about display, but that is not my case.
Command
xvfb-run /usr/bin/python2.7 /var/www/html/selenium-scripts/example.py >> /var/log/selenium 2>&1
Script
from __future__ import print_function
import logging
import time
import datetime
from pyvirtualdisplay import Display
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
binary = FirefoxBinary('/usr/local/firefox/firefox')
display = Display(visible=0, size=(1024, 768))
display.start()
print('>> TEST START')
browser = webdriver.Firefox(firefox_binary=binary, timeout=60)
browser.get("http://www.vandeel.com")
print('>> TEST ENDED')
driver.quit()
display.stop()
Error
>> TEST START
Traceback (most recent call last):
File "/var/www/html/selenium-scripts/example.py", line 16, in <module>
browser = webdriver.Firefox(firefox_binary=binary, timeout=60)
File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 78, in __init__
self.binary, timeout)
File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/firefox/extension_connection.py", line 51, in __init__
self.binary.launch_browser(self.profile, timeout=timeout)
File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/firefox/firefox_binary.py", line 68, in launch_browser
self._wait_until_connectable(timeout=timeout)
File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/firefox/firefox_binary.py", line 98, in _wait_until_connectable
raise WebDriverException("The browser appears to have exited "
selenium.common.exceptions.WebDriverException: Message: **The browser appears to have exited before we could connect. If you specified a log_file in the FirefoxBinary constructor, check it for details.**
Please advice

Categories

Resources