Trying to learn how to use selenium, I managed to overcome first error which involved chrome driver not being in the path name but it has thrown up another error.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome('/Users/williamneal/Scratch/Titanic/chromedriver')
driver.get("http://www.bbc.com")
The error:
Traceback (most recent call last):
File "<ipython-input-1-84256e62b8db>", line 5, in <module>
driver = webdriver.Chrome('/Users/williamneal/Scratch/Titanic/chromedriver')
File "/Users/williamneal/anaconda/lib/python3.5/site-packages/selenium/webdriver/chrome/webdriver.py", line 62, in __init__
self.service.start()
File "/Users/williamneal/anaconda/lib/python3.5/site-packages/selenium/webdriver/common/service.py", line 64, in start
stdout=self.log_file, stderr=self.log_file)
File "/Users/williamneal/anaconda/lib/python3.5/subprocess.py", line 950, in __init__
restore_signals, start_new_session)
File "/Users/williamneal/anaconda/lib/python3.5/subprocess.py", line 1544, in _execute_child
raise child_exception_type(errno_num, err_msg)
OSError: [Errno 8] Exec format error
There is a potential solution here, which involves installing Chrome Drivers via Home Brew but that option is not available to me.
Any ideas?
Looks like this is complaining about the format of chromedriver binary.
It might be because of platform and chromedriver format mismatch. For example windows requires chromedriver.exe while there are different formats for linux and mac.
If you don't want to install through package manager, just download chromedriver from https://sites.google.com/a/chromium.org/chromedriver/downloads
Note : Choose file as per your os
Then place it anywhere on the os and pass that path as an argument. You can also set webdriver.chrome.driver environment variable if you don't want to pass the location every time.
FYI you could also encounter this issue if you did not unzip the chromedriver before adding it to your PATH.
Related
I am using firefox gecko driver to make a bot, I keep getting error messages with my bot and I have found that the source of the message is the:
from selenium import webdriver
browser= webdriver.Firefox()
lines.
I have added every file mentioned in the error message to path including gecko driver, firefox, and the other smaller documents. I am at a complete loss
Here is the error message:
Traceback (most recent call last):
File "/usr/local/bin/fuckobot1.py", line 3, in <module>
browser= webdriver.Firefox()
File "/Users/'myname'/Library/Python/2.7/lib/python/site-packages/selenium/webdriver/firefox/webdriver.py", line 164, in __init__
self.service.start()
File "/Users/'myname'/Library/Python/2.7/lib/python/site-packages/selenium/webdriver/common/service.py", line 76, in start
stdin=PIPE)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
raise child_exception
OSError: [Errno 8] Exec format error
This error message...
OSError: [Errno 8] Exec format error
...implies that the GeckoDriver binary which was invoked was not in proper format.
Your main issue is the incompatibility of the GeckoDriver binary format with respect to the underlying Operating System.
As you are on MacOS you need to download geckodriver-v0.23.0-macos.tar.gz from mozilla/geckodriver, untar/unzip it and provide the absolute path of the GeckoDriver through the argument executable_path as follows:
from selenium import webdriver
browser= webdriver.Firefox(executable_path='/path/to/geckodriver')
This may be answer to this issue,
when it shows something like, Your Firefox profile cannot be loaded. It may be missing or inaccessible.
Actual problem is, Firefox 48.0 or greater not works with selenium 3.0.0 or greater version.
so please check your versions.
so, to make it work you need to update with recent selenium and respective geckodriver version too that fixes this issue.
then try basic commands,
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
I have created a test script to open a url in Eclipse using python and got the following error:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 76, in start
stdin=PIPE)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 769, in __init__
restore_signals, start_new_session)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 1516, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'geckodriver': 'geckodriver'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Applications/Eclipse.app/Contents/MacOS/C:\EclipseWorkspaces\csse120/PythonSeleniumProject/src/PythonSeleniumModule.py", line 13, in <module>
driver = webdriver.Firefox()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 164, in __init__
self.service.start()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 83, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable needs to be in PATH.
I have read in stack overflow about related topics but none of them answers/solves my problem.
Please advise.
Thank you.
This error message...
FileNotFoundError: [Errno 2] No such file or directory: 'geckodriver': 'geckodriver'
.
selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable needs to be in PATH.
...implies that your program was unable to locate the GeckoDriver within the mentioned directory.
As per your code trials you have used:
driver = webdriver.Firefox()
As you havn't mentioned the absolute path of the GeckoDriver explicitly, your program searches for the GeckoDriver within the paths mentioned within your underlying Operating System PATH variable and unable to locate.
Solution
As you are on Mac OS X download the latest geckodriver-v0.23.0-macos.tar.gz from mozilla/geckodriver, store it anywhere within your system.
In your program override the paths mentioned in your Operating System PATH variable through the argument executable_path as follows:
from selenium import webdriver
driver = webdriver.Firefox(executable_path='/path/to/geckodriver')
print("Firefox Browser Invoked")
driver.get('http://google.com/')
driver.quit()
The solution above will not work on every machine depending on the absolute path you choose. Also, the absolute paths that are easy to access via any program, e.g. root folder, requires admin permissions.
There's a DriverManager module to every Selenium WebDriver that exists, and you can use it to install the WebDriver on a directory inside PATH variable automatically.
It is important that you do the installing just once. Doing it again will cause errors, and I haven't found how to overcome them yet.
First, install webdriver-manager on your Python environment with
pip install webdriver-manager
or pip3 if you're using Python3.
Do this on your code and it will work fine:
from webdriver_manager.firefox import GeckoDriverManager
try:
driver = webdriver.Firefox()
except Exception:
driver = webdriver.Firefox(GeckoDriverManager().install())
Therefore it will install only once on the machine using the program.
Note: This has some issues with OperaDriver.
Tried adding the drivers to the right path. mentioned the path in the code and yet getting runtime errors with chromedriver. Cannot see what is wrong with the code. whats wrong?
Tried adding the path to the code. imported selenium and webdriver. placed the drivers in the python directory.
from selenium import webdriver
#to open the page in Chrome.firefox
driver = webdriver.Chrome("executable_path=D:\Python\Crawler\chromedriver_win32")
driver.get("http://econpy.pythonanywhere.com/ex/001.html")
given Below are the errors:
D:\Python\Crawler\venv\Scripts\python.exe D:/Python/Crawler/crawler_sel.py
Traceback (most recent call last):
File "D:\Python\Crawler\venv\lib\site-packages\selenium\webdriver\common\service.py", line 76, in start
stdin=PIPE)
File "C:\Users\vidwa\AppData\Local\Programs\Python\Python37-32\lib\subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "C:\Users\vidwa\AppData\Local\Programs\Python\Python37-32\lib\subprocess.py", line 1178, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:/Python/Crawler/crawler_sel.py", line 5, in <module>
driver = webdriver.Chrome("executable_path=D:\Python\Crawler\chromedriver_win32")
File "D:\Python\Crawler\venv\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 73, in __init__
self.service.start()
File "D:\Python\Crawler\venv\lib\site-packages\selenium\webdriver\common\service.py", line 83, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'chromedriver_win32' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
Process finished with exit code 1
Try the below this worked for me
from selenium import webdriver
#to open the page in Chrome.firefox
def set_up(self):
self.driver = webdriver.Chrome("D:/Python/Crawler/chromedriver_win32.exe")
Your syntax here is off. It should be:
driver = webdriver.Chrome(executable_path='D:/Python/Crawler/chromedriver_win32.exe')
I've taken quotation marks off executable_path, and added .exe extension to your file path.
Alternative solution -- If you do not want to use executable_path, you can try adding chromedriver_win32.exe to your Path variable:
from selenium import webdriver
driver = webdriver.Chrome()
driver.get("http://econpy.pythonanywhere.com/ex/001.html")
Then, in your environment variable for Path, you have something like:
D:\Python\Crawler\chromedriver_win32.exe
This guide may help you with the Path part: https://developers.refinitiv.com/sites/default/files/How%20To%20Add%20ChromeDriver%20To%20System%20Variables_0.pdf
I have created a test script to open a url in Eclipse using python and got the following error:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 76, in start
stdin=PIPE)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 769, in __init__
restore_signals, start_new_session)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 1516, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'geckodriver': 'geckodriver'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Applications/Eclipse.app/Contents/MacOS/C:\EclipseWorkspaces\csse120/PythonSeleniumProject/src/PythonSeleniumModule.py", line 13, in <module>
driver = webdriver.Firefox()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 164, in __init__
self.service.start()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 83, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable needs to be in PATH.
I have read in stack overflow about related topics but none of them answers/solves my problem.
Please advise.
Thank you.
This error message...
FileNotFoundError: [Errno 2] No such file or directory: 'geckodriver': 'geckodriver'
.
selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable needs to be in PATH.
...implies that your program was unable to locate the GeckoDriver within the mentioned directory.
As per your code trials you have used:
driver = webdriver.Firefox()
As you havn't mentioned the absolute path of the GeckoDriver explicitly, your program searches for the GeckoDriver within the paths mentioned within your underlying Operating System PATH variable and unable to locate.
Solution
As you are on Mac OS X download the latest geckodriver-v0.23.0-macos.tar.gz from mozilla/geckodriver, store it anywhere within your system.
In your program override the paths mentioned in your Operating System PATH variable through the argument executable_path as follows:
from selenium import webdriver
driver = webdriver.Firefox(executable_path='/path/to/geckodriver')
print("Firefox Browser Invoked")
driver.get('http://google.com/')
driver.quit()
The solution above will not work on every machine depending on the absolute path you choose. Also, the absolute paths that are easy to access via any program, e.g. root folder, requires admin permissions.
There's a DriverManager module to every Selenium WebDriver that exists, and you can use it to install the WebDriver on a directory inside PATH variable automatically.
It is important that you do the installing just once. Doing it again will cause errors, and I haven't found how to overcome them yet.
First, install webdriver-manager on your Python environment with
pip install webdriver-manager
or pip3 if you're using Python3.
Do this on your code and it will work fine:
from webdriver_manager.firefox import GeckoDriverManager
try:
driver = webdriver.Firefox()
except Exception:
driver = webdriver.Firefox(GeckoDriverManager().install())
Therefore it will install only once on the machine using the program.
Note: This has some issues with OperaDriver.
I'm running on Linux Debian 9 and PyCharmwith web scraping purposes; I'm currently using Python 3.5 as interpreter.
The script is the following:
from selenium import webdriver
import time
import datetime
from selenium.webdriver.common.keys import Keys
Till here, the script works fine, by importing the packages properly; when I try to set the driver by running the following line:
driver = webdriver.Firefox(executable_path='/home/quant/Desktop/DataDownload/venv/bin/geckodriver')
I get the following error message, relative to a format problem:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/home/quant/Desktop/DataDownload/venv/lib/python3.5/site-packages/selenium/webdriver/firefox/webdriver.py", line 157, in __init__
self.service.start()
File "/home/quant/Desktop/DataDownload/venv/lib/python3.5/site-packages/selenium/webdriver/common/service.py", line 76, in start
stdin=PIPE)
File "/usr/lib/python3.5/subprocess.py", line 676, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.5/subprocess.py", line 1282, in _execute_child
raise child_exception_type(errno_num, err_msg)
OSError: [Errno 8] Exec format error
Browsing on the web, I found that probably this problem comes out when the executable program has not been unzipped and made executable correctly; to do that, I followed the steps below, by running them on the terminal:
(1).zip file download from the official repository in the github website:
wget [here][1]
(2) unzipped the file:
cd /home/quant/Downloads
tar -xvzf geckodriver-v0.21.0-arm7hf.tar.gz
(3) made the file executable:
chmod +x geckodriver
(4) moved the file on the following path:
mv geckodriver /home/quant/PycharmProject/DataDownloads/venv/bin/
Could someone help me to understand what's wrong, please?
Thanks in advance all!!
[Errno 8] Exec format error
This means you are trying to run a version of geckodriver that is compiled for a different architecture... you downloaded the ARM version (geckodriver-v0.21.0-arm7hf.tar.gz) and are most likely running on a x86/amd64 machine.
Solution:
go back to the geckodriver releases page and download the correct version for your system: https://github.com/mozilla/geckodriver/releases.
For example, if you are running 64 bit Linux, you want to download: geckodriver-v0.21.0-linux64.tar.gz