I am running a script with Selenium, Firefox headless in my linux server. It is running well for my server. But I cannot install/configure the same thing for another one.
I am getting this error for my python script:
Traceback (most recent call last):
File "cde.py", line 290, in <module>
acde.Run()
File "cde.py", line 76, in Run
self.driver.get(self.link_to_explore)
File "/home/dev/.local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 213, in get
self.execute(Command.GET, {'url': url})
File "/home/dev/.local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 199, in execute
response = self.command_executor.execute(driver_command, params)
File "/home/dev/.local/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 395, in execute
return self._request(command_info[0], url, body=data)
File "/home/dev/.local/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 426, in _request
resp = self._conn.getresponse()
File "/usr/lib/python2.7/httplib.py", line 1127, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 453, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 417, in _read_status
raise BadStatusLine(line)
httplib.BadStatusLine: ''
May be I am missing to install something dependency. Is it possible to clone the configuration for certain app and use same in another machine ?
Related
I'm trying to get started with Selenium with Chromedriver.
I've downloaded the latest driver for Linux_64 to
/usr/local/share/chromedriver
And links to /usr/local/bin/chromdriver and /usr/bin/chromedriver
Python code
#!/usr/bin/env python
from selenium import webdriver
browser = webdriver.Chrome()
browser.get('http://www.google.com/')
browser.save_screenshot('screenie.png')
browser.quit()
The file is executable
When running it the terminal gets idle. Only way to get some output is by pressing CTRL+C which the returns this
Traceback (most recent call last):
File "./test3.py", line 5, in <module>
browser = webdriver.Chrome()
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/chrome/webdriver.py", line 75, in __init__
desired_capabilities=desired_capabilities)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 154, in __init__
self.start_session(desired_capabilities, browser_profile)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 243, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 310, in execute
response = self.command_executor.execute(driver_command, params)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/remote_connection.py", line 466, in execute
return self._request(command_info[0], url, body=data)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/remote_connection.py", line 490, in _request
resp = self._conn.getresponse()
File "/usr/lib/python2.7/httplib.py", line 1136, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 453, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 409, in _read_status
line = self.fp.readline(_MAXLINE + 1)
File "/usr/lib/python2.7/socket.py", line 480, in readline
data = self._sock.recv(self._rbufsize)
KeyboardInterrupt
Any ideas?
While installing the Google Cloud SDK - Python, a httplib2.SSLHandshakeError keeps occuring. I have configured the unfilled_client_secrets.json (shown below the return). And this has not solved the HandshakeError.
Similar questions have been asked on here below, but none have been explicitly answered. Thank you, in advance for any help you might be able to provide.
~ $ ./google-cloud-sdk/install.sh Welcome to the Google Cloud SDK!
Traceback (most recent call last):
File
"/Users/rptrainor/./google-cloud-sdk/bin/bootstrapping/install.py",
line 206, in
main()
File "/Users/rptrainor/./google-cloud-sdk/bin/bootstrapping/install.py",
line 184, in main
Install(pargs.override_components, pargs.additional_components)
File
"/Users/rptrainor/./google-cloud-sdk/bin/bootstrapping/install.py",
line 130, in Install
_CLI.Execute(['--quiet', 'components', 'list'])
File "/Users/rptrainor/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py",
line 759, in Execute
self._HandleAllErrors(exc, command_path_string, specified_arg_names)
File
"/Users/rptrainor/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py",
line 737, in Execute
resources = args.calliope_command.Run(cli=self, args=args)
File "/Users/rptrainor/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py",
line 741, in Run
display_info=self.ai.display_info).Display()
File "/Users/rptrainor/google-cloud-sdk/lib/googlecloudsdk/calliope/display.py",
line 427, in Display
self._printer.Print(self._resources)
File "/Users/rptrainor/google-cloud-sdk/lib/googlecloudsdk/core/resource/resource_printer_base.py", line 251, in Print
for resource in resources:
File "/Users/rptrainor/google-cloud-sdk/lib/surface/components/list.py",
line 86, in Run
result = update_manager.List()
File "/Users/rptrainor/google-cloud-sdk/lib/googlecloudsdk/core/updater/update_manager.py",
line 516, in List
_, diff = self._GetStateAndDiff(command_path='components.list')
File
"/Users/rptrainor/google-cloud-sdk/lib/googlecloudsdk/core/updater/update_manager.py",
line 446, in _GetStateAndDiff
command_path=command_path)
File "/Users/rptrainor/google-cloud-sdk/lib/googlecloudsdk/core/updater/update_manager.py",
line 429, in _GetLatestSnapshot
*effective_url.split(','), command_path=command_path)
File "/Users/rptrainor/google-cloud-sdk/lib/googlecloudsdk/core/updater/snapshots.py",
line 165, in FromURLs
for url in urls]
File "/Users/rptrainor/google-cloud-sdk/lib/googlecloudsdk/core/updater/snapshots.py",
line 186, in _DictFromURL
response = installers.ComponentInstaller.MakeRequest(url, command_path)
File
"/Users/rptrainor/google-cloud-sdk/lib/googlecloudsdk/core/updater/installers.py",
line 283, in MakeRequest
return url_opener.urlopen(req, timeout=timeout)
File "/Users/rptrainor/google-cloud-sdk/lib/googlecloudsdk/core/url_opener.py",
line 69, in urlopen
return opener.open(req, data, timeout)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py",
line 404, in open
response = self._open(req, data)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py",
line 422, in _open
'_open', req) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py",
line 382, in _call_chain
result = func(*args) File "/Users/rptrainor/google-cloud-sdk/lib/googlecloudsdk/core/url_opener.py",
line 54, in https_open
return self.do_open(build, req)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py",
line 1181, in do_open
h.request(req.get_method(), req.get_selector(), req.data, headers) File
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py",
line 995, in request
self._send_request(method, url, body, headers)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py",
line 1029, in _send_request
self.endheaders(body)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py",
line 991, in endheaders
self._send_output(message_body)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py",
line 844, in _send_output
self.send(msg)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py",
line 806, in send
self.connect()
File "/Users/rptrainor/google-cloud-sdk/lib/third_party/httplib2/init.py",
line 1081, in connect
raise SSLHandshakeError(e)
httplib2.SSLHandshakeError: [Errno 1] _ssl.c:510: error:14090086:SSL >routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
{
"web":{
"client_id":"[[CLIENT_ID_IS_HERE]]",
"project_id":"[[PROJECT_ID_IS_HERE]]",
"auth_uri":"https://accounts.google.com/o/oauth2/auth",
"token_uri":"https://accounts.google.com/o/oauth2/token",
"auth_provider_x509_cert_url":"https://www.googleapis.com/oauth2/v1/certs",
"client_secret":"[[CLIENT_SECRET_IS_HERE]]"
}
}
Try updating Python to the last 2.7.x version. I could resolve the very same issue updating Python to 2.7.13.
One silly yet effective solution could be accessing these URL's via browser once and accepting their certificate.
As well check the time of your computer. If it is not appropriate i mean not in the current date. Server will not share the certificate.
I recently upgraded to 16.04.
I installed selenium==3.0.2 and now when I run the script, this is what I see:
Traceback (most recent call last):
File "aux.py", line 3, in <module>
browser = webdriver.Chrome()
File "/usr/local/lib/python3.4/site-packages/selenium/webdriver/chrome/webdriver.py", line 69, in __init__
desired_capabilities=desired_capabilities)
File "/usr/local/lib/python3.4/site-packages/selenium/webdriver/remote/webdriver.py", line 92, in __init__
self.start_session(desired_capabilities, browser_profile)
File "/usr/local/lib/python3.4/site-packages/selenium/webdriver/remote/webdriver.py", line 179, in start_session
response = self.execute(Command.NEW_SESSION, capabilities)
File "/usr/local/lib/python3.4/site-packages/selenium/webdriver/remote/webdriver.py", line 234, in execute
response = self.command_executor.execute(driver_command, params)
File "/usr/local/lib/python3.4/site-packages/selenium/webdriver/remote/remote_connection.py", line 408, in execute
return self._request(command_info[0], url, body=data)
File "/usr/local/lib/python3.4/site-packages/selenium/webdriver/remote/remote_connection.py", line 440, in _request
resp = self._conn.getresponse()
File "/usr/local/lib/python3.4/http/client.py", line 1227, in getresponse
response.begin()
File "/usr/local/lib/python3.4/http/client.py", line 386, in begin
version, status, reason = self._read_status()
File "/usr/local/lib/python3.4/http/client.py", line 356, in _read_status
raise BadStatusLine(line)
http.client.BadStatusLine: ''
and this is all in the script:
from selenium import webdriver
# from selenium.webdriver.common.keys import Keys
browser = webdriver.Chrome()
browser.get('google.com')
I'm using selenium on my local computer, but sometimes I get a socket error that I don't fully understand. So far, restarting the (Python) application has worked for me.
What could be causing this?
driver = webdriver.Firefox(firefox_profile=someProfile)
driver.get(someLink)
Stack trace:
Traceback (most recent call last):
File "/home/foo/.conda/envs/myenv3/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2885, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-2-cdc559ddfda1>", line 1, in <module>
driver.get(link)
File "/home/foo/.conda/envs/myenv3/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 245, in get
self.execute(Command.GET, {'url': url})
File "/home/foo/.conda/envs/myenv3/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 231, in execute
response = self.command_executor.execute(driver_command, params)
File "/home/foo/.conda/envs/myenv3/lib/python3.5/site-packages/selenium/webdriver/remote/remote_connection.py", line 395, in execute
return self._request(command_info[0], url, body=data)
File "/home/foo/.conda/envs/myenv3/lib/python3.5/site-packages/selenium/webdriver/remote/remote_connection.py", line 425, in _request
self._conn.request(method, parsed_url.path, body, headers)
File "/home/foo/.conda/envs/myenv3/lib/python3.5/http/client.py", line 1083, in request
self._send_request(method, url, body, headers)
File "/home/foo/.conda/envs/myenv3/lib/python3.5/http/client.py", line 1128, in _send_request
self.endheaders(body)
File "/home/foo/.conda/envs/myenv3/lib/python3.5/http/client.py", line 1079, in endheaders
self._send_output(message_body)
File "/home/foo/.conda/envs/myenv3/lib/python3.5/http/client.py", line 911, in _send_output
self.send(msg)
File "/home/foo/.conda/envs/myenv3/lib/python3.5/http/client.py", line 854, in send
self.connect()
File "/home/foo/.conda/envs/myenv3/lib/python3.5/http/client.py", line 826, in connect
(self.host,self.port), self.timeout, self.source_address)
File "/home/foo/.conda/envs/myenv3/lib/python3.5/socket.py", line 711, in create_connection
raise err
File "/home/foo/.conda/envs/myenv3/lib/python3.5/socket.py", line 702, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
it looks the problem is related to some http libraries which ignores the proxy settings. what version of python you are using ? try using Python 2.x and see how it goes.
I want to create a web based scraper using Python, Selenium and PhantomJS where you can input a url into a form and the results from the scrape will be returned to the webpage. I can run it on my PC and I can also get it to work through the terminal.
It is located in a virtual environment on Dreamhost shared hosting with Python3.5 installed. I have tested that the parameters are being passed in fine, and it does work using just lxml and requests. However, when I try to run the script from the form on the webpage using PhantomJS then it doesn't work properly. The following error in returned...
Traceback (most recent call last):
File "testscrape.py", line 140, in <module>
driver = init_driver()
File "testscrape.py", line 69, in init_driver
driver = webdriver.PhantomJS(executable_path=phantomPATH,desired_capabilities=dcap)
File "/home/paul/.python35/bin/magenv/lib/python3.5/site-packages/selenium/webdriver/phantomjs/webdriver.py", line 56, in __init__
desired_capabilities=desired_capabilities)
File "/home/paul/.python35/bin/magenv/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 91, in __init__
self.start_session(desired_capabilities, browser_profile)
File "/home/paul/.python35/bin/magenv/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in start_session
'desiredCapabilities': desired_capabilities,
File "/home/paul/.python35/bin/magenv/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 231, in execute
response = self.command_executor.execute(driver_command, params)
File "/home/paul/.python35/bin/magenv/lib/python3.5/site-packages/selenium/webdriver/remote/remote_connection.py", line 395, in execute
return self._request(command_info[0], url, body=data)
File "/home/paul/.python35/bin/magenv/lib/python3.5/site-packages/selenium/webdriver/remote/remote_connection.py", line 463, in _request
resp = opener.open(request, timeout=self._timeout)
File "/home/paul/.python35/lib/python3.5/urllib/request.py", line 465, in open
response = self._open(req, data)
File "/home/paul/.python35/lib/python3.5/urllib/request.py", line 483, in _open
'_open', req)
File "/home/paul/.python35/lib/python3.5/urllib/request.py", line 443, in _call_chain
result = func(*args)
File "/home/paul/.python35/lib/python3.5/urllib/request.py", line 1268, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/home/paul/.python35/lib/python3.5/urllib/request.py", line 1243, in do_open
r = h.getresponse()
File "/home/paul/.python35/lib/python3.5/http/client.py", line 1174, in getresponse
response.begin()
File "/home/paul/.python35/lib/python3.5/http/client.py", line 282, in begin
version, status, reason = self._read_status()
File "/home/paul/.python35/lib/python3.5/http/client.py", line 243, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/home/paul/.python35/lib/python3.5/socket.py", line 575, in readinto
return self._sock.recv_into(b)
ConnectionResetError: [Errno 104] Connection reset by peer
I have tried a few different variations of desired_capabilities and even changing file permissions of everything in the virtual environment but to no avail. I must be missing something, or is it just not possible? Any suggestions gratefully received.