I have the following code trying to use stashy, to pull and push to Bit Bucket, Well right now I have code that specifically will just display projects/repositories that are given to my account. Very simple.
import stashy
bitbucket = stashy.connect("https://bitbucket.domain.come:port/projects/asdasd/asd/asdadasd/", "username","password")
print(bitbucket.projects.list())
I keep getting this error without any hang or delay, What could be this issue, I am assuming its client sided as again I am not given any delay, making me assume its specifically my machine or my client specifically.
Traceback (most recent call last):
File "C:/Users/User/PycharmProjects/Project/ProjectName/test_no_gui.py", line 3, in <module>
print(bitbucket.projects.list())
File "C:\Users\User\PycharmProjects\Project\venv\lib\site-packages\stashy\helpers.py", line 78, in list
return list(self.all())
File "C:\Users\User\PycharmProjects\Project\venv\lib\site-packages\stashy\helpers.py", line 44, in paginate
response = self._client.get(url, **kw)
File "C:\Users\User\PycharmProjects\Project\venv\lib\site-packages\stashy\client.py", line 59, in get
return requests.get(self.url(resource), auth=(self._username, self._password), verify=self._verify, **kw)
File "C:\Users\User\PycharmProjects\Project\venv\lib\site-packages\requests\api.py", line 55, in get
return request('get', url, **kwargs)
File "C:\Users\User\PycharmProjects\Project\venv\lib\site-packages\requests\api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\User\PycharmProjects\Project\venv\lib\site-packages\requests\sessions.py", line 279, in request
resp = self.send(prep, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
File "C:\Users\User\PycharmProjects\Project\venv\lib\site-packages\requests\sessions.py", line 374, in send
r = adapter.send(request, **kwargs)
File "C:\Users\User\PycharmProjects\Project\venv\lib\site-packages\requests\adapters.py", line 174, in send
timeout=timeout
File "C:\Users\User\PycharmProjects\Project\venv\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 417, in urlopen
conn = self._get_conn(timeout=pool_timeout)
File "C:\Users\User\PycharmProjects\Project\venv\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 232, in _get_conn
return conn or self._new_conn()
File "C:\Users\User\PycharmProjects\Project\venv\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 547, in _new_conn
strict=self.strict)
TypeError: __init__() got an unexpected keyword argument 'strict'
Looking at other libs/api's turns out stashy is depreciated? Please correct me if I am wrong. Gathered from other research/looking at best method/practice to get into bit bucket. Looking at atlassian-python-api instead, will report back.
Edit: Ended up using REST API, and Requests. All these libs, happened to be out of date.
Related
I have some really simple Python code that I'm using to do a GCP Google Vault export. However, about 4 out of ever 5 runs returns an EOF Error. The stacktrace is
Traceback (most recent call last):
File "src/initiate_job.py", line 57, in <module>
results = service.matters().list(pageSize=10).execute()
File "/Users/ipq500/opt/miniconda3/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
return wrapped(*args, **kwargs)
File "/Users/ipq500/opt/miniconda3/lib/python3.7/site-packages/googleapiclient/http.py", line 901, in execute
headers=self.headers,
File "/Users/ipq500/opt/miniconda3/lib/python3.7/site-packages/googleapiclient/http.py", line 177, in _retry_request
resp, content = http.request(uri, method, *args, **kwargs)
File "/Users/ipq500/opt/miniconda3/lib/python3.7/site-packages/google_auth_httplib2.py", line 190, in request
self._request, method, uri, request_headers)
File "/Users/ipq500/opt/miniconda3/lib/python3.7/site-packages/google/auth/credentials.py", line 133, in before_request
self.refresh(request)
File "/Users/ipq500/opt/miniconda3/lib/python3.7/site-packages/google/oauth2/service_account.py", line 359, in refresh
access_token, expiry, _ = _client.jwt_grant(request, self._token_uri, assertion)
File "/Users/ipq500/opt/miniconda3/lib/python3.7/site-packages/google/oauth2/_client.py", line 153, in jwt_grant
response_data = _token_endpoint_request(request, token_uri, body)
File "/Users/ipq500/opt/miniconda3/lib/python3.7/site-packages/google/oauth2/_client.py", line 105, in _token_endpoint_request
response = request(method="POST", url=token_uri, headers=headers, body=body)
File "/Users/ipq500/opt/miniconda3/lib/python3.7/site-packages/google_auth_httplib2.py", line 117, in __call__
url, method=method, body=body, headers=headers, **kwargs)
File "/Users/ipq500/opt/miniconda3/lib/python3.7/site-packages/httplib2/__init__.py", line 1994, in request
cachekey,
File "/Users/ipq500/opt/miniconda3/lib/python3.7/site-packages/httplib2/__init__.py", line 1651, in _request
conn, request_uri, method, body, headers
File "/Users/ipq500/opt/miniconda3/lib/python3.7/site-packages/httplib2/__init__.py", line 1621, in _conn_request
content = _decompressContent(response, content)
File "/Users/ipq500/opt/miniconda3/lib/python3.7/site-packages/httplib2/__init__.py", line 460, in _decompressContent
content = gzip.GzipFile(fileobj=io.BytesIO(new_content)).read()
File "/Users/ipq500/opt/miniconda3/lib/python3.7/gzip.py", line 276, in read
return self._buffer.read(size)
File "/Users/ipq500/opt/miniconda3/lib/python3.7/gzip.py", line 482, in read
raise EOFError("Compressed file ended before the "
EOFError: Compressed file ended before the end-of-stream marker was reached
The code that throws this is just:
service = build('vault', 'v1', credentials=delegated_credentials)
results = service.matters().list(pageSize=10).execute()
I tried to add some defensive programming wherein I simply retry anything that throws this error -- but now I'm hitting rate limits. So, I have to really debug why I'm getting this error.
Any and all help would be appreciated!
Based on this question and this conversation, It seems that the file that you read is damaged and the error of end of the file is expected, In this case I suggest to verify if the credentials exist, take the sample as references and also try to perform a double check before to perform any task, On the other hand, it does the same with the file for the file since it is possible that the writing of the file is not finished, including whether it is being compressed or decompressed
I have the following snippet of python (2.7.14) which is throwing an exception on the requests.get call. This is running on a FreeBSD server, with an apache front end and all http traffic being redirected to https. Note that I am not overly familiar with python, nor this code. Also the following works fine when running locally on my PC and without https/ssl.
urlstr = getApipath() + 'getData/' + id_data
logging.debug("URL used: %s", urlstr)
try:
resp = requests.get(urlstr)
except Exception as e:
logging.exception("caught exception str: " + str(e))
The urlstr is an api call and is using https. The same api call using curl on the same host's command line returns the expected and valid json.
curl https://my.domain.com/is/api/getData/D_01
The following is the log output including stacktrace when running this (obviously not using real url). Looks like an ssl issue, but the error message string being empty doesn't help. Any help on what the issue could be greatly appreciated.
2018-03-28 10:37:51,530 URL used: https://my.domain.org/is/api/getData/D_01
2018-03-28 10:37:51,531 Starting new HTTPS connection (1): my.domain.org
2018-03-28 10:37:51,540 caught excpetion str: []
Traceback (most recent call last):
File "/var/www/engines/controller/get_data.py", line 25, in getTemplate
resp = requests.get(urlstr)
File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/usr/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/usr/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
conn.connect()
File "/usr/local/lib/python2.7/site-packages/urllib3/connection.py", line 314, in connect
cert_reqs=resolve_cert_reqs(self.cert_reqs),
File "/usr/local/lib/python2.7/site-packages/urllib3/util/ssl_.py", line 254, in create_urllib3_context
context = SSLContext(ssl_version or ssl.PROTOCOL_SSLv23)
File "/usr/local/lib/python2.7/site-packages/urllib3/contrib/pyopenssl.py", line 379, in __init__
self._ctx = OpenSSL.SSL.Context(self.protocol)
File "/usr/local/lib/python2.7/site-packages/OpenSSL/SSL.py", line 724, in __init__
_openssl_assert(res == 1)
File "/usr/local/lib/python2.7/site-packages/OpenSSL/_util.py", line 67, in openssl_assert
exception_from_error_queue(error)
File "/usr/local/lib/python2.7/site-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
raise exception_type(errors)
Error: []
An upgrade to the python cryptography module fixed the issue. The installed version was 2.1.4. Turns out there was a newer version available - 2.2.2 which after installing cleared the problem.
I'm using the python response library to mock a call with requests, but I get this error:
File "/lib/python3.5/site-packages/requests/api.py", line 110, in post
return request('post', url, data=data, json=json, **kwargs)
File "/lib/python3.5/site-packages/requests/api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "/lib/python3.5/site-packages/requests/sessions.py", line 474, in request
prep = self.prepare_request(req)
File "/lib/python3.5/site-packages/requests/sessions.py", line 407, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "/lib/python3.5/site-packages/requests/models.py", line 306, in prepare
self.prepare_auth(auth, url)
File "/lib/python3.5/site-packages/requests/models.py", line 518, in prepare_auth
r = auth(self)
File "/lib/python3.5/site-packages/requests_oauthlib/oauth1_auth.py", line 88, in __call__
unicode(r.url), unicode(r.method), None, r.headers)
File "/lib/python3.5/site-packages/oauthlib/oauth1/rfc5849/__init__.py", line 313, in sign
('oauth_signature', self.get_oauth_signature(request)))
File "/lib/python3.5/site-packages/oauthlib/oauth1/rfc5849/__init__.py", line 150, in get_oauth_signature
sig = self.SIGNATURE_METHODS[self.signature_method](base_string, self)
File "/lib/python3.5/site-packages/oauthlib/oauth1/rfc5849/signature.py", line 505, in sign_rsa_sha1_with_client
return sign_rsa_sha1(base_string, client.rsa_key)
File "/lib/python3.5/site-packages/oauthlib/oauth1/rfc5849/signature.py", line 497, in sign_rsa_sha1
key = _prepare_key_plus(alg, rsa_private_key)
File "/lib/python3.5/site-packages/oauthlib/oauth1/rfc5849/signature.py", line 574, in _prepare_key_plus
return alg.prepare_key(keystr)
File "/lib/python3.5/site-packages/jwt/algorithms.py", line 169, in prepare_key
key = load_pem_public_key(key, backend=default_backend())
File "/lib/python3.5/site-packages/cryptography/hazmat/primitives/serialization.py", line 24, in load_pem_public_key
return backend.load_pem_public_key(data)
File "/lib/python3.5/site-packages/cryptography/hazmat/backends/multibackend.py", line 312, in load_pem_public_key
return b.load_pem_public_key(data)
File "/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1074, in load_pem_public_key
self._handle_key_loading_error()
File "/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1289, in _handle_key_loading_error
raise ValueError("Could not unserialize key data.")
ValueError: Could not unserialize key data
The code I try to mock is the following:
def function_to_test():
oauth = OAuth1(
self.consumer_key,
client_secret=self.consumer_secret,
resource_owner_key=self.oauth_token,
resource_owner_secret=self.oauth_token_secret,
rsa_key=self.rsa_key,
signature_method=self._signature_method
)
return requests.post(url="https://example.com", auth=oauth, cert="/path/to/certificate")
And the testing code:
#responses.activate
def test_token_expired(self):
responses.add(responses.POST, url='https://example.com/',
body='my_expected_result',
status=200)
response = function_to_test()
self.assertEqual(response.content, 'my_expected_result)
The test is failing with the error displayed above, does it mean that the responses mocking has not worked? Is it related to auth/cert parameters in my requests.post? Am I missing something?
I think that your issue is not related to responses library, but to OAuth1 initialisation.
Before even trying to mock the request, OAuth1 fails because he is trying to access the key, which is not probably correctly initialised in your test.
You can either give a fake value for this key, on even mock the OAuth1 object, responses will then be able to mock your call.
We could have expected responses to completely mock the requests call, but it seems that some initialisation still happens before that.
Trying to get Firebase set up and this code produces the error. I've also tried making the restful call simply using requests and I'm getting the exact same error. I'm using python 3.4. What's going on here?
from firebase import firebase
firebase = firebase.FirebaseApplication('https://testDB-72927.firebaseio.com/', authentication=None)
result = firebase.get('/test', None)
print(result)
Traceback (most recent call last):
File "/Users/Parthenon/Desktop/TestProject/Test.py", line 19, in <module>
firstFunc(mylist)
File "/Users/Parthenon/Desktop/TestProject/Test.py", line 14, in firstFunc
result = firebase.get('/test', None)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/firebase/decorators.py", line 19, in wrapped
return f(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/firebase/firebase.py", line 274, in get
return make_get_request(endpoint, params, headers, connection=connection)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/firebase/decorators.py", line 19, in wrapped
return f(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/firebase/firebase.py", line 38, in make_get_request
response = connection.get(url, params=params, headers=headers, timeout=timeout)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/sessions.py", line 310, in get
return self.request('GET', url, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/sessions.py", line 279, in request
resp = self.send(prep, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/sessions.py", line 374, in send
r = adapter.send(request, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/adapters.py", line 174, in send
timeout=timeout
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 417, in urlopen
conn = self._get_conn(timeout=pool_timeout)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 232, in _get_conn
return conn or self._new_conn()
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 547, in _new_conn
strict=self.strict)
TypeError: __init__() got an unexpected keyword argument 'strict'
My requests module was not up to date.
sudo pip install requests --upgrade
So I have a website which has a cookie with an integer in it, causing Requests to fail:
import requests
r = requests.get('http://www.c1000.nl/kies-uw-winkel.aspx')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/srv/www/li/venv/local/lib/python2.7/site-packages/requests/api.py", line 55, in get
return request('get', url, **kwargs)
File "/srv/www/li/venv/local/lib/python2.7/site-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/srv/www/li/venv/local/lib/python2.7/site-packages/requests/sessions.py", line 361, in request
resp = self.send(prep, **send_kwargs)
File "/srv/www/li/venv/local/lib/python2.7/site-packages/requests/sessions.py", line 464, in send
r = adapter.send(request, **kwargs)
File "/srv/www/li/venv/local/lib/python2.7/site-packages/requests/adapters.py", line 369, in send
r = self.build_response(request, resp)
File "/srv/www/li/venv/local/lib/python2.7/site-packages/requests/adapters.py", line 179, in build_response
extract_cookies_to_jar(response.cookies, req, resp)
File "/srv/www/li/venv/local/lib/python2.7/site-packages/requests/cookies.py", line 117, in extract_cookies_to_jar
jar.extract_cookies(res, req)
File "/usr/lib/python2.7/cookielib.py", line 1645, in extract_cookies
self.set_cookie(cookie)
File "/srv/www/li/venv/local/lib/python2.7/site-packages/requests/cookies.py", line 270, in set_cookie
if cookie.value.startswith('"') and cookie.value.endswith('"'):
AttributeError: 'int' object has no attribute 'startswith'
Is this a bug in Requests? (i.e. shouldn't it be able to handle integers?)
How can I avoid it? Is there a thing in a get request for avoid cookies?
System info:
Ubuntu 12.04LTS 64bit
Python 2.7.3
Request 2.0.1
This is a known bug, tracked by this GitHub issue. It will be fixed in the next release of Requests.
To handle cookies you should use a session that stores cookies between requests:
s = requests.Session()
r = s.get('http://www.c1000.nl/kies-uw-winkel.aspx',timeout=20)
This should do the trick. Im not sure why your code sample throws this exception though.. just a quick workaround.