Develop Reference

Can acces Bitbucket private repository with curl but not with Python

I want to use the Bitbucket API to get information of a private repository.
It works fine with curl:
curl -u username:apppassword https://api.bitbucket.org/2.0/repositories/company/repo
But not with Python (Unfortunately I have to use Python 3.4):
#!/usr/bin/env python3
from pybitbucket.auth import BasicAuthenticator
from pybitbucket.bitbucket import Client
from pybitbucket.repository import Repository
from pybitbucket.user import User
client = Client(BasicAuthenticator('username', 'apppassword ', 'usermail'))
print(User.find_current_user(client).display_name)
print(Repository.find_repository_by_full_name("company/repo"))
User name is printed correctly. But Repository.find_repository_by_full_name raises a 403 (forbidden).
Same thing, when I try to do it with urllib:
import urllib.request
base_url = 'https://api.bitbucket.org/2.0/'
password_mgr = urllib.request.HTTPPasswordMgrWithDefaultRealm()
password_mgr.add_password(None, base_url, 'username', 'apppassword ')
handler = urllib.request.HTTPBasicAuthHandler(password_mgr)
opener = urllib.request.build_opener(handler)
with opener.open(base_url + 'user') as f:
print(f.read())
with opener.open(base_url + 'repositories/company/repo') as f:
print(f.read())
Authentication must work, otherwise it could not return my user name correctly. Also, when I enter wrong credentials, I get a 401 (unauthorized) instead of a 403 (forbidden).
On the other hand, it works perfectly fine when I use curl.
Oh, it doesn't work with wget either:
wget --http-user=username --http-passwd=apppassword https://api.bitbucket.org/2.0/repositories/company/repository
What is curl doing different than wget and Python?

The Bitbucket API doesn't answer with 401 Unauthorized when I call https://api.bitbucket.org/2.0/repositories/company/repository. This can easily tested with a browser. It doesn't ask for credentials but shows Access denied. You must have write or admin access. So no authentication will take place.
But when I open https://api.bitbucket.org/2.0/user, Bitbucket responses with 401 Unauthorized and the header www-authenticate: Basic realm="Bitbucket.org HTTP", so the browser knows to show the authentication dialog. That's why getting the user data works on Python.
To solve this, I have to enforce to send the authentication even when the server doesn't ask for it. I haven't found a way to do this with a urllib.request.HTTPBasicAuthHandler but by adding the Authorization header manually:
import base64
import urllib.request
request = urllib.request.Request('https://api.bitbucket.org/2.0/repositories/company/repository')
base64string = base64.b64encode('{}:{}'.format('username', 'apppassword').encode('ascii'))
request.add_header('Authorization', 'Basic {}'.format(base64string.decode('ascii')))
with urllib.request.urlopen(request) as response:
print(response.read())

difference between curl and python requests

I am trying to access my firewall API.
and i got a good answer in curl with:
curl -k -i -u admin:xxxx -X POST https://10.0.0.2:9443/api/sonicos/auth
#=> Ok
with requests, api said HTTP 406
here is my code:
import requests
from requests.auth import HTTPBasicAuth
r = requests.post(
'https://10.0.0.2:9443/api/sonicos/auth',
auth=HTTPBasicAuth('admin', 'xxxx'),
headers={'Content-type': 'Application/JSON'},
verify=False
)
API documentation said: 406 Not acceptable
Mime-type in content-type not supported.
I try a lot of parameters without success.
Any idea to help me?
Thanx

As far as I know, and according to the RFC 2045, the content type should be lowercase. In your case, "'Content-type': 'application/json'"

Converting CURL Command to Python 2.7

I have a curl command that works perfectly fine and gives me a HTTP 200.
curl -i -H "Authorization: Basic jadkfhjkafDSKJ12DD=" http://<ip>/LoadTest/rest/authentication-point/authenticate
The above API needs the authorization in base64 format and the details have to be passed as Headers. This is a GET request.
When I try to run the same in Python 2.7, I get Response [403]. Code below.
import requests
headers = {'Authorization': 'Basic jadkfhjkafDSKJ12DD='}
authurl = "http://<ip>/LoadTest/rest/authentication-point/authenticate"
r = requests.get(authurl, headers=headers)
print r.status_code
What am I missing here? How should i pass the authorization values exactly like I passed in the curl command? I've tried multiple ways but still end up getting HTTP 403 always. Kindly guide.

Thanks all for your inputs. This is the final solution. I found that there is proxy that is stopping the payload. So added the session to the request.
import requests
session = requests.Session()
session.trust_env = False
headers = {'Authorization': 'Basic jadkfhjkafDSKJ12DD='}
authurl = "http://<ip>/LoadTest/rest/authentication-point/authenticate"
r = session.get(authurl, headers=headers)
print r.status_code
Setting the trust_env=False ignores the following:
Authentication information from .netrc (code)
CA bundles defined in
REQUESTS_CA_BUNDLE or CURL_CA_BUNDLE (code)

How should I send HTTP request with custom method in python

I need to send HTTP request with custom method to a custom server. I've been googling about executing curl command in python, and mostly I've found:
Don't do that!
I need to execute the following curl command:
curl -X MUX -i -H "Connection-Service: API" -H "Service-Address: API" http://172.16.117.40
I've been trying with requests library in python with no luck.

I constructed this solution from various stackoverflow answers:
import httplib, urllib2
httplib.HTTPConnection._http_vsn = 10
httplib.HTTPConnection._http_vsn_str = 'HTTP/1.0'
opener = urllib2.build_opener(urllib2.HTTPHandler)
request = urllib2.Request(self.url)
request.add_header('Connection-Service', 'API')
request.add_header('Service-Address', 'API')
request.get_method = lambda: 'MUX'
url = opener.open(request)
url.info().getheader('API')
It works as curl command posted in the question and I don't need to mess with the actual command.

Rest API authentication and access using Python Requests

I have been regularly accessing an API at work using curl. But now i need to do a bit of automation for the same.
Was trying to translate what i do with curl into python using the requests module.
But I keep receiving a 401 error.
My curl requests that i regularly are as below:
Step1: Session Authentication with cookies:
curl -b cookies -c cookies -v -X POST -H "Content-Type: application/json" --data-binary '{"auth":{"username":"aaa","password":"bbb"}}' http://api.xyz.at/auth
Step2: Access API URL for data retrieval
curl -b cookies -c cookies http://api.xyz.at/somepage?per_id=556677
Now using Python Requests, here is what I am doing
Step1: For Authentication
username = 'aaa'
password = 'bbb'
s = requests.Session()
s.post('http://api.xyz.at/auth',auth=('username','pasword'))
This "i think" works fine, and give me the below response
<Response [200]>
Step2: Access API URL for data retrieval
s.get('http://api.xyz.at/somepage?per_id=556677')
but this Step 2 keeps returning an error
<Response [401]>
The code is failing and not sure where.
My Python skills are visibly pedestrian. I have been looking at the Requests website. But unfortunately haven't been able to decipher.
Guidance would be appreciated :)

import urllib2, urllib
url = 'http://api.xyz.at/auth'
pm = urllib2.HTTPPasswordMgrWithDefaultRealm()
pm.add_password(None, url, 'user', 'password')
auth = urllib2.HTTPBasicAuthHandler(pm)
opener = urllib2.build_opener(auth)
urllib2.install_opener(opener)
request = urllib2.Request('http://api.xyz.at/somepage?per_id=556677', None)
handler = urllib2.urlopen(request)
handler.read()

Since you are getting a 401 error I guess you are not passing the authentication token which you get as response from login API. Use the same auth token to perform other options - Get-post-Delete.