I am trying to create a very basic app that will be able to connect to a web server which host my college assignments, results, and more and notify me when ever there's something new on it. Currently I am trying to get the hang of the requests module, but I am not able to login as the server uses this kind of authentication, and it gives me error 401 unauthorized.
I tried searching how to authenticate to web servers tried using sockets, with no luck. Could you please help me find out how to do this?
EDIT: I am using python 3.4
After inspecting the headers in the response for that URL, I think the server is trying to use NTLM authentication.
Try installing requests-ntlm (e.g. with pip install requests_ntlm) and then doing this:
import requests
from requests_ntlm import HttpNtlmAuth
requests.get('http://moodle.mcast.edu.mt:8085/',
auth=HttpNtlmAuth('domain\\username', 'password'))
You need to attach a simple authentication header within the socket request headers.
Example;
import base64
mySocket.send('GET / HTTP/1.1\r\nAuthorization: Basic %s\r\n\r\n' % base64.b64encode('user:pass'))
Python 3x;
import base64
template = 'GET / HTTP/1.1\r\nAuthorization: Basic %s\r\n\r\n'
mySocket.send(bytes(template % base64.b64encode(bytes('user:pass', 'UTF-8')), 'UTF-8'))
They might not supply a programmatic API with which to authorize requests. If that is the case, you could try using selenium to manually open a browser and fill the details in for you. Selenium has handling for alert boxes too apparently, though I haven't used it myself.
Related
How can I use automatic NTLM authentication from python on Windows?
I want to be able to access the TFS REST API from windows without hardcoding my password, the same as I do from the web browser (firefox's network.automatic-ntlm-auth.trusted-uris, for example).
I found this answer which works great for me because:
I'm only going to run it from Windows, so portability isn't a problem
The response is a simple json document, so no need to store an open session
It's using the WinHTTP.WinHTTPRequest.5.1 COM object to handle authentication natively:
import win32com.client
URL = 'http://bigcorp/tfs/page.aspx'
COM_OBJ = win32com.client.Dispatch('WinHTTP.WinHTTPRequest.5.1')
COM_OBJ.SetAutoLogonPolicy(0)
COM_OBJ.Open('GET', URL, False)
COM_OBJ.Send()
print(COM_OBJ.ResponseText)
You can do that with https://github.com/requests/requests-kerberos. Under the hood it's using https://github.com/mongodb-labs/winkerberos. The latter is marked as Beta, I'm not sure how stable it is. But I have requests-kerberos in use for a while without any issue.
Maybe a more stable solution would be https://github.com/brandond/requests-negotiate-sspi, which is using pywin32's SSPI implementation.
I found solution here https://github.com/mullender/python-ntlm/issues/21
pip install requests
pip install requests_negotiate_sspi
import requests
from requests_negotiate_sspi import HttpNegotiateAuth
GetUrl = "http://servername/api/controller/Methodname" # Here you need to set your get Web api url
response = requests.get(GetUrl, auth=HttpNegotiateAuth())
print("Get Request Outpot:")
print("--------------------")
print(response.content)
for request by https:
import requests
from requests_negotiate_sspi import HttpNegotiateAuth
import urllib3
urllib3.disable_warnings()
GetUrl = "https://servername/api/controller/Methodname" # Here you need to set your get Web api url
response = requests.get(GetUrl, auth=HttpNegotiateAuth(), verify=False)
print("Get Request Outpot:")
print("--------------------")
print(response.content)
NTLM credentials are based on data obtained during the interactive logon process, and include a one-way hash of the password. You have to provide the credential.
Python has requests_ntlm library that allows for HTTP NTLM authentication.
You can reference this article to access the TFS REST API :
Python Script to Access Team Foundation Server (TFS) Rest API
If you are using TFS 2017 or VSTS, you can try to use Personal Access Token in a Basic Auth HTTP Header along with your REST request.
I am writing a script to automatically scrape information from my companies directory website using mechanize. However, the interpreter returns _response.httperror_seek_wrapper: HTTP Error 401: Authorization Required onbr.open(url) when I run my script.
This is the portion of my code where the interpreter runs into the error.
from sys import path
path.append("./mechanize/mechanize")
import _mechanize
from base64 import b64encode
def login (url, username, password):
b64login = b64encode('%s:%s' % (username, password))
br = _mechanize.Browser()
br.set_handle_robots(False)
br.addheaders.append(('Authorization','Basic %s' % b64login))
br.open(url)
r = br.response()
print r.read()
The site I am trying to access is an internal site within my companies network, and it uses a GlobalSign Certificate for authentication on company-issued computers.
I am sure the authentication information I am inputting is correct, and I have looked everywhere for a solution. Any hints on how to resolve this? Thanks!
It looks like your authentication methods don't match up. You state that your company uses GlobalSign certificates but your code is using Basic authentication. They are NOT equal!!
From a brief look at the Mechanize documentation (limited as it is), you don't implement authentication by manually adding headers. It has it's own add_password method for handling authentication.
Also, as a general HTTP authentication policy, you should NOT use preemptive authentication by adding the authentication headers yourself. You should set up your code with the necessary authentication (based on your library's documentation) and let it handle the authentication negotiation.
I am struggling with Jira REST API authentication via kerberos. Basic authentication works as expected.
If I access the login page with the web browser (after I did kinit) and then use the generated JSESSIONID in my python script, I can use REST without getting 401. But I have no ide how to do that with my python script, I tried to use requests_kerberos, but when I request the login page, it simply returns the basic login form instead of automatic login.
Do you know how to use JIRA REST API with kerberos authentication?
Thanks for you answers.
After a day of struggle I finally figured it out.
First you have to send an HTTP GET request to ${jira-url}/step-auth-gss:
r = requests.get("https://example-jira.com/step-auth-gss", auth=requests_kerberos.HTTPKerberosAuth())
Then you get the JSESSIONID from the cookie header and you can REST away:
rd = requests.get(url, headers={"Cookie": "JSESSIONID=%s" % r.cookies['JSESSIONID']})
As explained by VaclavDedik, the first step is to get a valid JSESSIONID cookie (along with atlassian.xsrf.token and crowd.token_key cookies if you use Crowd for user management and SSO) upon successful Kerberos authentication on a private Jira resource / URL.
In Python, the PycURL package makes it very easy to authenticate with Kerberos. You can install it on Windows/Mac OS/Linux either with easy_install or pip. The PycURL package relies on libcurl. You will need to check that your libcurl version is >=7.38.0 as the HTTPAUTH_NEGOTIATE directive was introduced in that very version.
Then, it is as simple as:
import pycurl
curl = pycurl.Curl()
# GET JSESSIONID
curl.setopt(pycurl.COOKIEFILE, "")
curl.setopt(pycurl.HTTPAUTH, pycurl.HTTPAUTH_NEGOTIATE)
curl.setopt(pycurl.USERPWD, ':')
curl.setopt(pycurl.URL, <ANY_JIRA_PRIVATE_URL>)
curl.perform()
# Then REST request
curl.setopt(pycurl.URL, <YOUR_JIRA_REST_URL>)
curl.perform()
curl.close()
Please, check out the following page for detailed examples in Python, PowerShell and Groovy: https://www.cleito.com/products/iwaac/documentation/integrated-windows-authentication-for-non-browser-clients/
Though this is the official documentation of the Cleito IWAAC plugin mentioned by Xabs, this will work with any server-side Kerberos plugin for Jira
This is a newbie problem with python, advice is much appreciated.
no-ip.com provides an easy way to update a computer's changing ip-address, simply open the url
http://user:password#dynupdate.no-ip.com/nic/update?hostname=my.host.name
...both http and https work when entered in firefox. I tried to implement that in a script residing in "/etc/NetworkManager/dispatcher.d" to be used by Network Manager on a recent version of Ubuntu.
What works is the python script:
from urllib import urlopen;
urlopen("http://user:password#dynupdate.no-ip.com/nic/update?hostname=my.host.name")
What I want to have is the same with "https", which does not work as easily. Could anyone, please,
(1) show me what the script should look like for https,
(2) give me some keywords, which I can use to learn about this.
(3) perhaps even explain why it does not work any more when the script is changed to using "urllib2":
from urllib2 import urlopen;
urlopen("http://user:password#dynupdate.no-ip.com/nic/update?hostname=my.host.name")
Thank you!
The user:password part isn't in the actual URL, but a shortcut for HTTP authentication. The browser's URL parsing lib will filter them out. In urllib2, you want to
import base64, urllib2
user,password = 'john_smith','123456'
request = urllib2.Request('dynupdate.no-ip.com/nic/update?hostname=my.host.name')
auth = base64.base64encode(user + ':' + password)
request.add_header('Authorization', 'Basic ' + auth)
urllib2.urlopen(request)
The Microsoft Dynamics CRM service uses NTLM authentication, which makes connecting to it from a python process using suds somewhat complicated. I'm looking for a code sample that will:
Send and receive the response from a RetrieveAttributeRequest
Send and receive the response from an Execute request.
This must use Python 2.6 or Python 2.7, not Python 3. I already have a working implementation that uses curl to do this, but it's flaky at the best of times, and as part of some other work I have in this tool I'd like to clean it up and make it run using python/suds.
I know this is a bit late but hopefully it will help someone.
NTLM authentication was added to suds in version 0.3.8.
from suds.transport.https import WindowsHttpAuthenticated
from suds.client import Client
url = 'http://crmurl/XRMServices/2011/Discovery.svc?wsdl'
ntlm = WindowsHttpAuthenticated(username='DOMAIN\username', password='password')
client = Client(url, transport=ntlm)
I don't know if this will be of help to you, but I used PycURL to get through an NTLM proxy.
Here's a code snippet:
c = Curl()
c.setopt(URL, 'http://www.somesite.com')
c.setopt(FOLLOWLOCATION, 1) # follow redirects
c.setopt(MAXREDIRS, 5) # max redirects
c.setopt(PROXY, 'proxy.somesite.com')
c.setopt(PROXYUSERPWD, 'DOMAIN/USER:PASSWORD')
c.setopt(PROXYAUTH, HTTPAUTH_NTLM) # use NTLM
c.perform()
Here's the documentation on the Curl object.