I'm using Locust (which uses Requests) to do page load tests. The page has a popup requesting username and password to access the page and there is a standard login page.
I'm using the client.auth to authenticate the first time (on the popup) to open the page and send data on the POST request to login the account.
The problem is that Locust never displays a failure, even when I skip the second authentication. In other words, if I do the client.auth authentication I can GET any page (even the ones for which authentication is needed, and I skip the second authentication) and Locust does not display a 401 error. So I'm doing something wrong here.
Here is the code that I'm using:
class UserBehavior(TaskSet):
def on_start(self):
self.login()
def login(self):
# Set basic auth credentials
if BASIC_AUTH_CREDENTIALS:
self.client.auth = BASIC_AUTH_CREDENTIALS
headers = {
'referer': 'http://myreferer.com'
}
data = {
'username': 'John',
'password': 'Doe',
'csrfmiddlewaretoken': '123456712345671234567'
}
r = self.client.post('/login', data=data, headers=headers)
#task
def progress(self):
self.client.get("/needed/authorization")
What I want is to measure the time it takes a page to load when a number of users try to load it. I have a couple of pages that I measure this on, one that loads fast the other much slower, but the test displays the same values for both. I think that Locust's simulated users get redirected to the login page every time and therefor they have the same response time and don't report a 401 or 404, or any error for that matter.
Please help me do this properly.
For testing the login required pages you should send loged-in user's info (cookie) in the header.
class UserBehavior(TaskSet):
cookie = ""
def on_start(self):
self.cookie = self.get_user_cookie()
def get_user_cookie(self):
# Set basic auth credentials
if BASIC_AUTH_CREDENTIALS:
self.client.auth = BASIC_AUTH_CREDENTIALS
headers = {
'referer': 'http://myreferer.com'
}
data = {
'username': 'John',
'password': 'Doe',
'csrfmiddlewaretoken': '123456712345671234567'
}
r = self.client.post('/login', data=data, headers=headers)
if r.status_code == 200:
return r.request.headers['Cookie']
else:
print "User cannot login"
#task
def progress(self):
self.client.get("/needed/authorization", headers = {"Cookie": self.cookie})
Related
I'm building a discord bot with discord.py and I'm trying to get the spotify user saved tracks.
This is my auth def:
#classmethod
def get_token(self):
CLIENT_ID = 'myclientid'
CLIENT_SECRET = "myclientsecret"
SPOTIFY_TOKEN_URL = "https://accounts.spotify.com/api/token"
client_token = base64.b64encode("{}:{}".format(CLIENT_ID, CLIENT_SECRET).encode('UTF-8')).decode('ascii')
headers = {"Authorization": "Basic {}".format(client_token)}
payload = {"grant_type": "client_credentials"}
token_request = requests.post(SPOTIFY_TOKEN_URL, data=payload, headers=headers)
access_token = json.loads(token_request.text)["access_token"]
return access_token
This is my def where I try to get the user saved tracks:
#commands.command(name='splayfav', aliases=['splayfavorites', 'splaysavedtracks'], description="Command to see the info from spotify \n __Example:__ \u200b \u200b *!infos*")
async def play_saved_tracks_from_spotify(self, ctx):
token = self.get_token(ctx)
headers = {
'Accept': 'application/json',
'Content-Type': 'application/json',
'Authorization': f'Bearer {token}',
}
response = requests.get('https://api.spotify.com/v1/me/tracks', headers=headers)
print(response.text)
I get this error:
{
"error" : {
"status" : 401,
"message" : "Missing token"
}
}
But If I go in the Spotify API console and get manually a token and put it manually after bearer, then it works, obviously I'm trying to automate the process so I can't take it by hand every time. If I print the token it's actually a token(I mean it's not None or similar), but it's like if It is the wrong one.
How can I fix this problem?
This error message is stupid. What it should tell you is that your authorization token does not have the grant to access the user.
There's 2 different ways to authorize:
client credentials (as you do, can't access any user related stuff)
Authorization Code Flow, see: https://developer.spotify.com/documentation/general/guides/authorization/code-flow/
The second way requires a bit more setup in your app. Basically get redirected to spotify webpage, then the user selects allow for the required permissions, then you get back a code and with that you can get a token and the refresh token (this one is the interesting part).
Now this is quite a hustle to get through, but once you have the refresh token you can basically do almost the same call you do just to get a refreshed access token. (See "Request a refreshed Access Token" at the bottom of the page I linked above)
So with refresh token you can do:
POST https://accounts.spotify.com/api/token
HEADER:
Authorization: Basic $base64(clientId:clientSecret)
BODY Parameters (form url encoded)
grant_type: refresh_token
refresh_token: <your value here>
Use this improved access token and it will work.
I want to test an authenticated post request on an API using Pytest. This is what I am doing so far:
def test_auth_user_can_create(self, client):
url = api_reverse('crud-simulation_api')
data = {
"project": "testproject",
....
}
response = client.post(url, json=data)
assert response.status_code == 200
This doesn't work because it gives me back a 401 (Unauthorized) instead of a 200. That makes sense since the fixture is a client and not an admin client.
Yet if I pass in admin_client instead of client it gives me a Bad Request. The data that I send should be fine though.
I also tried to pass in the headers like so (since I use JWT authorization):
token = "bigassstringwhichismytoken"
headers = {
"Authorization": "JWT " + token
}
Finally I tried to log in before which gives me a 403 (Forbidden):
def test_auth_user_can_create_simulation_api(self, client, django_user_model):
username = "Jack"
password = "password"
django_user_model.objects.create_user(username=username, password=password)
client.login(username=username, password=password)
url = api_reverse('crud-simulation_api')
data = {
"project": "testproject",
...
}
response = client.post(url, json=data)
assert response.status_code == 200
If someone could point me into the right direction that would be fantastic! Thanks a lot in advance
To provide headers for client.{request} pass them individually as keyword agruments:
client.post(url, data, HTTP_FIRST_HEADER='...', HTTP_SECOND_HEADER='...')
Although you're unlikely to collide with any reserved parameter names in post call chain, better collect all headers you need in a dictionary:
headers = {
'HTTP_FIRST_HEADER': '...',
'HTTP_SECOND_HEADER': '...',
}
And pass them to request as arbitrary number of keyword arguments:
client.post(url, data, **headers)
In this case ** arguments are treated as extra information and are automatically added as headers.
You can hit the login url with username and password and get the token.
creade a header dictionary like headers = {'Authorization': 'JWT <token>'}
and use the header when using post.
client.post(url, json=data, headers=headers)
I would suggest installing the pytest-django package. Based on its docs, the easiest answer would be just using the admin_client fixture. As admin_client has the type of django.test.Client, it can be used for both get and post requests.
def test_sth_with_auth(admin_client):
response = admin_client.get('/private')
assert response.status_code == 200
Also if you want to use a specific user, you can try sth like this:
#pytest.fixture
def my_user(django_user_model):
return django_user_model.objects.create_user(username=username, password=password)
#pytest.fixture
def logged_in_client(client, my_user):
return client.force_login(my_user)
def test_sth_with_auth(logged_in_client):
response = logged_in_client.get('/private')
assert response.status_code == 200
this part of the doc can be helpful to write your desired logged_in_client().
I am trying to login to a website using python.
The login URL is :
https://login.flash.co.za/apex/f?p=pwfone:login
and the 'form action' url is shown as :
https://login.flash.co.za/apex/wwv_flow.accept
When I use the ' inspect element' on chrome when logging in manually, these are the form posts that show up (pt_02 = password):
There a few hidden items that I'm not sure how to add into the python code below.
When I use this code, the login page is returned:
import requests
url = 'https://login.flash.co.za/apex/wwv_flow.accept'
values = {'p_flow_id': '1500',
'p_flow_step_id': '101',
'p_page_submission_id': '3169092211412',
'p_request': 'LOGIN',
'p_t01': 'solar',
'p_t02': 'password',
'p_checksum': ''
}
r = requests.post(url, data=values)
print r.content
How can I adjust this code to perform a login?
Chrome network:
This is more or less your script should look like. Use session to handle the cookies automatically. Fill in the username and password fields manually.
import requests
from bs4 import BeautifulSoup
logurl = "https://login.flash.co.za/apex/f?p=pwfone:login"
posturl = 'https://login.flash.co.za/apex/wwv_flow.accept'
with requests.Session() as s:
s.headers = {"User-Agent":"Mozilla/5.0"}
res = s.get(logurl)
soup = BeautifulSoup(res.text,"lxml")
values = {
'p_flow_id': soup.select_one("[name='p_flow_id']")['value'],
'p_flow_step_id': soup.select_one("[name='p_flow_step_id']")['value'],
'p_instance': soup.select_one("[name='p_instance']")['value'],
'p_page_submission_id': soup.select_one("[name='p_page_submission_id']")['value'],
'p_request': 'LOGIN',
'p_arg_names': soup.select_one("[name='p_arg_names']")['value'],
'p_t01': 'username',
'p_arg_names': soup.select_one("[name='p_arg_names']")['value'],
'p_t02': 'password',
'p_md5_checksum': soup.select_one("[name='p_md5_checksum']")['value'],
'p_page_checksum': soup.select_one("[name='p_page_checksum']")['value']
}
r = s.post(posturl, data=values)
print r.content
since I cannot recreate your case I can't tell you what exactly to change, but when I was doing such things I used Postman to intercept all requests my browser sends. So I'd install that, along with browser extension and then perform login. Then you can view the request in Postman, also view the response it received there, what's more it provides you with Python code of request too, so you could simply copy and use it then.
Shortly, use Pstman, perform login, clone their request.
I'm currently trying to write a Python script that will use Deviantart's API to automatically shuffle my favourites. To do that I need to first log in in my script. Deviantart uses OAuth2 authentication, which requires a redirect_uri, which as I understand it is supposed to be the server where my application is running.
However I'm running the script locally on my computer (not on a server) and just sending http requests via Python's Requests library. How do I then authenticate, when the OAuth procedure sends the code required for the authentication token as a parameter of a GET call to the redirect_uri, which points to nowhere in particular for me? Is there no way to authenticate without running a server?
EDIT
My problem is still that I'm running a simple offline script, and I'm not sure how to authenticate from it.
This is my authentication code so far:
import binascii, os, requests
def auth():
request = 'https://www.deviantart.com/oauth2/authorize'
state = binascii.hexlify(os.urandom(160))
params = {
'response_type': 'token',
'client_id': 1,
'redirect_uri': 'https://localhost:8080',
'state': state,
'scope': 'user'
}
r = requests.get(request, params)
print(r)
The printed response is simply a 200 HTTP code, rather than an access token (obviously, since the username and password haven't been entered anywhere). The request is sent to DA's authorisation page, but since the page itself doesn't actually open in my script, I can't enter my username and password to log in anywhere. And I can't directly send the username and password in the GET request to authenticate that way either (again obviously, since it would be a terrible idea to send the password like that).
Preferably I'd like a way to simply have the user (me) prompted for the username and password in the console that the script is running in and then have the script continue executing after the user has successfully logged in.
Alternatively, if the above is not possible, the script should open the authorisation webpage in a browser, and then continue execution once the user logs in.
How would I go about realising either of these two solutions in Python?
If your application is offline, you cannot use the Authorization Code nor the Implicit grant type: both flows require a redirection URI.
As your python script cannot be reached from the Internet and because Deviantart does not allow the use of another grant type (except Client Credentials, but not relevant in you case), then you won't be able to issue any access token.
Your application must be accessible from the Internet.
You are supposed to get authorization token using received code. This token will be used to access DeviantArt afterwards.
Refer to https://www.deviantart.com/developers/authentication (section "Using The Authorization Code Grant").
Per request, I'm updating this question with the code I ended up using for my script's authentication in the hope that it helps somebody.
import webbrowser
import requests
import urllib.parse
import binascii
import os
import time
from http.server import HTTPServer, BaseHTTPRequestHandler
AUTH = 'https://www.deviantart.com/oauth2/authorize'
TOKEN = 'https://www.deviantart.com/oauth2/token'
code = ''
state = binascii.hexlify(os.urandom(20)).decode('utf-8')
class Communicator:
def __init__(self):
self.client_id = '<insert-actual-id>' # You get these two from the DA developer API page
self.client_secret = '<insert-actual-secret>' # but it's safer if you store them in a separate file
self.server, self.port = 'localhost', 8080
self._redirect_uri = f'http://{self.server}:{self.port}'
self._last_request_time = 0
def auth(self, *args):
scope = ' '.join(args)
params = {
'response_type': 'code',
'client_id': self.client_id,
'redirect_uri': self._redirect_uri,
'scope': scope,
'state': state
}
request = requests.Request('GET', AUTH, params).prepare()
request.prepare_url(AUTH, params)
webbrowser.open(request.url)
server = HTTPServer((self.server, self.port), RequestHandler)
server.handle_request()
params = {
'client_id': self.client_id,
'client_secret': self.client_secret,
'grant_type': 'authorization_code',
'code': code,
'redirect_uri': self._redirect_uri
}
self._get_token(params)
def _get_token(self, params):
r = requests.get(TOKEN, params).json()
self.token = r['access_token']
self.refresh_token = r['refresh_token']
def _refresh_token(self):
params = {
'client_id': self.client_id,
'client_secret': self.client_secret,
'grant_type': 'refresh_token',
'refresh_token': self.refresh_token
}
self._get_token(params)
def _request(self, func, url, params, sleep=5, cooldown=600):
t = time.time()
if t - self._last_request_time < sleep:
time.sleep(sleep - t + self._last_request_time)
self._last_request_time = t
max_sleep = 16 * sleep
params['access_token'] = self.token
while True:
try:
r = func(url, params).json()
if 'error_code' in r and r['error_code'] == 429:
sleep *= 2
time.sleep(sleep)
if sleep > max_sleep:
raise ConnectionError("Request timed out - server is busy.")
elif 'error' in r and r['error'] == 'user_api_threshold':
raise ConnectionError("Too many requests")
elif 'error' in r and r['error'] == 'invalid_token':
print("Refreshing token.")
self._refresh_token()
params['access_token'] = self.token
else:
return r
except ConnectionError:
print(f"Request limit reached - waiting {cooldown // 60} minutes before retrying...")
time.sleep(cooldown)
def get(self, url, params):
return self._request(requests.get, url, params)
def post(self, url, params):
return self._request(requests.post, url, params)
class RequestHandler(BaseHTTPRequestHandler):
def do_GET(self):
global code
self.close_connection = True
query = urllib.parse.parse_qs(urllib.parse.urlparse(self.path).query)
if not query['state'] or query['state'][0] != state:
raise RuntimeError("state argument missing or invalid")
code = query['code']
BROWSE = 'browse'
BROWSE_MORE_LIKE_THIS = 'browse.mlt'
COLLECTION = 'collection'
COMMENT = 'comment.post'
FEED = 'feed'
GALLERY = 'gallery'
MESSAGE = 'message'
NOTE = 'note'
PUBLISH = 'publish'
STASH = 'stash'
USER = 'user'
USER_MANAGE = 'user.manage'
if __name__ == '__main__':
com = Communicator()
com.auth(BROWSE, COLLECTION) # request specific permissions
... # do stuff with com.get() and com.post() requests
After much reading here on Stackoverflow as well as the web I'm still struggling with getting things to work.
My challenge: to get access to a restricted part of a website for which I'm a member using Python and urllib2.
From what I've read the code should be like this:
mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
url = 'http://www.domain.com'
mgr.add_password(None, url, 'username', 'password')
handler = urllib2.HTTPBasicAuthHandler(mgr)
opener = urllib2.build_opener(handler)
urllib2.install_opener(opener)
try:
response = urllib2.urlopen('http://www.domain.com/restrictedpage')
page = response.read()
print page.geturl()
except IOError, e:
print e
The print doesn't print "http://www.domain.com/restrictedpage", but shows "http://www.domain.com/login" so my credentials aren't stored/processed and I'm being redirected.
How can I get this to work? I've been trying for days and keep hitting the same dead ends. I've tried all the examples I could find to no avail.
My main question is: what's needed to authenticate to a website using Python and urllib2?
Quick question: what am I doing wrong?
Check first manually what is really happening when you are successfully authenticated (instructions with Chrome):
Open develper tools in Chrome (Ctrl + Shift + I)
Click Network tab
Go and do the authentication manually (go the the page, type user + passwd + submit)
check the POST method in the Network tab of the developer tools
check the Request Headers, Query String Parameters and Form Data. There you find all the information needed what you need to have in your own POST.
Then install "Advanced Rest Client (ARC)" Chrome extension
Use the ARC to construct a valid POST for authentication.
Now you know what to have in your headers and form data. Here's a sample code using Requests that worked for me for one particular site:
import requests
USERNAME = 'user' # put correct usename here
PASSWORD = 'password' # put correct password here
LOGINURL = 'https://login.example.com/'
DATAURL = 'https://data.example.com/secure_data.html'
session = requests.session()
req_headers = {
'Content-Type': 'application/x-www-form-urlencoded'
}
formdata = {
'UserName': USERNAME,
'Password': PASSWORD,
'LoginButton' : 'Login'
}
# Authenticate
r = session.post(LOGINURL, data=formdata, headers=req_headers, allow_redirects=False)
print r.headers
print r.status_code
print r.text
# Read data
r2 = session.get(DATAURL)
print "___________DATA____________"
print r2.headers
print r2.status_code
print r2.text
For HTTP Basic Auth you can refer this : http://www.voidspace.org.uk/python/articles/authentication.shtml