Develop Reference

What to put in query string in post request? -NiceHash API

I am working on a python program that turns on/off my mining rig with Nicehash API. I did everything that I need to but I am stuck with what to put in query string.
In header of post request must be X-auth which which is built from API Key and HMAC signature. On NiceHash website it says that HMAC signature must be build like this:
Input structure is the following:
-API Key
-X-Time request header value
-X-Nonce request header value
-Empty field
-X-Organization-Id request header value
-Empty field
-Request method (example: GET, POST, DELETE, PUT)
-Request path (example: /main/api/v2/hashpower/orderBook)
-Request query string (example: algorithm=X16R&page=0&size=100, The query string should be the same as
passed to the server - without the leading question mark)
Input is a byte array composed of ordered fields using zero byte (0x00) as a separator. There is no
separator before the first field or after the last field. Some fields are always empty in which case the
separators immediately follow one another. For request body you should use the raw bytes as they are sent
to the server. For JSON messages the character encoding should always be UTF-8.
So input should look like this (already hashed in UTF-8) and then be hashed again (in order from first to last:
API KEY,TIME,NONCE,ORGANISATION ID,REQUEST METHOD,REQUEST PATH,REQUEST QUERY STRING) :
4ebd366d-76f4-4400-a3b6-e51515d054d6 ⊠ 1543597115712 ⊠ 9675d0f8-1325-484b-9594-c9d6d3268890 ⊠ ⊠ da41b3bc-3d0b-4226-b7ea-aee73f94a518 ⊠ ⊠ GET ⊠ /main/api/v2/hashpower/orderBook ⊠ algorithm=X16R&page=0&size=100
(this sign: ⊠ is zero byte: 0x00)
My code:
import hashlib
import uuid
import requests
import json
import hmac
url = "https://api2.nicehash.com/main/api/v2/mining/rigs/status2"
path = "/main/api/v2/mining/rigs/status2"
ms = str(json.loads(requests.get('https://api2.nicehash.com/api/v2/time').text)['serverTime'])
req_id = str(uuid.uuid4())
nonce = str(uuid.uuid4())
org_id = organizationId
sec_key = secretKey
apiKey = apiKey
method = "POST"
query = ?
input = bytearray(f"{apiKey}\00{ms}\00{nonce}\00\00{org_id}\00\00{method}\00{path}\00{query}", "utf-8")
secret=hmac.new(bytearray(sec_key, "utf-8"), input, hashlib.sha256).hexdigest()
auth = apiKey + ":" + secret
#HEADER
header = {
"X-Time":ms,
"X-Nonce":nonce,
"X-Organization-Id":org_id,
"X-Auth":auth,
"X-Request-Id":req_id}
#BODY
body = {
"rigId": "SOMETHING",
"action": "STOP"
}
r=requests.post(url=url, headers=header, params=body)
print(r)
print(r.json())
But on website the example demostrates if you want to get a hashpower order book and I understand query string there. But I dont know what to put in query string in my case.
For now this is the main problem for me so the rest of the code is not finished yet.

try this:
query="action=STOP&rigId=yourigid" #stops certain rig
I tried to use https://www.nicehash.com/docs/rest and https://www.nicehash.com/docs/
picture of what worked for me

Failed Validation while setting up LinkedIn Webhook

#app.route('/webhooks/linkedin', methods=['GET'])
def webhook_challenge_linkedIn():
# creates HMAC SHA-256 hash from incoming token and your consumer secret
sha256_hash_digest = hmac.new(bytes({api_secret},'utf-8'), msg=bytes(request.args.get('challengeCode'),'utf-8'), digestmod=hashlib.sha256)
# construct response data with base64 encoded hash
print((sha256_hash_digest))
val = {
"challengeCode" : request.args.get('challengeCode'),
"challengeResponse" : sha256_hash_digest.hexdigest()
}
return json.dumps(val)
When I try to set up the server and send an authentication request from LinkedIn for webhook to this endpoint it says Failed Validation
all i need to do for the verification is return the challenge code in
challengeResponse = Hex-encoded(HMACSHA256(challengeCode, clientSecret))
I think this is what I did but the still validation gets failed.
I don't seem to see the issue.

Try the following:
#app.route('/webhooks/linkedin', methods=['GET'])
def webhook_challenge_linkedIn():
challenge_code = request.args.get('challengeCode')
# creates HMAC SHA-256 hash from incoming token and your consumer secret
sha256_hash_digest = hmac.new(api_secret.encode(), challenge_code.encode(), hashlib.sha256).hexdigest()
# construct response data with base64 encoded hash
# print(sha256_hash_digest)
val = {
"challengeCode" : challenge_code,
"challengeResponse" : sha256_hash_digest
}
return json.dumps(val)
See this answer from How to use SHA256-HMAC in python code?

How do I send credentials so I can unit test MSAL for Python?

The flow to this admin app is basically:
User goes to /admin.
#azure/msal-react checks if the user is logged in and if not redirects them to login.
When the accessToken, idToken, and oid have been received on the FE, they are sent to my API (/api) where they are validated.
If the tokens are validated, and verified as belonging to the oid that came from the FE, then a JWT is issued for the user indicating they are authenticated with the API as well.
The JWT is sent back to the FE where having both the MSAL isAuthenticated() and their API JWT isApiAuthenticated() indicates they are fully authenticated and it renders the FE components.
I'm trying to write unit tests for the what I have in the aad.py (bottom).
The issue I'm seeing is that I need to retrieve and send it a legit MSAL access_token, id_token, and oid. That means I actually need to log in a user against our AAD.
I do have a dummy user to use for this purpose, but I'm not seeing in the Python MSAL documentation how to sign the user in without interaction and just submit username and password.
Suggestions for how to write unit tests to test? Do I need to mockup something else, or should I be able to send a dummy user's username and password with Python MSAL?
# Python libraries
import json
# Third-party dependencies
import jwt
import requests
from base64 import b64decode, b64encode
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization
from config import settings
def ms_graph(access_token):
"""
Takes the access_token from the FE and uses it query MS Graph for user
information.
Returns an object.
Takes one argument:
access_token: sent from FE.
"""
graph_response = requests.get( # Use token to call downstream service
settings.GRAPH_URI,
headers={'Authorization': 'Bearer ' + access_token},)
return json.loads(graph_response.text)
def validate_token(token):
"""
Used to decode the token sent from the FE. The result is
a verified signature that contains information about the user and tokens.
Returns an object.
Takes one argument:
token: The idToken sent from the FE that needs to be validated.
"""
# Get a list of the possible public keys from the JWKS_URI endpoint
jwkeys = requests.get(settings.JWKS_URI).json()['keys']
# Extract the 'kid' from the unverified header to get the public key
token_key_id = jwt.get_unverified_header(token)['kid']
# Get the object corresponding to the 'kid' key
jwk = [key for key in jwkeys if key['kid'] == token_key_id][0]
# Encode the 'x5c' to eventually get the public key for decoding
der_cert = b64decode(jwk['x5c'][0])
cert = x509.load_der_x509_certificate(der_cert, default_backend())
public_key = cert.public_key()
pem_key = public_key.public_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PublicFormat.SubjectPublicKeyInfo
)
# Finally, validate and decode the token
token_claims = jwt.decode(
token,
pem_key,
algorithms=['RS256'],
audience=settings.CLIENT_ID
)
return token_claims
def validate_user(access_token, id_token, client_oid):
"""
Takes the id_token sent from the FE and validates the signature.
The validated signature also contains an oid claim which is used to verify
the oid sent from the FE.
Returns an object containing:
{
"user_validated": bool,
"graph_response": object
}
Takes two arguments:
access_token: access_token sent from the FE.
id_token: the id_token sent from the FE.
client_oid: the oid sent from the FE.
"""
# Decode the token to retrieve claims
id_token_payload = validate_token(id_token)
# Check if the claim oid and matches the oid sent from the FE
if id_token_payload['oid'] == client_oid:
graph_response = ms_graph(access_token)
res = dict()
res['user_validated'] = graph_response['id'] == client_oid
res['graph_response'] = graph_response
return res

If you really really want to obtain tokens for unit tests with username and password then look at acquire_token_by_username_password() and its example in MS docs.
But there are reasons to not let unit tests use the real MS Graph API:
dependency on network connection can make unit tests run slow or fail, even if your code still works fine,
storing credentials in repository is dangerous, keeping them properly is difficult,
the dummy account can be locked,
numerous test calls can cause throttling,
it is kind of unpolite to abuse the infrastructure with non-productive requests.
The suggestion for unit tests is usual: isolate test cases, then mock and patch any external dependencies not related to code being tested.
MSAL uses requests for calling the cloud, and there is a great package for mocking its work: meet responses. Prevent MSAL for sending out any requests and mock OpenID configuration call:
import responses
import time
import unittest
MS_OPENID_CONFIG = {
"authorization_endpoint": "https://login.microsoftonline.com/common/"
"oauth2/v2.0/authorize",
"token_endpoint": "https://login.microsoftonline.com/common/oauth2/"
"v2.0/token",
}
MS_TOKENS = {
'id_token_claims': {
'exp': time.time() + 60,
'oid': '00000000-0000-0000-0000-000000000000',
'upn': 'test_user_1_'
},
'id_token': {},
'access_token': {}
}
class BaseTest(unittest.TestCase):
def afterSetUp(self):
self.responses = responses.RequestsMock()
self.responses.start()
self.responses.add(
responses.GET,
'https://login.microsoftonline.com/common/v2.0/'
'.well-known/openid-configuration',
json=MS_OPENID_CONFIG
)
# add_calback() allows modifying mock token in runtime
self.responses.add_callback(
responses.POST,
'https://login.microsoftonline.com/common/oauth2/v2.0/token',
callback=lambda request: (200, {}, json.dumps(MS_TOKENS)),
content_type='application/json',
)
self.addCleanup(self.responses.stop)
self.addCleanup(self.responses.reset)
For testing validate_token() you can bundle pre-generated test RSA key pair:
openssl genrsa -out test_rsa_private.pem 512
openssl rsa -in test_rsa_private.pem -pubout -outform PEM -out test_rsa_public.pem
Then you can mock response to settings.JWKS_URI in the same way as OpenID config, returning test public key with known kid.
If you'd prefer to save time on signing test tokens, then just mock jwt signature verification to always return True:
from unittest import Mock, patch
def test_validate_token(self)
...
with patch('jwt.algorithms.RSAAlgorithm.verify', new=Mock(return_value=True)):
# jwt.decode(at, key=open('test_rsa_public.pem').read(), algorithms=['RS256'], options={'verify_exp': False, 'verify_aud': False})
validate_token(test_token)
And so on, unit test should only test the code of unit itself, everything around can be mocked as needed. Read the source of MSAL and jwt. Don't mock what looks like implementation details, they might change in future versions.

implementing USER_SRP_AUTH with python boto3 for AWS Cognito

Amazon provides iOS, Android, and Javascript Cognito SDKs that offer a high-level authenticate-user operation.
For example, see Use Case 4 here:
https://github.com/aws/amazon-cognito-identity-js
However, if you are using python/boto3, all you get are a pair of primitives: cognito.initiate_auth and cognito.respond_to_auth_challenge.
I am trying to use these primitives along with the pysrp lib authenticate with the USER_SRP_AUTH flow, but what I have is not working.
It always fails with "An error occurred (NotAuthorizedException) when calling the RespondToAuthChallenge operation: Incorrect username or password." (The username/password pair work find with the JS SDK.)
My suspicion is I'm constructing the challenge response wrong (step 3), and/or passing Congito hex strings when it wants base64 or vice versa.
Has anyone gotten this working? Anyone see what I'm doing wrong?
I am trying to copy the behavior of the authenticateUser call found in the Javascript SDK:
https://github.com/aws/amazon-cognito-identity-js/blob/master/src/CognitoUser.js#L138
but I'm doing something wrong and can't figure out what.
#!/usr/bin/env python
import base64
import binascii
import boto3
import datetime as dt
import hashlib
import hmac
# http://pythonhosted.org/srp/
# https://github.com/cocagne/pysrp
import srp
bytes_to_hex = lambda x: "".join("{:02x}".format(ord(c)) for c in x)
cognito = boto3.client('cognito-idp', region_name="us-east-1")
username = "foobar#foobar.com"
password = "123456"
user_pool_id = u"us-east-1_XXXXXXXXX"
client_id = u"XXXXXXXXXXXXXXXXXXXXXXXXXX"
# Step 1:
# Use SRP lib to construct a SRP_A value.
srp_user = srp.User(username, password)
_, srp_a_bytes = srp_user.start_authentication()
srp_a_hex = bytes_to_hex(srp_a_bytes)
# Step 2:
# Submit USERNAME & SRP_A to Cognito, get challenge.
response = cognito.initiate_auth(
AuthFlow='USER_SRP_AUTH',
AuthParameters={ 'USERNAME': username, 'SRP_A': srp_a_hex },
ClientId=client_id,
ClientMetadata={ 'UserPoolId': user_pool_id })
# Step 3:
# Use challenge parameters from Cognito to construct
# challenge response.
salt_hex = response['ChallengeParameters']['SALT']
srp_b_hex = response['ChallengeParameters']['SRP_B']
secret_block_b64 = response['ChallengeParameters']['SECRET_BLOCK']
secret_block_bytes = base64.standard_b64decode(secret_block_b64)
secret_block_hex = bytes_to_hex(secret_block_bytes)
salt_bytes = binascii.unhexlify(salt_hex)
srp_b_bytes = binascii.unhexlify(srp_b_hex)
process_challenge_bytes = srp_user.process_challenge(salt_bytes,
srp_b_bytes)
timestamp = unicode(dt.datetime.utcnow().strftime("%a %b %d %H:%m:%S +0000 %Y"))
hmac_obj = hmac.new(process_challenge_bytes, digestmod=hashlib.sha256)
hmac_obj.update(user_pool_id.split('_')[1].encode('utf-8'))
hmac_obj.update(username.encode('utf-8'))
hmac_obj.update(secret_block_bytes)
hmac_obj.update(timestamp.encode('utf-8'))
challenge_responses = {
"TIMESTAMP": timestamp.encode('utf-8'),
"USERNAME": username.encode('utf-8'),
"PASSWORD_CLAIM_SECRET_BLOCK": secret_block_hex,
"PASSWORD_CLAIM_SIGNATURE": hmac_obj.hexdigest()
}
# Step 4:
# Submit challenge response to Cognito.
response = cognito.respond_to_auth_challenge(
ClientId=client_id,
ChallengeName='PASSWORD_VERIFIER',
ChallengeResponses=challenge_responses)

There are many errors in your implementation. For example:
pysrp uses SHA1 algorithm by default. It should be set to SHA256.
_ng_const length should be 3072 bits and it should be copied from amazon-cognito-identity-js
There is no hkdf function in pysrp.
The response should contain secret_block_b64, not secret_block_hex.
Wrong timestamp format. %H:%m:%S means "hour:month:second" and +0000 should be replaced by UTC.
Has anyone gotten this working?
Yes. It's implemented in the warrant.aws_srp module.
https://github.com/capless/warrant/blob/master/warrant/aws_srp.py
from warrant.aws_srp import AWSSRP
USERNAME='xxx'
PASSWORD='yyy'
POOL_ID='us-east-1_zzzzz'
CLIENT_ID = '12xxxxxxxxxxxxxxxxxxxxxxx'
aws = AWSSRP(username=USERNAME, password=PASSWORD, pool_id=POOL_ID,
client_id=CLIENT_ID)
tokens = aws.authenticate_user()
id_token = tokens['AuthenticationResult']['IdToken']
refresh_token = tokens['AuthenticationResult']['RefreshToken']
access_token = tokens['AuthenticationResult']['AccessToken']
token_type = tokens['AuthenticationResult']['TokenType']
authenticate_user method supports only PASSWORD_VERIFIER challenge. If you want to respond to other challenges, just look into the authenticate_user and boto3 documentation.

Unfortunately it's a hard problem since you don't get any hints from the service with regards to the computations (it mainly says not authorized as you mentioned).
We are working on improving the developer experience when users are trying to implement SRP on their own in languages where we don't have an SDK. Also, we are trying to add more SDKs.
As daunting as it sounds, what I would suggest is to take the Javascript or the Android SDK, fix the inputs (SRP_A, SRP_B, TIMESTAMP) and add console.log statements at various points in the implementation to make sure your computations are similar. Then you would run these computations in your implementation and make sure you are getting the same. As you have suggested, the password claim signature needs to be passed as a base64 encoded string to the service so that might be one of the issues.
Some of the issues I encountered while implementing this was related to BigInteger library differences (the way they do byte padding and transform negative numbers to byte arrays and inversely).

Steam WebAPI AuthenticateUser

How to auth user via https://api.steampowered.com/ISteamUserAuth/AuthenticateUser/v0001 api method?
For example, I will get steam public key data from https://steamcommunity.com/login/getrsakey/, do some encryption and then send this data to specified api url as POST.
But server returns '403 Forbidden' everytime.
My code example:
from Crypto.Cipher import AES
from Crypto.Cipher.PKCS1_v1_5 import PKCS115_Cipher
from Crypto.PublicKey import RSA
from Crypto.Random import get_random_bytes
import hashlib
import json
import requests
steamid = '<MY_STEAMID64>'
steampass = '<MY_STEAM_PASSWORD>'
loginkey = hashlib.md5(bytes(steampass, 'utf-8')).hexdigest()
blob32 = get_random_bytes(32)
getrsa_url = 'https://steamcommunity.com/login/getrsakey/'
getrsa_data = {'username': '<MY_STEAM_USERNAME>'}
getrsa_resp = requests.get(getrsa_url, params=getrsa_data)
response = json.loads(getrsa_resp.text)
if response.get('success'):
steam_publickey_mod = response.get('publickey_mod')
steam_publickey_mod = int(steam_publickey_mod, 16)
steam_publickey_exp = response.get('publickey_exp')
steam_publickey_exp = int(steam_publickey_exp, 16)
steam_rsa_key = RSA.construct((steam_publickey_mod, steam_publickey_exp))
steam_rsa = PKCS115_Cipher(steam_rsa_key)
if steam_rsa_key.can_encrypt():
sessionkey = steam_rsa.encrypt(blob32)
if type(sessionkey) is tuple:
sessionkey = sessionkey[0]
steam_aes = AES.new(blob32)
encrypted_loginkey = steam_aes.encrypt(loginkey)
if all([steamid, sessionkey, encrypted_loginkey]):
authenticate_user_url = (
'https://api.steampowered.com/ISteamUserAuth/AuthenticateUser/v0001')
authenticate_user_json = {
'steamid': steamid,
'sessionkey': sessionkey,
'encrypted_loginkey': encrypted_loginkey,
}
if __name__ == '__main__':
import ipdb
ipdb.set_trace()
authenticate_user_resp = requests.post(url=authenticate_user_url,
data=authenticate_user_json)
authenticate_user_resp.ok returns False
authenticate_user_resp.status_code returns 403
authenticate_user_resp.reason returns Forbidden
Sorry for my bad English, please

AuthenticateUser doesn't do what you think it does. It's used by the Steam client to get web session logon cookies for the user who is currently logged into the client. The loginkey that AuthenticateUser asks for comes from the CM (the server which the client connects to).
If you want to log a user into the websites, you need to use the HTTP endpoints to do so. Once you have the RSA key and have encrypted your password with that key, you can authenticate by POSTing to https://steamcommunity.com/login/dologin/ with these urlencoded parameters in the body:
captcha_text - Empty string or the text of a CAPTCHA you've been prompted with
captchagid - The GID of the CAPTCHA you've been prompted with, or -1 if you haven't been
emailauth - The Steam Guard code sent to your email address, or empty string if not applicable
emailsteamid - Empty string
loginfriendlyname - Empty string
password - Your password, encrypted with the RSA public key, and the resulting ciphertext in base64
remember_login - true if you want to remember your login or false if not (the strings true and false)
rsatimestamp - The timestamp that you got with the RSA key
twofactorcode - The TOTP code you got from your mobile app, or empty string if not applicable
username - Your account name

As far as I am concerned, you are not allowed to do this operation, hence the "403 forbidden" So, you simply are not "authorized" to perform this with the credentials you have.
https://en.wikipedia.org/wiki/HTTP_403
A 403 response generally indicates one of two conditions:
Authentication was provided, but the authenticated user is not
permitted to perform the requested operation. The operation is
forbidden to all users. For example, requests for a directory listing
return code 403 when directory listing has been disabled.