Develop Reference

How to translate Python HMAC code to Swift

I've been struggling to convert the code from Python to Swift and can't get the same signature value. I've been trying different combinations in Swift but my requests are failing as the signature is incorrect. I've tried looking at other HMAC posts in swift but haven't had much luck.
Python code:
hashed = hmac.new(base64.urlsafe_b64decode(
(secretMessage_string).encode('utf-8'),
),
msg=message_string.encode('utf-8'),
digestmod=hashlib.sha256,
)
signature = base64.urlsafe_b64encode(hashed.digest()).decode()
Swift code:
let keyString = "specialKey"
let messageString = "This is a basic message"
let key = SymmetricKey(data: Data(keyString.utf8))
let signature = HMAC<SHA256>.authenticationCode(for: Data(messageString.utf8), using: key)

Try this:
import CryptoKit
let keyString = "specialKey"
let keyData = Data(base64Encoded: keyString)!
let messageString = "This is a basic message"
let key = SymmetricKey(data: keyData)
let signatureObj = HMAC<SHA256>.authenticationCode(for: Data(messageString.utf8), using: key)
let signatureHex = Data(signatureObj).map { String(format: "%02hhx", $0) }.joined()
let signature = Data(signatureHex.utf8).base64EncodedString()

How SurveyMonkey API creates the hmac?

We are trying to verify that the SurveyMonkey hmac we receive (sm-signature) is correct. To check this we create an hmac and compare it to SurveyMonkey's hmac.
We create the hmac as follows (we are working with nodejs):
let bodyString = JSON.stringify(req.body);
let body = Buffer.from(bodyString, "ascii");
let apiClientId = Buffer.from(surveyMonkeyClientId, "ascii");
let apiSecret = Buffer.from(surveyMonkeyApiSecret, "ascii");
let hmac = crypto
.createHmac('sha1', apiClientId+'&'+apiSecret)
.update(Buffer.from(body))
.digest()
.toString('base64');
We have verified this code with (it is with python): https://github.com/SurveyMonkey/public_api_docs/blob/main/includes/_webhooks.md
But for some reason this doesn't work as expected. Because the hmac we generated is not the same as the hmac generated by SurveyMonkey (sm-signature).
Could someone help us? Thanks!

The problem is the signature, that comes with spaces between the json fields and when you do JSON.stringify this removes the spaces.
One possible solution is:
let payloadString = JSON.stringify(req.body);
payloadString = payloadString.replace(/":/g, '": ');
payloadString = payloadString.replace(/,/g, ', ');

JSON to Protobuf in Python

Hey I know there is a solution for this in Java, I'm curious to know if anyone knows of a Python 3 solution for converting a JSON object or file into protobuf format. I would accept either or as converting to an object is trivial. Searching the stackoverflow site, I only found examples of protobuf->json, but not the other way around. There is one extremely old repo that may do this but it is in Python 2 and our pipeline is Python 3. Any help is as always, appreciated.

The library you're looking for is google.protobuf.json_format. You can install it with the directions in the README here. The library is compatible with Python >= 2.7.
Example usage:
Given a protobuf message like this:
message Thing {
string first = 1;
bool second = 2;
int32 third = 3;
}
You can go from Python dict or JSON string to protobuf like:
import json
from google.protobuf.json_format import Parse, ParseDict
d = {
"first": "a string",
"second": True,
"third": 123456789
}
message = ParseDict(d, Thing())
# or
message = Parse(json.dumps(d), Thing())
print(message.first) # "a string"
print(message.second) # True
print(message.third) # 123456789
or from protobuf to Python dict or JSON string:
from google.protobuf.json_format import MessageToDict, MessageToJson
message_as_dict = MessageToDict(message)
message_as_dict['first'] # == 'a string'
message_as_dict['second'] # == True
message_as_dict['third'] # == 123456789
# or
message_as_json_str = MessageToJson(message)
The documentation for the json_format module is here.

Here is a much simpler way by using xia-easy-proto module. No need to pre-define anything.
pip install xia-easy-proto
And then
from xia_easy_proto import EasyProto
if __name__ == '__main__':
songs = {"composer": {'given_name': 'Johann', 'family_name': 'Pachelbel'},
"title": 'Canon in D',
"year": [1680, 1681]}
song_class, song_payload = EasyProto.serialize(songs)
print(song_class) # It is the message class
print(song_payload) # It is the serialized message

How do you generate the signature for an Azure Blob storage SAS token in Python?

I am trying to build the SAS token required for a blob download URL in Python, following the instructions from MSDN.
My string to sign looks like:
r\n
2016-12-22T14%3A00%3A00Z\n
2016-12-22T15%3A00%3A00Z\n
%2Fblob%2Fmytest%2Fprivatefiles%2F1%2Fqux.txt\n
\n
\n
https\n
2015-12-11\n
\n
\n
\n
\n
_
I've added the newline symbols for clarity and the last line is supposed to be an empty line (with no newline at the end).
The Python method I use for signing the string is:
def sign(self, string):
hashed = hmac.new(base64.b64decode(self.account_key), digestmod=sha256)
hashed.update(string)
base64_str = base64.encodestring(hashed.digest()).strip()
return base64_str
The final URL I build looks like:
https://mytest.blob.core.windows.net/privatefiles/1/qux.txt?sv=2015-12-11&st=2016-12-22T14%3A00%3A00Z&se=2016-12-22T15%3A00%3A00Z&sr=b&sp=r&spr=https&sig=BxkcpoRq3xanEHwU6u5%2FYsULEtOCJebHmupUZaPmBgM%3D
Still, the URL fails with a 403. Any idea on what I am doing wrong?

Update, with the latest storage python library, this is what I used to generate the sas token:
def generate_sas_token(file_name):
sas = generate_blob_sas(account_name=AZURE_ACC_NAME,
account_key=AZURE_PRIMARY_KEY,
container_name=AZURE_CONTAINER,
blob_name=file_name,
permission=BlobSasPermissions(read=True),
expiry=datetime.utcnow() + timedelta(hours=2))
logging.info('https://'+AZURE_ACC_NAME+'.blob.core.windows.net/'+AZURE_CONTAINER+'/'+file_name+'?'+sas)
sas_url ='https://'+AZURE_ACC_NAME+'.blob.core.windows.net/'+AZURE_CONTAINER+'/'+file_name+'?'+sas
return sas_url
Python 3.6 and azure-storage-blob package.

The easiest way to generate SAS token in python is to leverage Azure Storage SDK for Python. Please consider following code snippet:
import time
import uuid
import hmac
import base64
import hashlib
import urllib
from datetime import datetime, timedelta
from azure.storage import (
AccessPolicy,
ResourceTypes,
AccountPermissions,
CloudStorageAccount,
)
from azure.storage.blob import (
BlockBlobService,
ContainerPermissions,
BlobPermissions,
PublicAccess,
)
AZURE_ACC_NAME = '<account_name>'
AZURE_PRIMARY_KEY = '<account_key>'
AZURE_CONTAINER = '<container_name>'
AZURE_BLOB='<blob_name>'
def generate_sas_with_sdk():
block_blob_service = BlockBlobService(account_name=AZURE_ACC_NAME, account_key=AZURE_PRIMARY_KEY)
sas_url = block_blob_service.generate_blob_shared_access_signature(AZURE_CONTAINER,AZURE_BLOB,BlobPermissions.READ,datetime.utcnow() + timedelta(hours=1))
#print sas_url
print 'https://'+AZURE_ACC_NAME+'.blob.core.windows.net/'+AZURE_CONTAINER+'/'+AZURE_BLOB+'?'+sas_url
generate_sas_with_sdk()
Furthermore, to generate SAS token via plain python script, you can refer to the source code at https://github.com/Azure/azure-storage-python/blob/master/azure/storage/sharedaccesssignature.py#L173 for more hints.

Here's an updated code snippet for Python3 and the updated Azure Storage Blob SDK:
from datetime import datetime, timedelta
from azure.storage.blob import (
BlockBlobService,
ContainerPermissions,
BlobPermissions,
PublicAccess,
)
AZURE_ACC_NAME = '<account_name>'
AZURE_PRIMARY_KEY = '<account_key>'
AZURE_CONTAINER = '<container_name>'
AZURE_BLOB='<blob_name>'
block_blob_service = BlockBlobService(account_name=AZURE_ACC_NAME, account_key=AZURE_PRIMARY_KEY)
sas_url = block_blob_service.generate_blob_shared_access_signature(AZURE_CONTAINER,AZURE_BLOB,permission=BlobPermissions.READ,expiry= datetime.utcnow() + timedelta(hours=1))
print('https://'+AZURE_ACC_NAME+'.blob.core.windows.net/'+AZURE_CONTAINER+'/'+AZURE_BLOB+'?'+sas_url)

Based on the documentation (Please see Constructing the Signature String section), the parameters passed to string to sign must be URL decoded. From the link:
To construct the signature string of a shared access signature, first
construct the string-to-sign from the fields comprising the request,
then encode the string as UTF-8 and compute the signature using the
HMAC-SHA256 algorithm. Note that fields included in the string-to-sign
must be URL-decoded.
Please use un-encoded parameter values in your string to sign and that should fix the problem.

How to read field from JSON encoding of XML message

Is there a way I can get the errorMessage tag from this response using some code in python?
{
"movies_search":{
"#xmlns":"http://www.ourmoviest.com/mv",
"#xmlns:xsi":"http://www.w3.org/2001/XMLSchema-instance",
"#xsi:schemaLocation":"http://www.ourmoviest.com/mv mv.xsd ",
"error":{
"errorMessage":"We cannot proceed search",
"statusCode":"00004"
}
}
}

The following Python script,
import json
respStr = """
{
"movies_search":{
"#xmlns":"http://www.ourmoviest.com/mv",
"#xmlns:xsi":"http://www.w3.org/2001/XMLSchema-instance",
"#xsi:schemaLocation":"http://www.ourmoviest.com/mv mv.xsd ",
"error":{
"errorMessage":"We cannot proceed search",
"statusCode":"00004"
}
}
}
"""
respObj = json.loads(respStr)
print respObj['movies_search']['error']['errorMessage']
will print the errorMessage string,
We cannot proceed search
as requested.

Yes, there is. See https://docs.python.org/2/library/json.html "Decoding JSON".
(I'm really tempted to simply ask "what have you tried yourself to reach the goal"...)
Edit: as requested by the first comment, here the essential code instead of a link-only answer:
import json;
responseJson = json.loads(response)
responseJson['movies_search']['error']['errorMessage']