I am trying to generate the same signature from http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-post-example.html using python,
DateKey = hmac.new(b'AWS4wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', b'20151229', hashlib.sha256).digest()
DateRegionKey = hmac.new(DateKey, b'us-east-1', hashlib.sha256).digest()
DateRegionServiceKey = hmac.new(DateRegionKey, b's3', hashlib.sha256).digest()
SigningKey = hmac.new(DateRegionServiceKey, b'aws4_request', hashlib.sha256).digest()
signature = hmac.new(other_policy, SigningKey, hashlib.sha256).hexdigest()
But my signature is cb0b0ec487fd5e01382c9c3b6b6a6dfa170da312ddab58a4b18869e7413951be and expected signature is 46503978d3596de22955b4b18d6dfb1d54e8c5958727d5bdcd02cc1119c60fc9
Where am I doing wrong ?
Note
other_policy = b'''eyAiZXhwaXJhdGlvbiI6ICIyMDE1LTEyLTMwVDEyOjAwOjAwLjAwMFoiLA0KICAiY29uZGl0aW9ucyI6IFsNCiAgICB7ImJ1Y2tldCI6ICJzaWd2NGV4YW1wbGVidWNrZXQifSwNCiAgICBbInN0YXJ0cy13aXRoIiwgIiRrZXkiLCAidXNlci91c2VyMS8iXSwNCiAgICB7ImFjbCI6ICJwdWJsaWMtcmVhZCJ9LA0KICAgIHsic3VjY2Vzc19hY3Rpb25fcmVkaXJlY3QiOiAiaHR0cDovL3NpZ3Y0ZXhhbXBsZWJ1Y2tldC5zMy5hbWF6b25hd3MuY29tL3N1Y2Nlc3NmdWxfdXBsb2FkLmh0bWwifSwNCiAgICBbInN0YXJ0cy13aXRoIiwgIiRDb250ZW50LVR5cGUiLCAiaW1hZ2UvIl0sDQogICAgeyJ4LWFtei1tZXRhLXV1aWQiOiAiMTQzNjUxMjM2NTEyNzQifSwNCiAgICB7IngtYW16LXNlcnZlci1zaWRlLWVuY3J5cHRpb24iOiAiQUVTMjU2In0sDQogICAgWyJzdGFydHMtd2l0aCIsICIkeC1hbXotbWV0YS10YWciLCAiIl0sDQoNCiAgICB7IngtYW16LWNyZWRlbnRpYWwiOiAiQUtJQUlPU0ZPRE5ON0VYQU1QTEUvMjAxNTEyMjkvdXMtZWFzdC0xL3MzL2F3czRfcmVxdWVzdCJ9LA0KICAgIHsieC1hbXotYWxnb3JpdGhtIjogIkFXUzQtSE1BQy1TSEEyNTYifSwNCiAgICB7IngtYW16LWRhdGUiOiAiMjAxNTEyMjlUMDAwMDAwWiIgfQ0KICBdDQp9'''
from the http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-post-example.html
Bug in the code
Your code to generate the signing key looks fine. However, when you generate the signature, the parameters are switched around. The pseudocode is:
Hex(HMAC-SHA256(SigningKey, StringToSign))
So instead of
signature = hmac.new(other_policy, SigningKey, hashlib.sha256).hexdigest()
you should have
signature = hmac.new(SigningKey, other_policy, hashlib.sha256).hexdigest()
Still wrong?
This produces the following signature, which despite the Amazon documentation, I believe is correct for the provided base64 string to sign:
8afdbf4008c03f22c2cd3cdb72e4afbb1f6a588f3255ac628749a66d7f09699e
So why does Amazon say the signature should be 465039...c60fc9???
I'm sorry to say I have no idea. I suspect the documentation may actually be inaccurate, either with respect to the signature value or with respect to the input parameter values used to generate it (secret key, date, region, service, string to sign).
I do feel like I'm going out on a limb a bit suggesting such a thing, but at the same time, I know from experience that not every piece of technical documentation online is 100% accurate (even when produced by a reputable source).
I couldn't reproduce this signature no matter what I tried. I'd love to see an answer that successfully produces this hash.
Evidence
There are really only two things going on here:
Calculate a signing key.
Pass it along with the "string to sign" to a hashing function, to generate the signature.
Your code produces the expected signing key using the parameters in the example under "Deriving the Signing Key with Other Languages" here. This suggests you're calculating the signing key correctly. Python code here.
Your code also produces the expected signature when using the parameters from this example. This suggests you're both calculating the correct signing key and the correct signature. Python code here.
Running the parameters from your question through some existing Python code that I know to work with several services (based on this) also produces the same 8afdb...9699e signature.
I also threw the java signing code into an existing spring boot application, and it also produces the same 8afdb...9699e signature for your input parameters.
What to try?
I suggest you assume your signing code is good, and the AWS documentation on this particular page is mistaken. After all, your code works fine with at least two other AWS examples.
Sign a real request, using your credentials, your bucket policy, your region, the current date, etc.
Then post the sample form and see if it works. If it doesn't work, you could update the question with the error you get from the form POST.
Update (7/15/2018)
AWS has updated their documentation, and this page now contains the correct signature. The incorrect version can be viewed here, for historical purposes.
Interestingly, the wayback machine suggests that the page was correct until at least June 23, 2017. By July 14 it had changed to the wrong signature, and it stayed incorrect at least until December 19, 2017 (over 5 months!).
Related
Does anyone have experience with the Dota 2 API library in Python called 'dota2api'? I wish to pull a list of 200 recent games filtered by various criteria. I'm using the get_match_history() request (see link). Here's my code:
import dota2api
key = '<key>'
api = dota2api.Initialise(key)
match_list = api.get_match_history(matches_requested=200)
I haven't specified any filters yet, since I can't even get the matches_requested argument to work. When I run this code, I get exactly 100 matches. I fact, no matter how I specify the matches_requested argument, I allways get 100 matches.
Does anyone know if I'm specifying the argument wrong or some other reason why it's working as intended?
Thanks in advance.
For such rarely used libraries it is hard to get an answer here.
I have found this issue on the library's Github:
You can't get more than 500 matches through get_match_history, it's
limited by valve api. One approach you can do is alternate hero_id,
like, requesting with account_id, hero_id and start_at_match_id (none
if first request), values assigned, this way you can get at least 500
matches of each hero from that account_id.
Probably that has since changed and now the parameter is ignored by the API completely. Try creating a new issue on the Github.
I have never been able to get Rest APIs to completely work with AWS. The error messages I have seen have been about the time not being correct or the command not being recognized (e.g., list-users). I have verified the "version" was appropriate for the command with AWS's website documentation.
I am trying to use curl with Linux to list the users or instances in my AWS account. I have a problem when I run it. My current error, that I would like to focus on, is "request signatures calculated does not match the signature provided." I went through the process of creating a signature carefully. It wasn't that surprising that it did not work given the hours of trouble and the seemingly many potential pitfalls in the tedious task of creating a signature.
I used this link to generate the hexadecimal string for the signature:
http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
I analyzed the output of the signatureKey using a modification of the Python code in the above link. The result is not hexadecimal nor alphanumeric. The result is a combination of special non-alphabet, non-numeric symbols and alphabet letters. I tried to work around this problem by using import binascii and binascii.hexlify. I was able to get a hexadecimal string from otherwise strictly adhering to the sample of Python code from the above link. I tend to think my signatureKey is not right because of this binascii work that I had to do. But what did I do wrong? How is that Python code supposed to calculate a signature?
Alternatively, are there thorough directions not written by Amazon to create a signature key? The process is not simple and seemingly error prone. I could start over with creating a signature if someone cannot clearly tell me how to create a signature. Amazon's forums have few postings related to this topic. I'd prefer to create the signature with Python. If someone recommends Ruby (an accessible language for me), I could try something like that.
I'm trying to create a python script which installs a default signature for a Mac Mail account. So far, my script successfully installs the signature by manipulating ubiquitous_AllSignatures.plist and creating a corresponding ubiquitous_<SignatureId>.mailsignature. Both of these files are at ~/Library/Mobile Documents/com~apple~mail/Data/V3/MailData/Signatures/. The problem is the installed signature is not recognized by Mail as the default signature, and in order to use the signature you have to manually select it in the signature dropdown while composing an email.
I've searched the file system long and hard to find a relevant plist which dictates the 'default' signature behavior for a given account. I have grep'd for the Signature ID and the only place I can see it other than the places mentioned above is in the sqlite files in ~/Library/Application Support/CloudDocs/session/db/. I don't think this is a good sign for the feasibility of my default signature install.
I understand this is a very specialized/weird/hacky problem, but I'm looking for some insight into the underlying mechanism controlling the default signature for an account in Mac Mail, or at least confirmation of my suspicions that it's impossible to do this without the consent of Mail. Any feedback is much appreciated.
Ok I realize this is incredibly obscure, but because I have been helped greatly by self-answers to obscure questions I'm gonna try to help anyone who may face this problem in the future.
The 'default' signature is indeed set in a plist. This plist is ~/Library/Containers/com.apple.mail/Data/Library/Preferences/com.apple.mail.plist. Be weary, this is a binary plist, which is different and much less readable than an xml plist. But fear not, you can transform back and forth b/w binary and xml with the plutil utility. You can access this utility in python with biplist, hooray! Now it's just a matter of manipulating the SignaturesSelected dictionary within the plist by adding the signature id (string) for the account id (key). And you will have set your default signature!
I'm working with the application, which I have made test in my old tool (HP QTP), now I need convert it to selenium(python webdriver), I have issue with specific part, during the test flow I receive image with the value, (data is static only for testing), so I know what value I should send for response.
In HP QTP I solved it by 'if conditional' and loop (VB code below):
For Each elem As HtmlElement In WebBrowser1.Document.GetElementsByTagName("img")
If elem.GetAttribute("src") = "http://URL/image_1.png" Then
WebBrowser1.Document.GetElementById("response").SetAttribute("value", "70")
End If
If elem.GetAttribute("src") = "http://URL/image_2.png" Then
WebBrowser1.Document.GetElementById("response").SetAttribute("value", "80")
End If
I don't know how it should looks in python. I didn't found equivalent function for "elem.GetAttribute("")" in Python.
Unfortunately I have lack experience with python so I'm looking for solution or hints for this issue.
There is a get_attribute method available in python API. You need to find the element and be able to perform same action you are doing in vb. Consult this and here is an example how to do it.
I need to digitally sign some text in python using a private key stored in a .pem file. It seems like M2Crypto is the preferred way to do that these days, so that's what I'm using. I think I get most of it, but I'm confused about how to configure padding. To be specific, I need to verify the signature in an iPhone app, using a padding scheme called kSecPaddingPKCS1SHA1 and described like this:
Data to be signed is a SHA1 hash.
Standard ASN.1 padding will be done, as well as PKCS1 padding of the underlying RSA operation.
Not being a crypto expert, I have only a fuzzy idea what this means. I've tried to look at some of the RFCs but found them impenetrable. I see that the encryption/decryption methods of RSA objects take padding types, but I don't see anything similar related to signature verification.
Any help, especially with code, will be appreciated.
(In some sense this is the converse of this question.)
Ok, the answer given below is correct AFAICT. The following code generates a signature for text that validates on the iPhone using the kSecPaddingPKCS1SHA1 padding scheme.
from M2Crypto import EVP
privkey = EVP.load_key("privkey.pem")
privkey.sign_init()
privkey.sign_update(text)
signature = privkey.sign_final()
(Sorry to editorialize, but can I just say that crypto hackers are some of the lousiest documentation writers in the universe?)
AFAIK M2Crypto adds padding where it's required.
PKCS1 padding is the default.
But, (again only AFAIK), signatures don't have padding, padding is only added to encrypted data to prevent a possible attack.
EDIT: user caf, in a comment says that a padding is essnetial to a good signature. I'm still recommending you try it with the default M2Crypto behavior, it might add it.
On M2Crypto's generated docs you can see that the {public,private}_{encrypt,decrypt} methods have a padding option, which is PKCS1 by default, while the sign menthod has none.
IMO just give it a shot with the default M2Crypto params, it will probably work.