Develop Reference

How do I use Public Key Encryption with the AWS Encryption SDK in Python?

I'm a bit out to sea on this one, so I was wondering whether anyone could help.
Does anyone know how to use Public Key encryption/decryption, using RSA keys in PEM format?
I can get it to work if I use the private key in both directions, I can get the public key to encrypt, but I don't know how to structure a script to get it to work if I want to use a public key to encrypt and a private key to decrypt. I see there is an example in the Java based version of the SDK, but I can't even figure it out from that.
Can anyone lead me in the right direction?
Some sample code of the encryption process i'm using with a public key:
import os
import aws_encryption_sdk
from aws_encryption_sdk.internal.crypto import WrappingKey
from aws_encryption_sdk.key_providers.raw import RawMasterKeyProvider
from aws_encryption_sdk.identifiers import WrappingAlgorithm, EncryptionKeyType
class StaticPublicMasterKeyProvider(RawMasterKeyProvider):
provider_id = 'static-public'
def __init__(self, **kwargs):
self._public_keys = {}
def _get_raw_key(self, key_id):
with open("public_key.pem", "rb") as key_file:
public_key = key_file.read()
self._public_keys[key_id] = public_key
return WrappingKey(
wrapping_algorithm=WrappingAlgorithm.RSA_OAEP_SHA512_MGF1,
wrapping_key=public_key,
wrapping_key_type=EncryptionKeyType.PUBLIC
)
if __name__ == '__main__':
source_file = r'myfile.jpg'
source_file_enc = source_file + '.encrypt'
public_key_id = os.urandom(8)
master_key_provider = StaticPublicMasterKeyProvider()
master_key_provider.add_master_key(public_key_id)
with open(source_file, 'rb') as sf, open(source_file_enc, 'wb') as sfe:
with aws_encryption_sdk.stream(
mode='e',
source=sf,
key_provider=master_key_provider
) as encryptor:
for chunk in encryptor:
sfe.write(chunk)
I have reviewed the python examples on AWS and they are using private keys in both directions.
Any help would be greatly appreciated.
EDIT: links to documentation:
AWS Encryption SDK Developers Guide
Python example generating RSA Key but using private key
Java example using RSA Public key
Note: the two examples use multiple key providers, but still incorporate RSA Keys

OK, I have finally been given the example that I needed. For current context, the current example resides in a feature branch only on github (so caution in the future, as this link may be broken. You may need to search in master to find the example needed):
https://github.com/aws/aws-encryption-sdk-python/blob/keyring/examples/src/master_key_provider/multi/aws_kms_with_escrow.py
The guts of it can be described as follows (directly out of the above example):
# Create the encrypt master key that only has access to the public key.
escrow_encrypt_master_key = RawMasterKey(
# The provider ID and key ID are defined by you
# and are used by the raw RSA master key
# to determine whether it should attempt to decrypt
# an encrypted data key.
provider_id="some managed raw keys", # provider ID corresponds to key namespace for keyrings
key_id=b"my RSA wrapping key", # key ID corresponds to key name for keyrings
wrapping_key=WrappingKey(
wrapping_key=public_key_pem,
wrapping_key_type=EncryptionKeyType.PUBLIC,
# The wrapping algorithm tells the raw RSA master key
# how to use your wrapping key to encrypt data keys.
#
# We recommend using RSA_OAEP_SHA256_MGF1.
# You should not use RSA_PKCS1 unless you require it for backwards compatibility.
wrapping_algorithm=WrappingAlgorithm.RSA_OAEP_SHA256_MGF1,
),
)
# Create the decrypt master key that has access to the private key.
escrow_decrypt_master_key = RawMasterKey(
# The key namespace and key name MUST match the encrypt master key.
provider_id="some managed raw keys", # provider ID corresponds to key namespace for keyrings
key_id=b"my RSA wrapping key", # key ID corresponds to key name for keyrings
wrapping_key=WrappingKey(
wrapping_key=private_key_pem,
wrapping_key_type=EncryptionKeyType.PRIVATE,
# The wrapping algorithm MUST match the encrypt master key.
wrapping_algorithm=WrappingAlgorithm.RSA_OAEP_SHA256_MGF1,
),
)
If needs be, the escrow_encrypt_master_key can be added to a key ring to provide multiple keys to encrypt your payload.
I hope this helps someone in the future.
Thanks

How to use decrypt with RSA private key and SHA256 on python

I am learning for school to encrypt and decrypt a file using public and private keys en encoding.
I used this code to encode the message. (which generates public key ≠ not private key error)
from Crypto.Signature import pkcs1_15
from Crypto.Hash import SHA256
from Crypto.PublicKey import RSA
def signing():
#open file = message als binary
message = open('C:/Users/Gebruiker/Desktop/message.txt', "rb").read()
#open public key -> key
key = RSA.import_key(open('C:/Users/Gebruiker/Desktop/public.pem').read())
#message becomes a hash
h = SHA256.new(message)
#f = open file as write binary
f = open('C:/Users/Gebruiker/Desktop/message.signature', 'wb')
# sign hash message with private key
signature = pkcs1_15.new(key).sign(h)
#write signed hash to file
f.write(signature)
f.close()
But now i am trying to decode this message and I found all these people that do it in different ways and work with different type of encoding and encrypting. And I cant find a clear answer.
what i have right now is this
First i have to read the message so
def decrypt():
f = open('C:/Users/Gebruiker/Desktop/message.signature', 'rb').read()
then i open up my private key
key = RSA.import_key(open('C:/Users/Gebruiker/Desktop/private.pem').read())
Because I write and read in binary i have to turn it back into a hash
h = SHA256.new(f)
And then i have to decrypt the hash using my private key.???
Instead i see a lot of people using something like this.
h = pkcs1_15.new(key).sign(h) # sign hash message with private key
which i don't get. You are supposed to decode it right? not sign it again. this part makes no sense to me.
Now I have 2 problems.
I get an encoding error saying my public key is not a private key. Which is kinda the point of public keying right. so only private key can decrypt? Why am i getting an error?
I don't know how to proceed with the decrypting of my message
Can anybody help me with this?
Thanks a lot!

There is confusion in your question. Signature generation for RSA requires modular exponentiation using the values of the private key, not the public key. Modular exponentiation is also used for encryption with the public key. But although the same mathematical formula is used - at least on the surface - doesn't mean that signature generation is encryption with the private key because such a thing does not exist. Current PKCS#1 standards go out of their way to explain this fact, even though earlier PKCS#1 standards used to identify signature generation with RSA encryption.
What you are trying to do is to verify the message. That's the function you would expect rather than sign. Verification is performed by a trusted public key, not by a private key. You are not trying to decode the message, you are trying to verify that the bytes of the message are indeed signed by the private key that belongs to the same key pair as the trusted public key. Generally, the message is not recovered, not even partially. PKCS#1 is called signature generation with appendix, which contrasts with other schemes called signature generation giving message recovery. The appendix is the signature value, it needs to be appended (included with) the message to be of any use.
Actually, the fact that you can at least recover the hash over the message is specific to some schemes like PKCS#1 signature generation (officially called RSASSA-PKCS1-v1_5 in the standard). Other schemes such as PSS in the same standard may not even recover the hash. This is OK as long the verification (which can take place given the data and therefore hash) can succeed or fail. In other words, the verification should at least result in a boolean true / false, but it doesn't need to generate any other information.
Or, in simplified pseudo-code:
ciphertext = encrypt(publicKey, plaintext)
(recovered) plaintext = decrypt(privateKey, ciphertext)
and
signature = sign(privateKey, data)
verificationResult = verify(publicKey, data, signature)
where the data hashing algorithm is a configuration parameter for the signature generation & verification algorithm. If you want to include it you could e.g. include it as initial parameter:
signature = sign(SHA256alg, privateKey, data)
verificationResult = verify(SHA256alg, publicKey, data, signature)
Finally, you are talking about "decoding". You decode messages that have been encoded using an encoding scheme. Encoding/decoding does not presume the presence of a key. We're talking about encryption / decryption and signature generation / verification instead. Examples of encoding are hexadecimals / base 64 that convert binary to text. Character encoding such as UTF-8 is about converting text to binary.

The convention is to encrypt with a receives public RSA key so that only the holder of the corresponding private key can decrypt the message.
Also by convention, you would use your private RSA key to create a signatur that everybody else with the corresponding public key can verify.
In principle you could use a public key for creating a signature, but this would be an erroneous use case of the key, and is often prevented in the libraries implementing RSA. In your case you get a "public key ≠ not private key error" as you try to use a public key in a sign(..) call.
When signing you don't use the full message as input to RSA, but instead calculates a hash (you use SHA256). This hash would not need to be "decryped" in the signature verification, but instead recalculated on the original message, that you want to verify.

How to validate a private key using a public key in python

I was wondering how I use a public key to validate a private key.
I have absolute no idea what I am doing
I am given a public key in the form :
"<RSAKeyValue><Modulus>tuRctbsnB4OSsR7gqNy1ovYZ4vDTn543o4ldX8Wthfjk7dAQKPHQYUmB7EyC4qFQ2GY3/Q+mDjJBDCWbsb8gyFuyU3L93UJ/7szvO+2A/t520srjCN4Yv7HirgpAI0LaWlo1UUUixMU2+kYNv/kBeVUL47TvOIpm0JqstQVDHhJtNMwcbY+3Q0nN4D1jNkSrQitCF3Sdms1kwsIFcdHcUh3WcUBkIefcB97DZKVY915IFbhf1/xdpPBa/E0WjNgtF5q4FI5ClH2CxsDwy2mL6qzZMvRPNWUhaFKlX+CcGvFQOtuJ4K8PZ0P3Wsq55ccxafZp3BQrEcBbto5Cll/E0Q==</Modulus><Exponent>AQAB</Exponent></RSAKeyValue>"
which is a string and I am trying to validate a private key (which is also provided as a string) with it using python. I know what the issuer and audience needs to be but I'm not sure what to do with all this information.
I have looked at some peoples examples of using various packages but I can't seem to figure out what type pub_key, private_key, and message would be... Please help me... I wish to learn

Base 64 decode the components - modulus and public exponent - of the public key from within the XML and create the public key from the decoded unsigned, big endian number values.
Then create a signature with the private key over any data and verify it with the public key using the same RSA signature algorithm such as RSA with PKCS#1 v1.5 padding. If it verifies, the keys form a key pair.
OK, since one user seems to keep struggling, let's actually give some code for you to work with (using Python3 for the integer creation using from_bytes):
#!/bin/python
from Crypto.PublicKey import RSA
from xml.dom import minidom
from base64 import b64decode
document = """\
<RSAKeyValue>
<Modulus>tuRctbsnB4OSsR7gqNy1ovYZ4vDTn543o4ldX8Wthfjk7dAQKPHQYUmB7EyC4qFQ2GY3/Q+mDjJBDCWbsb8gyFuyU3L93UJ/7szvO+2A/t520srjCN4Yv7HirgpAI0LaWlo1UUUixMU2+kYNv/kBeVUL47TvOIpm0JqstQVDHhJtNMwcbY+3Q0nN4D1jNkSrQitCF3Sdms1kwsIFcdHcUh3WcUBkIefcB97DZKVY915IFbhf1/xdpPBa/E0WjNgtF5q4FI5ClH2CxsDwy2mL6qzZMvRPNWUhaFKlX+CcGvFQOtuJ4K8PZ0P3Wsq55ccxafZp3BQrEcBbto5Cll/E0Q==</Modulus>
<Exponent>AQAB</Exponent>
</RSAKeyValue>
"""
xmldoc = minidom.parseString(document)
modulusB64 = xmldoc.getElementsByTagName('Modulus')[0].firstChild.data
modulusBin = b64decode(modulusB64)
modulus = int.from_bytes(modulusBin, 'big', signed=False)
exponentB64 = xmldoc.getElementsByTagName('Exponent')[0].firstChild.data
exponentBin = b64decode(exponentB64)
exponent = int.from_bytes(exponentBin, 'big', signed=False)
public_key = RSA.construct((modulus, exponent))
Unfortunately I don't havea signature to verify (or a private key to decrypt) so I cannot help you verifying that the public key belongs to the private key. I guess there should be code samples out there that shows basic signature verification or encryption with the public key. Note that the modulus should also be unique and present in both keys for RSA, but that only helps with identification of the keys in the key pair, not so much validation.
Beware that I'm not a python programmer by profession, so there may be shortcuts for what I'm doing; feel free to edit them in.

How to access a dictionary key inside a string

I have an API that expects to receive the data in a string format. The data looks like this:
test = """{"API_name":"getScenario","token":"1112223333","clientId":"1","clientEmail":"yup#nope#gmail.com", "more": "hello"}"""
I am used to accessing the dictionary keys rather easily test[token] but in this case it is all encased in a multi-line string.
How is this supposed to be accessed?

Parse the string and then find access by key
import json
data = json.loads(test)
data['API_name']

How to encrypt using public key?

msg = "this is msg to encrypt"
pub_key = M2Crypto.RSA.load_pub_key('mykey.py') // This method is taking PEM file.
encrypted = pub_key.public_encrypt(msg, M2Crypto.RSA.pkcs1_padding)
Now I am trying to give file containing radix64 format public key as a parameter in this method and unable to get expected result i.e encryption using radix64 format public key.
Is there any other method in the Python API which can encrypt msg using public key after trying some mechanism?
I am getting my public key from public key server with some HTML wrapping and in radix64 format. How can I convert the public key so that it can be accepted by any encryption method?

In case someone searches for this question, I was getting the same error message:
M2Crypto.RSA.RSAError: no start line
In my case, it turned out that my keys were of type unicode. This was solved by converting the key back to ascii:
key.encode('ascii')

Did you ask this question before? See my answer to this question,
how to convert base64 /radix64 public key to a pem format in python

There is an error somewhere in the public key. It appears to be a PGP public key but with different line lengths, so it can't be used directly as an RSA public key.