Develop Reference

Decrypt function only returns partial string and not the entire value

I'm writing a test script which I'm using to try and decrypt an encrypted string I have the key to. However, while the code somewhat runs, it is not printing the full string/value that I am expecting (and know the result for).
For example, rather than returning ThisIsTheStringThatWorks it is returning atWorks
Here is the code:
import base64
import hashlib
from Crypto.Cipher import AES
BLOCK_SIZE = 16
unpad = lambda s : s[0:-s[-1]]
def decrypt(enc, secret_key):
private_key = hashlib.sha256(secret_key.encode('utf-8')).digest()
enc = base64.b64decode(enc)
iv = enc[:BLOCK_SIZE]
cipher = AES.new(private_key, AES.MODE_CBC, iv)
return unpad(cipher.decrypt(enc[BLOCK_SIZE:]))
decrypted = decrypt(mail_pass, secret_key)
print(bytes.decode(decrypted))
Any help would be greatly appreciated. Thanks in advance.

You have not included the IV in the encrypted message. You are using the first encrypted block as the IV during decryption and as a result the data encrypted in the first block is lost.
You must concatenate the IV and the cipher text in the encryption routine if you want this decryption routine to work:
enc = iv + cipher.encrypt(clear)

how do I use this pycrypto code in pycryptodome

how do I use this pycrypto code in pycryptodome:
#!/usr/bin/env python
from Crypto.Cipher import AES
import base64
import os
# the block size for the cipher object; must be 16 per FIPS-197
BLOCK_SIZE = 16
# the character used for padding--with a block cipher such as AES, the value
# you encrypt must be a multiple of BLOCK_SIZE in length. This character is
# used to ensure that your value is always a multiple of BLOCK_SIZE
PADDING = '{'
# one-liner to sufficiently pad the text to be encrypted
pad = lambda s: s + (BLOCK_SIZE - len(s) % BLOCK_SIZE) * PADDING
# one-liners to encrypt/encode and decrypt/decode a string
# encrypt with AES, encode with base64
EncodeAES = lambda c, s: base64.b64encode(c.encrypt(pad(s)))
DecodeAES = lambda c, e: c.decrypt(base64.b64decode(e)).rstrip(PADDING)
# generate a random secret key
secret = os.urandom(BLOCK_SIZE)
# create a cipher object using the random secret
cipher = AES.new(secret)
# encode a string
encoded = EncodeAES(cipher, 'password')
print 'Encrypted string:', encoded
# decode the encoded string
decoded = DecodeAES(cipher, encoded)
print 'Decrypted string:', decoded

At the very least, you need to replace:
cipher = AES.new(secret)
with:
cipher = AES.new(secret, AES.MODE_ECB)
PyCryptodome is almost compatible with PyCrypto, with the exceptions of a few dangeours APIs (like leaving ECB as default encryption mode).

pycryprodome AES CBC mismatch after decryption in Python

my goal is to have a very simple AES 128 CBC scheme which encrypts a plaintext and then decrypts it based on a given key in Python. I'm using pycryptodome framework and I couldnt find any documentation with an example of the AES CBC scheme.
Following is my code. The decrypted data is not same as the data before encryption. Will be fantastic if someone can help me identify what is going wrong here.
key = b'Sixteen byte key'
data = 'Jeevan B Manoj'.encode("UTF-8")
data = pad(data,16)
cipher = AES.new(key, AES.MODE_CBC)
print("data before encryption")
print(data)
ciphertext = cipher.encrypt(data)
cipher = AES.new(key, AES.MODE_CBC)
plaintext = cipher.decrypt(ciphertext)
print(plaintext)

As t.m.adam noted, the CBC mode of operation requires an initialization vector (IV) to work. Because the IV is commonly forgotten (also that it has to be unique and unpredictable, e.g. random), Pycryptodome creates a random one when a cipher object is initialized.
The IV must be unique for each encryption and is required for decryption. Common practice (source?) is to put the IV at the start of the ciphertext (the IV does not to need to be secret).
To make your example work:
from Crypto.Cipher import AES
from Crypto.Util.Padding import pad, unpad
# Do not use raw passwords as keys,
# use a derivation functions to generate keys from them
key = b'Sixteen byte key'
data = 'Jeevan B Manoj'.encode("UTF-8")
data = pad(data, AES.block_size)
encryptor = AES.new(key, AES.MODE_CBC)
iv = encryptor.IV
decryptor = AES.new(key, AES.MODE_CBC, IV=iv)
ciphertext = encryptor.encrypt(data)
plaintext = decryptor.decrypt(ciphertext)
assert plaintext == data
Important note: The ciphertext and IV must be authenticated for security (so data cannot be tampered with). For that, Pycryptodome offers AEAD modes like EAX and GCM as pointed out by Hans-Peter Jansen on GitHub. For many of them padding is not required.

If you use MODE_ECB instead of MODE_CBC it works. I also didn't know what padding routine you are using so I used this one.
I have several other examples here:
https://github.com/SolarDon/pycryptodome/tree/master/Examples
from Crypto.Cipher import AES
# Padding for the input string --not related to encryption itself.
BLOCK_SIZE = 16 # Bytes
pad = lambda s: s + (BLOCK_SIZE - len(s) % BLOCK_SIZE) * chr(BLOCK_SIZE - len(s) % BLOCK_SIZE)
unpad = lambda s: s[:-ord(s[len(s) - 1:])]
key = b'Sixteen byte key'
data = 'Jeevan B Manoj'.encode("UTF-8")
data = pad(data)
cipher = AES.new(key, AES.MODE_ECB) # AES.MODE_CBC
print("data before encryption")
print(data)
ciphertext = cipher.encrypt(data)
cipher = AES.new(key, AES.MODE_ECB) # MODE_CBC
plaintext = cipher.decrypt(ciphertext)
print(unpad(plaintext))

CBC decryption using ECB function

Trying to solve Cryptopals Challenge 10 where have to CBC decrypt a text file against "YELLOW SUBMARINE" with an IV of all ASCII 0 (\x00\x00\x00 &c).
Link to the text file is following:
http://cryptopals.com/static/challenge-data/10.txt
I have followed the algorithm CBC uses by taking cipher text, decrypting(using ECB decryption) and then taking xor with Initialization Vector for first block and ciphertext(i-1) for subsequent blocks. However for some not-understandable reason I am not getting a readable decryption. I just see some weird characters when I print after decryption:
from Crypto.Cipher import AES
key ='YELLOW SUBMARINE'
iv = "%00%00%00"*32
iv = iv.replace('%',r'\x')
#XOR-ing function
def xor_strings(a, b):
return "".join(chr(ord(a1) ^ ord(b1)) for a1, b1 in zip(a, b))
#Taking input file and converting it into a single string
file = open('10.txt','r')
data = file.read()
block = 128
obj = AES.new(key, AES.MODE_ECB)
def split_len(string, size):
return [string[i:i+size] for i in range(0, len(string), size)]
mylist = split_len(data,block)
decrypted = ""
for i in range (0,len(mylist)):
mystr = obj.decrypt(mylist[i])
if (i==0):
decrypted = decrypted + xor_strings(mystr,iv)
else:
decrypted = decrypted + xor_strings(mystr, mylist[i-1])
print decrypted
What might be the problem here ?

The iv needs to be 16 zero bytes (the question is not clearly worded here when it says “ASCII 0”):
iv = "\x00" * 16
You need to base64 decode the file before decrypting it:
from base64 import b64decode
#...
file = open('10.txt','r')
data = file.read()
data = b64decode(data)
Finally your block size needs to be in bytes for this code to work, not bits:
block = 16

AES Encryption in PowerShell and Python

My goal is to be able to AES encrypt a string in PowerShell, send it to a UNIX system with python available, and decrypt the string back to plain text. I would also like to be able to do the inverse. I am not a crypto guy or a PowerShell/python programmer, but this is what I have been able to do with the code so far:
function Create-AesManagedObject($key, $IV) {
$aesManaged = New-Object "System.Security.Cryptography.AesManaged"
$aesManaged.Mode = [System.Security.Cryptography.CipherMode]::CBC
$aesManaged.Padding = [System.Security.Cryptography.PaddingMode]::Zeros
$aesManaged.BlockSize = 128
$aesManaged.KeySize = 256
if ($IV) {
if ($IV.getType().Name -eq "String") {
$aesManaged.IV = [System.Convert]::FromBase64String($IV)
}
else {
$aesManaged.IV = $IV
}
}
if ($key) {
if ($key.getType().Name -eq "String") {
$aesManaged.Key = [System.Convert]::FromBase64String($key)
}
else {
$aesManaged.Key = $key
}
}
$aesManaged
}
function Encrypt-String($key, $unencryptedString) {
$bytes = [System.Text.Encoding]::UTF8.GetBytes($unencryptedString)
$aesManaged = Create-AesManagedObject $key $IV
$encryptor = $aesManaged.CreateEncryptor()
$encryptedData = $encryptor.TransformFinalBlock($bytes, 0, $bytes.Length);
[byte[]] $fullData = $aesManaged.IV + $encryptedData
$aesManaged.Dispose()
[System.Convert]::ToBase64String($fullData)
}
function Decrypt-String($key, $encryptedStringWithIV) {
$bytes = [System.Convert]::FromBase64String($encryptedStringWithIV)
$IV = $bytes[0..15]
$aesManaged = Create-AesManagedObject $key $IV
$decryptor = $aesManaged.CreateDecryptor();
$unencryptedData = $decryptor.TransformFinalBlock($bytes, 16, $bytes.Length - 16);
$aesManaged.Dispose()
[System.Text.Encoding]::UTF8.GetString($unencryptedData).Trim([char]0)
}
# key passphrase is a 16 byte string that is used to create the AES key.
$key_passphrase = "MypassphraseKey1"
# base64 encode the key. The resulting key should be exactly 44 characters (43 characters with a single = of padding) (256 bits)
$Bytes = [System.Text.Encoding]::Ascii.GetBytes($key_passphrase)
$key =[Convert]::ToBase64String($Bytes)
# init is used to create the IV
$init = "This is an IV123"
# converts init to a byte array (e.g. T = 84, h = 104) and then sha1 hash it
$IV = (new-Object Security.Cryptography.SHA1Managed).ComputeHash( [Text.Encoding]::UTF8.GetBytes($init) )[0..15]
write-output "IV is equal to $IV"
write-output "AES key is $key"
$unencryptedString = "testing"
$encryptedString = Encrypt-String $key $unencryptedString
$backToPlainText = Decrypt-String $key $encryptedString
write-output "Unencrypted string: $unencryptedString"
write-output "Encrypted string: $encryptedString"
write-output "Unencrytped string: $backToPlainText"
The PowerShell script seems to be working fine for encrypting and decrypting. For the python side, I can define the same AES key value since it is just base64 encoded of my key's passphrase. However, I do not get the same encrypted value of the string when executing (e.g. PowerShell outputs UXKWIhtaUgFOvN13bvA4tx4+2Hjkv4v6I1G3Xfl6zp0= and Python outputs BOJ3Ox4fJxR+jFZ0CBQ25Q==). I believe these would need to match in order to be able to decrypt but I could be mistaken. I know setting a static IV and key makes it insecure, but I am willing to do that in order to be able to encrypt and decrypt across platforms (unless there is a better method using AES). Any help would be appreciated.
Python code
import base64, array
import Crypto
import Crypto.Random
from Crypto.Cipher import AES
def pad_data(data):
if len(data) % 16 == 0:
return data
databytes = bytearray(data)
padding_required = 15 - (len(databytes) % 16)
databytes.extend(b'\x80')
databytes.extend(b'\x00' * padding_required)
return bytes(databytes)
def unpad_data(data):
if not data:
return data
data = data.rstrip(b'\x00')
if data[-1] == 128: # b'\x80'[0]:
return data[:-1]
else:
return data
def encrypt(key, iv, data):
aes = AES.new(key, AES.MODE_CBC, iv)
data = pad_data(data)
return aes.encrypt(data)
def decrypt(key, iv, data):
aes = AES.new(key, AES.MODE_CBC, iv)
data = aes.decrypt(data)
return unpad_data(data)
def test_crypto ():
key = "MypassphraseKey1"
# found using the debugger in the PowerShell ISE to get the value byte value which was converted to hex
iv = "\x51\x72\x96\x22\x1b\x5a\x52\x01\x4e\xbc\xdd\x77\x6e\xf0\x38\xb7"
msg = b"testing"
# hex value of IV in powershell script is 51 72 96 22 1b 5a 52 01 4e bc dd 77 6e f0 38 b7
print("Value of IV: " + iv)
# base64 encode key
b64key = base64.b64encode(key)
print("AES key encoded: " + b64key)
code = encrypt(key, iv, msg)
# convert encrypted string to base64
b64encoded = base64.b64encode(code)
print("Encrypted string: " + b64encoded)
decoded = decrypt(key, iv, code)
print("Decoded: " + decoded)
if __name__ == '__main__':
test_crypto()

A couple suggestions:
A 16 character ASCII string is 128^16 = 5.19229686e33 possible key inputs. Base64-encoding 16 bytes yields 24 bytes (4*ceil(16/3)). So even though you are using a 192 bit AES key (theoretically 6.27710174e57 key combinations), you can use only 1/1208925820422879877545683 [one over one trillion trillion] of them. In fact, you set the key size to be 256 bits, and apparently the code is ignoring that/allowing the 192 bit key without an error.
Use Rfc2898DeriveBytes to derive your AES key rather than a Base64 transformation of a raw string. RFC 2898 defines PBKDF2 (Password-Based Key Derivation Function 2), an HMAC-based Key Derivation Function to securely derive encryption keys from passwords, and provides for HMAC/SHA1 used with a high number of iterations to mitigate brute-force attacks against your key.
You are only invoking TransformFinalBlock() on encrypt and decrypt in PowerShell. I imagine this will fail to encrypt or decrypt the complete message if the message is longer than one block (16 bytes). Try this with an input message like This is a plaintext message. (29 bytes). I believe you want to use both TransformBlock() and TransformFinalBlock().
You are correct that a static IV is unsafe (defeats the purpose of an IV, which should be unique and non-predictable for every encryption operation with the same key). AesManaged already provides a method GenerateIV() to generate a satisfactory IV, which you can access from the IV property and prepend to the cipher text.
Your PowerShell output of the Base64-encoded cipher text is 44 characters (16 byte IV + 16 byte ciphered message = 32 bytes -> 44 bytes in Base64). Your Python Base64 output is 24 characters (16 bytes -> 24 bytes in Base64). Either this output is not including the IV or the message (or some other less likely reason for the limited output). Looking at the code, your encrypt method does not prepend the IV onto the cipher text.
Finally, at this point, your code should work and be both internally consistent and cross-compatible. Here you should revisit a couple design decisions:
Zero padding is non-standard and while you have implemented it manually, a well-defined padding scheme like PKCS #5/#7 is more desirable. There are bountiful implementations and code examples for implementing this in both Python and .NET.
You are using the CBC block cipher mode of operation. While CBC is fine for confidentiality, it does not provide integrity. You should use an authenticated encryption mode (AE/AEAD) like GCM or EAX. If you cannot, provide a message authentication code (MAC) over the cipher text by using an HMAC construction like HMAC/SHA-256 with a different shared secret than your encryption key and verify the MAC with a constant-time method before attempting decryption.