Develop Reference

Decrypt with Fernet, TypeError: token must be bytes

Programming language: Python
i have three files, one is to generate the key, 2nd is to encrypt, and the other is to decrypt.. the 1st and the 2nd files work.. but the decrypt file won't work
generate key file:
from cryptography.fernet import Fernet
def generate_key():
"""
Generates a key and save it into a file
"""
key = Fernet.generate_key()
with open("secret.key", "wb") as key_file:
key_file.write(key)
generate_key()
encrypt file -
from cryptography.fernet import Fernet
def load_key():
return open("secret.key", "rb").read()
def encrypt_message(message):
key = load_key()
encoded_message = message.encode()
f = Fernet(key)
encrypted_message = f.encrypt(encoded_message)
print(encrypted_message)
EncryptedTextWriteFile = open("encrypt.txt", "r+")
EncryptedTextWriteFile.write(str(encrypted_message))
notEncryptedFile = open("text.txt", "r")
notEncryptedText = notEncryptedFile.read()
if __name__ == "__main__":
encrypt_message(notEncryptedText)
notEncryptedFile.close()
Decrypt file -
from cryptography.fernet import Fernet
def load_key():
return open("secret.key", "rb").read()
def decrypt_message(encrypted_message):
key = load_key()
f = Fernet(key)
decrypted_message = f.decrypt(encrypted_message)
shit = decrypted_message.decode()
print(shit)
DencryptedTextWriteFile = open("decrypt.txt", "r+")
DencryptedTextWriteFile.write(shit)
EncryptedFile = open("decrypt.txt", "r")
EncryptedText = EncryptedFile.read()
if __name__ == "__main__":
decrypt_message(EncryptedText)
the string i tried - ( this test is in the text.txt )
hirusha
the error i get:
Traceback (most recent call last):
File "d:/development/Python/Everything/Releases/0.2/Build 02/_Releases/Encoder and Decoder/decrypt.py", line 27, in <module>
decrypt_message(EncryptedText)
File "d:/development/Python/Everything/Releases/0.2/Build 02/_Releases/Encoder and Decoder/decrypt.py", line 15, in decrypt_message
decrypted_message = f.decrypt(encrypted_message)
File "C:\Users\hirusha\AppData\Local\Programs\Python\Python38\lib\site-packages\cryptography\fernet.py", line 75, in decrypt
timestamp, data = Fernet._get_unverified_token_data(token)
File "C:\Users\hirusha\AppData\Local\Programs\Python\Python38\lib\site-packages\cryptography\fernet.py", line 100, in _get_unverified_token_data
utils._check_bytes("token", token)
File "C:\Users\hirusha\AppData\Local\Programs\Python\Python38\lib\site-packages\cryptography\utils.py", line 29, in _check_bytes
raise TypeError("{} must be bytes".format(name))
TypeError: token must be byte

The problem is that when you open the file decrypt.txt and read the file, it gets converted into string. But the .decrypt function only accepts input in byte.
I think this should solve your problem:
Generates a key and stores it in key.key file:
from cryptography.fernet import Fernet
def write_key():
"""
Generates a key and save it into a file
"""
key = Fernet.generate_key()
with open("key.key", "wb") as key_file:
key_file.write(key)
write_key()
Encrypts a file and stores it in enc_text.txt file:
from cryptography.fernet import Fernet
with open("key.key","rb") as f:
key=f.read()
f = Fernet(key)
with open('text.txt', 'rb') as original_file:
original = original_file.read()
encrypted = f.encrypt(original)
with open ('enc_text.txt', 'wb') as encrypted_file:
encrypted_file.write(encrypted)
Decrypts the enc_text.txt file and writes the output in decrypt.txt
from cryptography.fernet import Fernet
with open("key.key","rb") as f:
key=f.read()
f = Fernet(key)
with open('enc_text.txt', 'rb') as encrypted_file:
encrypted = encrypted_file.read()
decrypted = f.decrypt(encrypted)
with open('decrypt.txt', 'wb') as decrypted_file:
decrypted_file.write(decrypted)

Sujay's answer was posted while I was writing this one. It is correct, but I still wanted to post this to perhaps give some more insight into what this means.
Most (if not all) encryption tools work on bytes, not str. This is both because these days encrypting binary (non-string) data is more common, and because the encryption algorithms work on numbers, and bytes are strings of numbers in range(0, 256).
In the decryption program, you are loading the ciphertext in this line:
EncryptedFile = open("decrypt.txt", "r")
Contrast this to how you load the encryption key:
def load_key():
return open("secret.key", "rb").read()
The difference is the mode parameter to open: if there is a b in the string, it means it operates in binary mode, working on bytes rather than str. Here's an example:
>>> text = open('test.txt', 'r').read()
>>> text
'Hello World!\n'
>>> binary = open('test.txt', 'rb').read()
>>> binary
b'Hello World!\n'
>>> type(text)
<class 'str'>
>>> type(binary)
<class 'bytes'>
This distinction is important because str represents character sequences, and those can be encoded into binary in vastly different ways. Which of these you choose will affect what ones and zeros go into the encryption algorithm, and consequently what comes out.
The reason the encryption program worked, despite also reading the plaintext file in text mode, is because of this line:
encoded_message = message.encode()
str.encode converts the string into bytes using the encoding supplied, or the default one (usually UTF-8) if it is not specified. The bytes.decode method, seen in the decryption program, does the opposite: it converts bytes to str.
>>> text = '稲妻の腕を借らん草枕'
>>> text.encode()
b'\xe7\xa8\xb2\xe5\xa6\xbb\xe3\x81\xae\xe8\x85\x95\xe3\x82\x92\xe5\x80\x9f\xe3\x82\x89\xe3\x82\x93\xe8\x8d\x89\xe6\x9e\x95'
>>> text.encode('utf-16')
b'\xff\xfe2z\xbbYn0U\x81\x920\x1fP\x890\x930I\x83\x95g'
>>> u16_data = text.encode('utf-16')
>>> u16_data.decode('utf-16')
'稲妻の腕を借らん草枕'
>>> u16_data.decode() # this is why specifying the encoding is important
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

Python3 - Encrypting and decrypting an image (Fernet) Issue

Good day,
I am doing an assignment for cryptography. It's an easy task I need to take any image, turn it into HEX, encrypt it and then decrypt it.
As I am working in Python and there was no specific encryption method in the task I just use Fernet.
I have an encryptor and decryptor scripts.
Encryption seems to be working because as a test I create a txt document with original HEX and after decryption the program states that original HEX and decrypted one are the same, however the decrypted image is not loading.
Could anyone help out a newbie?
Encryptor:
import binascii
from cryptography.fernet import Fernet
img = 'panda.png'
with open(img, 'rb') as f:
content = f.read()
hexValue = binascii.hexlify(content)
key = Fernet.generate_key()
with open('info/key.txt', mode='w+') as keyValue:
keyValue.write(key)
keyValue.seek(0)
f = Fernet(key)
encHexVal = f.encrypt(hexValue)
with open('info/encryptedHex.txt', mode='w+') as hexValueFile:
hexValueFile.write(encHexVal)
hexValueFile.seek(0)
a = f.decrypt(encHexVal)
with open('info/realValue.txt', mode='w+') as writeHex:
originalHex = writeHex.write(hexValue)
with open('info/realValue.txt', mode='r') as reading:
realValue = reading.read()
if realValue == a:
print("We're good to go!")
else:
print("Oops something went wrong. Check the source code.")
Decryptor:
import binascii
from cryptography.fernet import Fernet
with open('info/key.txt', mode='rb') as keyValue:
key = keyValue.read()
f = Fernet(key)
with open('info/encryptedHex.txt', mode='rb') as imageHexValue:
hexValue = imageHexValue.read()
a = f.decrypt(hexValue)
with open('info/realValue.txt', mode='r') as compare:
realContents = compare.read()
print("Small test in safe environment...")
if realContents == a:
print("All good!")
else:
print("Something is wrong...")
data = a.encode()
data = data.strip()
data = data.replace(' ', '')
data = data.replace('\n', '')
with open('newImage.png', 'wb') as file:
file.write(data)
I am using a random image from the internet of Po from Kung Fu Panda:

The principle problem is that although you hexlify then encrypt in the encryptor you don't unhexlify after decrypting in the decryptor. Its far more common to do things the other way, encrypt then hexlify so that the encrypted binary can be stored in regular text files or sent via http.
You have several problems with trying to write bytes objects to files open in text. I fixed those along the way. But it does leave me puzzled why a file called 'info/encryptedHex.txt' would be binary.
Encryptor
import binascii
from cryptography.fernet import Fernet
# Generate keyfile
#
# TODO: Overwrites key file on each run, invalidating previous
# saves. You could do `if not os.path.exists('info/key.txt'):`
key = Fernet.generate_key()
with open('info/key.txt', mode='wb') as keyValue:
keyValue.write(key)
# Encrypt image
img = 'panda.png'
with open(img, 'rb') as f:
content = f.read()
hexValue = binascii.hexlify(content)
f = Fernet(key)
encHexVal = f.encrypt(hexValue)
with open('info/encryptedHex.txt', mode='wb') as hexValueFile:
hexValueFile.write(encHexVal)
# Verification checks
a = f.decrypt(encHexVal)
# hexed bytes is same encoding as 'ascii'
with open('info/realValue.txt', mode='wb') as writeHex:
originalHex = writeHex.write(hexValue)
with open('info/realValue.txt', mode='r', encoding='ascii') as reading:
realValue = reading.read()
if realValue == a.decode('ascii'):
print("We're good to go!")
else:
print("Oops something went wrong. Check the source code.")
Decryptor
import binascii
from cryptography.fernet import Fernet
# Generate keyfile
#
# TODO: Overwrites key file on each run, invalidating previous
# saves. You could do `if not os.path.exists('info/key.txt'):`
key = Fernet.generate_key()
with open('info/key.txt', mode='wb') as keyValue:
keyValue.write(key)
# Encrypt image
img = 'panda.png'
with open(img, 'rb') as f:
content = f.read()
hexValue = binascii.hexlify(content)
f = Fernet(key)
encHexVal = f.encrypt(hexValue)
with open('info/encryptedHex.txt', mode='wb') as hexValueFile:
hexValueFile.write(encHexVal)
# Verification checks
a = f.decrypt(encHexVal)
# hexed bytes is same encoding as 'ascii'
with open('info/realValue.txt', mode='wb') as writeHex:
originalHex = writeHex.write(hexValue)
with open('info/realValue.txt', mode='r', encoding='ascii') as reading:
realValue = reading.read()
if realValue == a.decode('ascii'):
print("We're good to go!")
else:
print("Oops something went wrong. Check the source code.")
(base) td#timpad:~/dev/SO/Encrypting and decrypting in image$ cat de.py
import binascii
from cryptography.fernet import Fernet
with open('info/key.txt', mode='rb') as keyValue:
key = keyValue.read()
f = Fernet(key)
with open('info/encryptedHex.txt', mode='rb') as imageHexValue:
encHexValue = imageHexValue.read()
hexValue = f.decrypt(encHexValue)
binValue = binascii.unhexlify(hexValue)
with open('info/realValue.txt', mode='rb') as compare:
realContents = compare.read()
print("Small test in safe environment...")
if realContents == hexValue:
print("All good!")
else:
print("Something is wrong...")
with open('newImage.png', 'wb') as file:
file.write(binValue)

Error cryptography Fernet read token from file

I create the key and I encrypt the message send by user, and store this message in a file and after i try to decrypt this message in the file, but i get this error cryptography.fernet.InvalidToken. I don't use the time for the token and I have also converted in byte after.
My code is:
from cryptography.fernet import Fernet
key = Fernet.generate_key()
f = Fernet(key)
def encry():
global f
what = input("What i need to encrypt?")
what_b = str.encode(what)
token = f.encrypt(what_b)
print("key generated")
print("encrypted")
file1=open("string.txt", "w")
file1.write(str(token))
file1.close()
file2=open("key.txt", "w")
file2.write(str(key))
file2.close()
def decry():
global f
print("decrypted")
file1=open("string.txt", "r").read()
file_de=str.encode(file1)
file2=open("de.txt", "w")
what_d = f.decrypt(file_de)
file1.close()
file2.write(str(what_d))
file2.close()
choose = input("Do you want, encrypt or decrypt?" + " " + "(e/d)")
if choose == "e":
encry()
elif choose == "d":
decry()
else:
print("Nothing to do")

The problem comes from the way you write and read the message and the key to and from the files. Indeed, the encrypted text and the keys are not strings, but byte strings and need to be treated in a different way.
In your case, when reading and writing it is enough to specify that you do it in binary mode and you can do it by adding the b flag. See the documentation for more detailed information.
Moreover, in your decrypt you also need to read the key from the file.
def encry():
key = Fernet.generate_key()
f = Fernet(key)
what = "example"
what_b = str.encode(what)
token = f.encrypt(what_b)
with open("string.txt", "wb") as f1, open("key.txt", "wb") as f2:
f1.write(token)
f2.write(key)
def decry():
with open("string.txt", "rb") as f1, open("key.txt", "rb") as f2:
token = f1.read()
key = f2.read()
f = Fernet(key)
what_d = str(f.decrypt(token),'utf-8') #converting back to string
encry()
decry()

Python 2.7 CSV file read/write \xef\xbb\xbf code

I have a question about Python 2.7 read/write csv file with 'utf-8-sig' code, my csv . header is
['\xef\xbb\xbfID;timestamp;CustomerID;Email']
there have some code("\xef\xbb\xbfID") I read from file A.csv and I want write the same code and header to file B.csv
My print log is shows:
['\xef\xbb\xbfID;timestamp;CustomerID;Email']
But the actual output file header it looks like
ÔªøID;timestamp
Here is the code:
def remove_gdpr_info_from_csv(file_path, file_name, temp_folder, original_header):
new_temp_folder = tempfile.mkdtemp()
new_temp_file = new_temp_folder + "/" + file_name
# Blanked new file
with open(new_temp_file, 'wb') as outfile:
writer = csv.writer(outfile, delimiter=";")
print original_header
writer.writerow(original_header)
# File from SFTP
with open(file_path, 'r') as infile:
reader = csv.reader(infile, delimiter=";")
first_row = next(reader)
email = first_row.index('Email')
contract_detractor1 = first_row.index('Contact Detractor (Q21)')
contract_detractor2 = first_row.index('Contact Detractor (Q20)')
contract_detractor3 = first_row.index('Contact Detractor (Q43)')
contract_detractor4 = first_row.index('Contact Detractor(Q26)')
contract_detractor5 = first_row.index('Contact Detractor(Q27)')
contract_detractor6 = first_row.index('Contact Detractor(Q44)')
indexes = []
for column_name in header_list:
ind = first_row.index(column_name)
indexes.append(ind)
for row in reader:
output_row = []
for ind in indexes:
data = row[ind]
if ind == email:
data = ''
elif ind == contract_detractor1:
data = ''
elif ind == contract_detractor2:
data = ''
elif ind == contract_detractor3:
data = ''
elif ind == contract_detractor4:
data = ''
elif ind == contract_detractor5:
data = ''
elif ind == contract_detractor6:
data = ''
output_row.append(data)
writer.writerow(output_row)
s3core.upload_files(SPARKY_S3, DESTINATION_PATH, new_temp_file)
shutil.rmtree(temp_folder)
shutil.rmtree(new_temp_folder)

'\xef\xbb\xbf' is the UTF8 encoded version of the unicode ZERO WIDTH NO-BREAK SPACE U+FEFF. It is often used as a Byte Order Mark at the beginning of unicode text files:
when you have 3 bytes: '\xef\xbb\xbf', then the file is utf8 encoded
when you have 2 bytes: '\xff\xfe', then the file is in utf16 little endian
when you have 2 bytes: '\xfe\xff', then the file is in utf16 big endian
The 'utf-8-sig' encoding explicitely asks for writing this BOM at the beginning of the file
To process it automatically at read time of a csv file in Python 2, you can use the codecs module:
with open(file_path, 'r') as infile:
reader = csv.reader(codecs.EncodedFile(infile, 'utf-8', 'utf-8-sig'), delimiter=";")
EncodedFile will wrap the original file object by decoding it in utf8-sig, actually skipping the BOM and re-encoding it in utf8 with no BOM.

You want to use the EncodedFile method from the codecs library as in Serge Ballesta's answer.
However using Python 2.7 the encoding utf-8-sig is not a supported alias for the UTF8-sig encoding, you need to use utf_8_sig. Additionally the order of the method properties needs to define the output data encoding first, and the file encoding second: codecs.EncodedFile(file,datacodec,filecodec=None,errors=’strict')
Here's the full result:
import codecs
with open(file_path, 'r') as infile:
reader = csv.reader(codecs.EncodedFile(infile, 'utf8', 'utf_8_sig'), delimiter=";")

How to decrypt a RC2 ciphertext?

Python 3.5, pycrypto 2.7a1, Windows, RC2 ciphering
Example:
print('Введите текс, который хотите зашифровать:')
text = input()
with open('plaintext.txt', 'w') as f:
f.write(text)
key = os.urandom(32)
with open('rc2key.bin', 'wb') as keyfile:
keyfile.write(key)
iv = Random.new().read(ARC2.block_size)
cipher = ARC2.new(key, ARC2.MODE_CFB, iv)
ciphertext = iv + cipher.encrypt(bytes(text, "utf-8"))
with open('iv.bin', 'wb') as f:
f.write(iv)
with open('ciphertext.bin', 'wb') as f:
f.write(ciphertext)
print(ciphertext.decode("cp1251"))
And I'd like to know how can I decrypt this text, I tried, but couldn't do it.
My try to decrypt:
os.system('cls')
print('Дешифруем значит')
with open('ciphertext.bin', 'rb') as f:
ciphertext = f.read()
with open('rc2key.bin', 'rb') as f:
key = f.read()
with open('iv.bin', 'rb') as f:
iv = f.read()
ciphertext = ciphertext.decode('cp1251')
iv = iv.decode('cp1251')
text = ciphertext.replace(iv, '')
text = cipher.decrypt(text)
with open('plaintext.txt', 'w') as f:
f.write(text)
print(text.decode("ascii"))
But I understood that I need cipher variable, and I can't save it to .txt or .bin file, so that why I'm asking for help.

The IV is a non-secret value and is commonly written in front of the ciphertext. Since, you've done that already, you don't need to write an additional IV file. RC2 has a block size of 64 bit, so the IV will always be 8 byte long.
with open('ciphertext.bin', 'rb') as f:
ciphertext = f.read()
with open('rc2key.bin', 'rb') as f:
key = f.read()
iv = ciphertext[:ARC2.block_size]
ciphertext = ciphertext[ARC2.block_size:]
cipher = ARC2.new(key, ARC2.MODE_CFB, iv)
text = cipher.decrypt(ciphertext).decode("utf-8")
with open('plaintext.txt', 'w') as f:
f.write(text)
print(text)
Other problems:
Don't simply decode binary data such as ciphertexts, keys or IV, because those are most likely not printable.
Don't re-use the same cipher object if you're doing something different. The decryption needs a freshly initialized ARC2 object.