I need help to manage special characters in python. I'm working with a FPE library (pyffx) to encrypt data. Well, when my software encrypts the data there are no problems. When the software decrypts the data, it changes the format for example, by transforming a single \ into \\\\. I tried to encode/decode with utf-8 and also to maintain the raw value of the string. Here you can find an example of the encryption:
Encrypted data phase: uc?WL7+&u3*RAZuYPW1hc%/8sqX0?CBMlCCV-\mPC^RQ3&EsMRt|UIjs8KRLpzCbTI=3uBQcGWx6eXdVeV?nZmqCZtE/UAA\F7dbkuCTubyjng.%6AfhWh3H6V/b.ic7msw|lw-kiUuxCxCjPP%Uj.94h.*?ugB2j?TnujjL\2^aA2iYHGax?AQb7.VA76QTFcd7Ds2YL5OjI8Ue^Kmm9kC5t20M=mu8TDG4XK5xIifkUZMfWYYRa3£mbH/mGtAw\pN5nLOD2zEI/£82T7EKUa-v%*-l^GFS1-92t0N?EU&d7NEm&d|mbgl|fIKA-n|fEFV0OQ?OO^NmfDxiVsboMFho
Decription phase: b'uc?WL7+&u3*RAZuYPW1hc%/8sqX0?CBMlCCV-\\\\mPC^RQ3&EsMRt|UIjs8KRLpzCbTI=3uBQcGWx6eXdVeV?nZmqCZtE/UAA\\\\F7dbkuCTubyjng.%6AfhWh3H6V/b.ic7msw|lw-kiUuxCxCjPP%Uj.94h.*?ugB2j?TnujjL\\\\2^aA2iYHGax?AQb7.VA76QTFcd7Ds2YL5OjI8Ue^Kmm9kC5t20M=mu8TDG4XK5xIifkUZMfWYYRa3\\xa3mbH/mGtAw\\\\pN5nLOD2zEI/\\xa382T7EKUa-v%*-l^GFS1-92t0N?EU&d7NEm&d|mbgl|fIKA-n|fEFV0OQ?OO^NmfDxiVsboMFho'
In attachment you can find the example code:
def fpeencryption(data):
e = pyffx.String(b'secret-key', alphabet="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789.|/%&£+-*=?^\\", length=len(data))
print("Encription phase: " + str(data) + " size: " + str(len(data)))
encrypted = (e.encrypt(data))
print("Encrypted data phase: " + str((encrypted)) + " size: " + str(len(data)))
return (encrypted)
def fpedecryption(data):
e = pyffx.String(b'secret-key', alphabet="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789.|/%&£+-*=?^\\", length=len(data))
print("Decription phase: " + str((data)))
decrypted = e.decrypt(data.decode("utf-8"))
print("Decr: " + str(decrypted))
return decrypted
Data is a simple string with symbols/numbers/letter. Could you please help me? Thanks
SOLUTION
I solved the problem by changing the alphabet to:
alphabet=string.ascii_letters+string.digits+SPECIAL_CHARS
Related
i want some example for encryption between python and flutter to encrypt request and response body between client and server
i found some example code for AES CRT encryption , but i cant see same result in flutter and python
can anybody help me ?
UPDATE :
Flutter crypt package not have counter parameter but python Crypto.Cipher package have counter parameter
this is sample code for python:
plaintext = '123'.encode('utf-8')
key = '12345678911234567891123456789123'.encode("utf-8")
iv = '12345'.encode('utf')
iv_int = int(binascii.hexlify(iv), 16)
ctr = Counter.new(AES.block_size * 8, initial_value=iv_int)
aes = AES.new(key, AES.MODE_CTR, counter=ctr)
ciphertext = aes.encrypt(plaintext)
print('ctr = ' + str(ctr))
print('iv = ' + str(base64.b64encode(iv)))
print('iv_int = ' + str(iv_int))
print('plaintext = ' + str(plaintext))
print('key = ' + str(base64.b64encode(key)))
print('ciphertext = ' + str(base64.b64encode(ciphertext)))
this is sample code for flutter :
final plainText = '123';
final key = encrypt.Key.fromUtf8('12345678911234567891123456789123');
final iv = encrypt.IV.fromUtf8('12345');
final encrypter = encrypt.Encrypter(encrypt.AES(key, mode: encrypt.AESMode.ctr));
final encrypted = encrypter.encrypt(plainText, iv: iv);
final decrypted = encrypter.decrypt(encrypted, iv: iv);
print('key = ' + key.base64);
print('iv =' + iv.base64);
print('encrypted = ' + encrypted.base64);
I had the same issue and posted a very similar question. Fortunately, I found the error by myself.
Here is the link: Python AES CTR Mode only encrypts first two bytes
To summarize, the problem: Dart automatically uses the PKCS7 Padding when using AES Encryption, but Python does not anything like that.
So either you set the padding in your Dart code to null e.g.:
aes = AES.new(key, AES.MODE_CTR, counter=ctr, padding: null)
or you add a PKCS7 padding to your python code with like the following:
# The message has to in bytes format
def pkcs7padding(message):
pl = 16 - (len(message) % 16)
return message + bytearray([pl for i in range(pl)])
ciphertext = aes.encrypt(pkcs7padding(plaintext))
I am trying to use pyaes for AES encryption.
My below code is running perfectly.
text = 'Hello world !!!!'
encrypter = pyaes.AESModeOfOperationCBC('my_test_key_0001', 'my_test_vec_0001')
encrypted = base64.b64encode(encrypter.encrypt(text))
print(encrypted)
However, when I change the text value to
text = 'rO0ABXVyAAJbQqzzF/gGCFTgAgAAeHAAAAAI3z7LN2KbyKE='
It returns with error.
Traceback (most recent call last):
File "/home/path/cryptolib/test.py", line 54, in <module>
encrypted = base64.b64encode(encrypter.encrypt(text))
File "/home/path/pyaes/aes.py", line 389, in encrypt
raise ValueError('plaintext block must be 16 bytes')
ValueError: plaintext block must be 16 bytes
I am not expert on AES so maybe missing basics.
I can not use pycrypto because I am developing UDF for redshift and according to my findings pycrypto is not supported there.
pyaes#AESModeOfOperationCBC only allows the encryption of a text that is exactly one block (16 bytes) long. For longer texts BlockFeeder must be used:
import pyaes, base64
#Encryption
text = 'rO0ABXVyAAJbQqzzF/gGCFTgAgAAeHAAAAAI3z7LN2KbyKE='
encrypter = pyaes.Encrypter(pyaes.AESModeOfOperationCBC('my_test_key_0001', 'my_test_vec_0001'))
ciphertext = encrypter.feed(text)
ciphertext += encrypter.feed()
ciphertext = base64.b64encode(ciphertext)
#Decryption
decrypter = pyaes.Decrypter(pyaes.AESModeOfOperationCBC('my_test_key_0001', 'my_test_vec_0001'))
decrypted = decrypter.feed(base64.b64decode(ciphertext))
decrypted += decrypter.feed()
print('>' + decrypted + '<\n')
BlockFeeder also automatically perform the padding. Padding is the adding of data to the end of a message until the length corresponds to an integer multiple of the block length (generally important, but not relevant for your examples, since the length condition is already met).
EDIT:
Encrypter#feed(<plaindata>) buffers the plaintext, encrypts the data except for the last block (if the last block is complete) or the last two blocks (if the last block is incomplete), and returns the encrypted data. The final Encrypter#feed() call signals the end of the plaintext, triggers padding and encryption of the remainder, and returns the encrypted data. This can be
illustrated with the following code snippet:
import pyaes
#Encryption
encrypter = pyaes.Encrypter(pyaes.AESModeOfOperationCBC('my_test_key_0001', 'my_test_vec_0001'))
ciphertext = encrypter.feed ('0123456789ABCDEF') # 1. block buffered, ciphertext = ''
ciphertext += encrypter.feed('0123456789ABCDE') # 1. and incomplete 2. block buffered, ciphertext += ''
print("Current length of ciphertext: " + str(len(ciphertext)) + " Bytes\n")
ciphertext += encrypter.feed('F') # 1. block flushed, 2. block buffered, ciphertext += '<encrypted 1. block>'
print("Current length of ciphertext: " + str(len(ciphertext)) + " Bytes\n")
ciphertext += encrypter.feed('0123456789ABCDEF') # 2. block flushed, 3. block buffered, ciphertext += '<encrypted 2. block>'
print("Current length of ciphertext: " + str(len(ciphertext)) + " Bytes\n")
ciphertext += encrypter.feed('0123456') # 3. and incomplete 4. block buffered, ciphertext += ''
print("Current length of ciphertext: " + str(len(ciphertext)) + " Bytes\n")
ciphertext += encrypter.feed() # 3. and padded 4. block flushed, ciphertext += '<encrypted 3. and 4. padded block >'
print("Current length of ciphertext: " + str(len(ciphertext)) + " Bytes\n")
#Decryption
decrypter = pyaes.Decrypter(pyaes.AESModeOfOperationCBC('my_test_key_0001', 'my_test_vec_0001'))
decrypted = decrypter.feed(ciphertext)
decrypted += decrypter.feed()
print('>' + decrypted + '<\n')
In the example, the last block of plain text is incomplete. If the last block of the plain text is already complete, an additional complete block is padded. The padding used here is PKCS7.
I'm currently trying to implement a call to the Royal Mail API usingNodeJs and SOAP; I'm having difficulty recreating the security headers implementation following an example python script
The python script is as follows
#!/usr/local/bin/python2.7
import os
import sha
import binascii
import base64
password = 'test'
CREATIONDATE = '2016-03-29T14:03:46Z'
nonce = '7715776714'
HASH = sha.new(password).digest()
BASE64PASSWORD = base64.b64encode(HASH)
digest = sha.new(nonce + CREATIONDATE + HASH).digest()
PASSWORDDIGEST = base64.b64encode(digest)
ENCODEDNONCE = base64.b64encode(nonce)
print 'NONCE = ', nonce
print 'BASE64PASSWORD', BASE64PASSWORD
print 'PASSWORDDIGEST ', PASSWORDDIGEST
print 'ENCODEDNONCE ', ENCODEDNONCE
print 'CREATIONDATE ', CREATIONDATE
Which outputs:
NONCE 7715776714
BASE64PASSWORD qUqP5cyxm6YcTAhz05Hph5gvu9M=
PASSWORDDIGEST coDzcnSZObFfrM0FY33GcfxjOj4=
ENCODEDNONCE NzcxNTc3NjcxNA==
CREATIONDATE 2016-03-29T14:03:46Z
I've re-created this using NodeJs but I seem to get a differing output - inputting the correct password in the python version and using the resulting data allows me to make a valid call to the API, using the output from NodeJs gives me an authorisation failure
Node Js Code
var createdDate, password = 'test', nonce;
createdDate = '2016-03-29T14:03:46Z';
nonce = '7715776714';
var crypto = require("crypto"),
passHash = crypto.createHash('sha1'),
digestHash = crypto.createHash('sha1');
passHash.update(password);
var HASH = passHash.digest();
console.log('NONCE ' + nonce)
console.log('BASE64PASSWORD ' + base64_encode_string(HASH))
digestHash.update(nonce + createdDate + HASH);
var digest = digestHash.digest();
var PASSWORDDIGEST = base64_encode_string(digest);
console.log('PASSWORDDIGEST ' + PASSWORDDIGEST);
var ENCODEDNONCE = base64_encode_string(nonce.toString());
console.log('ENCODEDNONCE ' + ENCODEDNONCE);
console.log('CREATIONDATE ' + createdDate);
Which outputs
NONCE 7715776714
BASE64PASSWORD qUqP5cyxm6YcTAhz05Hph5gvu9M=
PASSWORDDIGEST FRMDpkDOi1j9KB/sDHg1b7BYQgA=
ENCODEDNONCE NzcxNTc3NjcxNA==
CREATIONDATE 2016-03-29T14:03:46Z
It appears that the Sha for the HASH is identical but the second Sha (digest) gives a differing result in the NodeJs version. Any pointers to where I'm going wrong?
For reference I'm using sha library in python and crypto in NodeJs
One problem is that you're implicitly converting the HASH Buffer to a UTF-8 string which is liable to cause corrupt output when converting binary to UTF-8. Instead, you can call .update() multiple times and preserve the binary data in HASH:
digestHash.update(nonce + createdDate);
digestHash.update(HASH);
var digest = digestHash.digest();
On an unrelated note, you don't need base64_encode_string() for Buffers, as for those you can simply do buffer.toString('base64') (e.g. HASH.toString('base64')).
I made a Python script to encrypt plaintext files using the symmetric-key algorithm described in this video. I then created a second script to decrypt the encrypted message. Here is the original text:
I came, I saw, I conquered.
Here is the text after being encrypted and decrypted:
I came, I saw, I conquerdd.
Almost perfect, except for a single letter. For longer texts, there will be multiple letters which are just off ie the numerical representation of the character which appears is one lower than the numerical representation of the original character. I have no idea why this is.
Here's how my scripts work. First, I generated a random sequence of digits -- my PAD -- and saved it in the text file "pad.txt". I won't show the code because it is so straightforward. I then saved the text which I want to be encrypted in "text.txt". Next, I run the encryption script, which encrypts the text and saves it in the file "encryptedText.txt":
#!/usr/bin/python3.4
import string
def getPad():
padString = open("pad.txt","r").read()
pad = padString.split(" ")
return pad
def encrypt(textToEncrypt,pad):
encryptedText = ""
possibleChars = string.printable[:98] # last two elements are not used bec
# ause they don't show up well on te
# xt files.
for i in range(len(textToEncrypt)):
char = textToEncrypt[i]
if char in possibleChars:
num = possibleChars.index(char)
else:
return False
encryptedNum = num + int(pad[(i)%len(pad)])
if encryptedNum >= len(possibleChars):
encryptedNum = encryptedNum - len(possibleChars)
encryptedChar = possibleChars[encryptedNum]
encryptedText = encryptedText + encryptedChar
return encryptedText
if __name__ == "__main__":
textToEncrypt = open("text.txt","r").read()
pad = getPad()
encryptedText = encrypt(textToEncrypt,pad)
if not encryptedText:
print("""An error occurred during the encryption process. Confirm that \
there are no forbidden symbols in your text.""")
else:
open("encryptedText.txt","w").write(encryptedText)
Finally, I decrypt the text with this script:
#!/usr/bin/python3.4
import string
def getPad():
padString = open("pad.txt","r").read()
pad = padString.split(" ")
return pad
def decrypt(textToDecrypt,pad):
trueText = ""
possibleChars = string.printable[:98]
for i in range(len(textToDecrypt)):
encryptedChar = textToDecrypt[i]
encryptedNum = possibleChars.index(encryptedChar)
trueNum = encryptedNum - int(pad[i%len(pad)])
if trueNum < 0:
trueNum = trueNum + len(possibleChars)
trueChar = possibleChars[trueNum]
trueText = trueText + trueChar
return trueText
if __name__ == "__main__":
pad = getPad()
textToDecrypt = open("encryptedText.txt","r").read()
trueText = decrypt(textToDecrypt,pad)
open("decryptedText.txt","w").write(trueText)
Both scripts seem very straightforward, and they obvious work almost perfectly. However, every once in a while there is an error and I cannot see why.
I found the solution to this problem. It turns out that every character that was not decrypted properly was encrypted to \r, which my text editor changed to a \n for whatever reason. Removing \r from the list of possible characters fixed the issue.
if data.find('PRIVMSG') != -1:
nick = data.split('!')[ 0 ].replace(':','')
text = ''
if data.count(text) >= 200:
sck.send('KICK ' + " " + chan + " :" 'flooding' + '\r\n')
I'm trying to code a flood protection for the bot, I want it to kick a user if he enters more then 200 characters, how can I make it so it can read the other lines instead of just the first line? and the code above doesn't work, it doesnt kick the user but if I change the sck.send() to sck.send('PRIVMSG ' + chan + " :" 'flooding' + '\r\n') it works.
fixed the kicking problem, and the code works now, but it only reads the first line, not sure how to make it read the other lines if the user keeps flooding the channel.
if data.find('PRIVMSG') != -1:
nick = data.split('!')[ 0 ].replace(':','')
text = ''
if data.count(text) >= 200:
sck.send('KICK ' + " " + chan + " " + nick + " :" 'flooding' + '\r\n')
As far as I remember, the colon is a reserved character in the IRC protocol. That is, the first colon in a server message denotes the start of user-supplied data (that's also why ":" is not allowed in nicks/channel names). Hence, it suffices to search for the first colon and calculate the length of the remaining string.
Furthermore, data.find('PRIVMSG') is pretty unreliable. What if a user types the word "PRIVMSG" in regular channel conversation? Go look up the IRC RFC, it specifies the format of PRIVMSGs in detail.
Besides, you should be a little more specific. What exactly is the problem you're facing? Extracting the nick? Calculating the message length? Connecting to IRC?