1I am trying to read-in some old text document file using python.
This file written in 1995 and has a file extension of ".WPF"
I had tried
f = open('/Users/zachary/Downloads/2R.WPF', mode = 'r')
print(f.read())
If I open it up through libreoffice, it well appears.
Any hint how to process text in .WPF using python?
linke address:
WTO Dispute Settlment DS2 Panel Report
Someone had marked it as the duplicated under the notion that the file is just wrongly named in WPF, however, it looks it's not a .doc file since the textract.process returns the error "it's not .doc"
As can be determined from the very first bytes, that file is a WordPerfect 5.x file (where x is 0, 1, or possibly 2), a file format dating back to around 1989.
According to its description, the Tika interface for Python should be able to convert this for you, but as far as word processor formats go, these older WordPerfect files are fairly easy to decode, without anything more than a plain Python installation.
The format consists of a large header (which, among other information, defines the printer that the document was formatted for, the list of fonts used, and some basic "style" information – I chose to skip it entirely in my program below), followed by plain text which is interspersed with binary codes which govern the formatting.
The binary codes appear in 3 distinct forms:
single-byte: 0x0A is a Return, 0xA9 is a breaking hyphen, 0xAA is a breaking hyphen when the line is broken at that position, and so on.
multi-byte, fixed length: the byte is followed by one or more specifications. For example, 0xC0 is a "special character code". It is followed by the character set index and the index of the actual character inside that set. The final byte of a fixed-length code is always the starting byte again.
multi-byte, variable length: the code determines a main category of formatting and is followed by a second to indicate a subcategory; after that, 2 bytes in little-endian indicate the length of the following data (excluding the first 4 bytes). This code always ends with the same items in reversed order: 2 bytes (little-endian) for the length, the subcategory, then the main category.
Codes between 0x00..0x1F and 0x7F..0xBF are single-byte control codes (not all are used). Codes between 0xC0..0xCF are fixed-length control codes, with various predefined lengths. Codes from 0xD0 onward are always variable-length codes.
With only this information, it's already possible to extract the plain text of this document, and just skip all possible formatting. Comparing the output codes against the PDF from the same site reveals the meaning of some of the codes, such as the various types of Return, the Tab, and plain text formatting such as bold and italics.
In addition, footnotes are stored in-line inside a variable-length code, so this needs some form of re-entrant parser.
The following Python 3 program is a basic framework which you can use as-is (it extracts the text, with a hint for the footnotes), or you can enable the commented-out lines at the bottom and find further information on parsing more of the formatting code.
# -*- coding: utf-8 -*-
import sys
WPType_None = 0
WPType_Text = 1
WPType_Byte = 2
WPType_Fixed = 3
WPType_Variable = 4
plain_remap = {10:'\n', 11:' ', 13:' ', 160:' ', 169:'-', 170:'-'}
WpCharacterSet = { 0x0121:'à', 0x0406:'§', 0x041c:u'’', 0x041d:u'‘', 0x041f:'”', 0x0420:'“', 0x0422:'—' }
textAttributes = [
"Extra Large",
"Very Large",
"Large",
"Small",
"Fine",
"Superscript",
"Subscript",
"Outline",
"Italic",
"Shadow",
"Redline",
"Double Underline",
"Bold",
"Strikeout",
"Underline",
"SmallCaps" ]
class WPElem:
def __init__(self, type=WPType_None, data = [], code=None):
self.type = type
self.code = code
if type == WPType_Text:
self.data = data
else:
self.data = data
class WordPerfect:
def __init__(self, filename):
with open(filename, "rb") as file:
self.data = bytearray(file.read())
sig = ''.join(chr(x) for x in self.data[1:4])
if self.data[0] != 255 or sig != 'WPC':
raise TypeError('Invalid file type')
self.data_start = self.data[4]+256*(self.data[5]+256*(self.data[6]+256*self.data[7]))
self.length = len(self.data)
self.elements = []
self.parse (self.data_start, self.length)
def parse (self, start,maxlength):
pos = start
while pos < maxlength:
byte = self.data[pos]
if byte in plain_remap:
byte = ord(plain_remap[byte])
if byte == 10 or byte >= 32 and byte <= 126:
if len(self.elements) == 0 or self.elements[-1].type != WPType_Text:
self.elements.append(WPElem(WPType_Text, ''))
self.elements[-1].data += chr(byte)
pos += 1
elif byte == 12:
self.elements.append(WPElem(WPType_Text, '\n\n'))
pos += 1
elif byte == 0x8c: # [HRt/Pg Break]
self.elements.append(WPElem(WPType_Text, '\n'))
pos += 1
elif byte == 0x8d: # [Ftn Num]
self.elements.append(WPElem(WPType_Text, '[Ftn Num]'))
pos += 1
elif byte == 0x99: # [HRt/Top of Pg]
self.elements.append(WPElem(WPType_Text, '\n'))
pos += 1
elif byte == 0xc0 and pos+3 < maxlength and self.data[pos+3] == 0xc0:
wpchar = self.data[pos+1]+256*self.data[pos+2]
if wpchar in WpCharacterSet:
self.elements.append(WPElem(WPType_Text, WpCharacterSet[wpchar]))
else:
self.elements.append(WPElem(WPType_Text, '{CHAR:%04X}' % wpchar))
pos += 4
elif byte == 0xc1 and self.data[pos+8] == 0xc1:
# self.elements.append(WPElem(WPType_Fixed, self.data[pos:pos+7]))
self.elements.append(WPElem(WPType_Text, '\t'))
pos += 9
elif byte == 0xc2 and self.data[pos+10] == 0xc2:
# self.elements.append(WPElem(WPType_Fixed, self.data[pos:pos+9]))
self.elements.append(WPElem(WPType_Text, '\t'))
pos += 11
elif byte == 0xc3:
self.elements.append(WPElem(WPType_Fixed, self.data[pos:pos+1], '%s On' % textAttributes[self.data[pos+1]]))
pos += 3
elif byte == 0xc4:
self.elements.append(WPElem(WPType_Fixed, self.data[pos:pos+1], '%s Off' % textAttributes[self.data[pos+1]]))
pos += 3
elif byte == 0xc6:
self.elements.append(WPElem(WPType_Fixed, self.data[pos:pos+5]))
pos += 6
elif byte == 0xd6 and self.data[pos+1] == 0: # Footnote
self.elements.append(WPElem(WPType_Text, '[Footnote:'))
length = self.data[pos+2]+256*self.data[pos+3]
self.parse (pos+0x13, pos+length)
pos += 4+length
self.elements.append(WPElem(WPType_Text, ']'))
else:
self.elements.append(WPElem(WPType_Byte, [byte]))
if byte >= 0xd0 and pos+4 <= maxlength:
length = self.data[pos+2]+256*self.data[pos+3]
if pos+4+length <= self.length:
if pos+4+length <= self.length and self.data[pos+4+length-1] == byte:
self.elements[-1].type = WPType_Variable
self.elements[-1].data += [x for x in self.data[pos+1:pos+length]]
pos += 4+length
else:
pos += 1
else:
pos += 1
else:
pos += 1
if len(sys.argv) != 2:
print("usage: read_wpf.py [suitably ancient WordPerfect file]")
sys.exit(1)
wpdata = WordPerfect (sys.argv[1])
for i in wpdata.elements:
if i.type == WPType_Text:
print (i.data, end='')
'''
elif i.code:
print ('[%s]' % i.code, end='')
elif i.type == WPType_Variable:
print ('[%02X:%d]' % (i.data[0],i.data[1]), end='')
else:
print ('[%02X]' % i.data[0], end='')
'''
Running it prints out the text to the console:
$ python3 read_wpf.py 2R.WPF
RESTRICTED
World Trade WT/DS2/R
29 January 1996
Organization
(96-0326)
(.. several thousands of lines omitted for brevity ..)
8.2 The Panel recommends that the Dispute Settlement Body request the
United States to bring this part of the Gasoline Rule into conformity
with its obligations under the General Agreement.
and you can either rewrite the program to store this into a plain text file, or redirect via your console into a file.
I've only added a translation for the handful of special characters that appear in the sample file. For a full-featured version, you'd need to look up a 90s data sheet somewhere, and provide Unicode translations for each of the thousands of characters.
Similarly, I've only 'parsed' some of the special formatting codes, and to a very limited extend. If you need to be able to extract particular formatting – say, tab settings, margins, font sizes, et cetera –, you must locate a full specification of the file format and add specific parsing code for these functions.
Does Python have a built-in, simple way of encoding/decoding strings using a password?
Something like this:
>>> encode('John Doe', password = 'mypass')
'sjkl28cn2sx0'
>>> decode('sjkl28cn2sx0', password = 'mypass')
'John Doe'
So the string "John Doe" gets encrypted as 'sjkl28cn2sx0'. To get the original string, I would "unlock" that string with the key 'mypass', which is a password in my source code. I'd like this to be the way I can encrypt/decrypt a Word document with a password.
I would like to use these encrypted strings as URL parameters. My goal is obfuscation, not strong security; nothing mission critical is being encoded. I realize I could use a database table to store keys and values, but am trying to be minimalist.
Python has no built-in encryption schemes, no. You also should take encrypted data storage serious; trivial encryption schemes that one developer understands to be insecure and a toy scheme may well be mistaken for a secure scheme by a less experienced developer. If you encrypt, encrypt properly.
You don’t need to do much work to implement a proper encryption scheme however. First of all, don’t re-invent the cryptography wheel, use a trusted cryptography library to handle this for you. For Python 3, that trusted library is cryptography.
I also recommend that encryption and decryption applies to bytes; encode text messages to bytes first; stringvalue.encode() encodes to UTF8, easily reverted again using bytesvalue.decode().
Last but not least, when encrypting and decrypting, we talk about keys, not passwords. A key should not be human memorable, it is something you store in a secret location but machine readable, whereas a password often can be human-readable and memorised. You can derive a key from a password, with a little care.
But for a web application or process running in a cluster without human attention to keep running it, you want to use a key. Passwords are for when only an end-user needs access to the specific information. Even then, you usually secure the application with a password, then exchange encrypted information using a key, perhaps one attached to the user account.
Symmetric key encryption
Fernet – AES CBC + HMAC, strongly recommended
The cryptography library includes the Fernet recipe, a best-practices recipe for using cryptography. Fernet is an open standard,
with ready implementations in a wide range of programming languages and it packages AES CBC encryption for you with version information, a timestamp and an HMAC signature to prevent message tampering.
Fernet makes it very easy to encrypt and decrypt messages and keep you secure. It is the ideal method for encrypting data with a secret.
I recommend you use Fernet.generate_key() to generate a secure key. You can use a password too (next section), but a full 32-byte secret key (16 bytes to encrypt with, plus another 16 for the signature) is going to be more secure than most passwords you could think of.
The key that Fernet generates is a bytes object with URL- and file-safe base64 characters, so printable:
from cryptography.fernet import Fernet
key = Fernet.generate_key() # store in a secure location
# PRINTING FOR DEMO PURPOSES ONLY, don't do this in production code
print("Key:", key.decode())
To encrypt or decrypt messages, create a Fernet() instance with the given key, and call the Fernet.encrypt() or Fernet.decrypt(), both the plaintext message to encrypt and the encrypted token are bytes objects.
encrypt() and decrypt() functions would look like:
from cryptography.fernet import Fernet
def encrypt(message: bytes, key: bytes) -> bytes:
return Fernet(key).encrypt(message)
def decrypt(token: bytes, key: bytes) -> bytes:
return Fernet(key).decrypt(token)
Demo:
>>> key = Fernet.generate_key()
>>> print(key.decode())
GZWKEhHGNopxRdOHS4H4IyKhLQ8lwnyU7vRLrM3sebY=
>>> message = 'John Doe'
>>> token = encrypt(message.encode(), key)
>>> print(token)
'gAAAAABciT3pFbbSihD_HZBZ8kqfAj94UhknamBuirZWKivWOukgKQ03qE2mcuvpuwCSuZ-X_Xkud0uWQLZ5e-aOwLC0Ccnepg=='
>>> decrypt(token, key).decode()
'John Doe'
Fernet with password – key derived from password, weakens the security somewhat
You can use a password instead of a secret key, provided you use a strong key derivation method. You do then have to include the salt and the HMAC iteration count in the message, so the encrypted value is not Fernet-compatible anymore without first separating salt, count and Fernet token:
import secrets
from base64 import urlsafe_b64encode as b64e, urlsafe_b64decode as b64d
from cryptography.fernet import Fernet
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
backend = default_backend()
iterations = 100_000
def _derive_key(password: bytes, salt: bytes, iterations: int = iterations) -> bytes:
"""Derive a secret key from a given password and salt"""
kdf = PBKDF2HMAC(
algorithm=hashes.SHA256(), length=32, salt=salt,
iterations=iterations, backend=backend)
return b64e(kdf.derive(password))
def password_encrypt(message: bytes, password: str, iterations: int = iterations) -> bytes:
salt = secrets.token_bytes(16)
key = _derive_key(password.encode(), salt, iterations)
return b64e(
b'%b%b%b' % (
salt,
iterations.to_bytes(4, 'big'),
b64d(Fernet(key).encrypt(message)),
)
)
def password_decrypt(token: bytes, password: str) -> bytes:
decoded = b64d(token)
salt, iter, token = decoded[:16], decoded[16:20], b64e(decoded[20:])
iterations = int.from_bytes(iter, 'big')
key = _derive_key(password.encode(), salt, iterations)
return Fernet(key).decrypt(token)
Demo:
>>> message = 'John Doe'
>>> password = 'mypass'
>>> password_encrypt(message.encode(), password)
b'9Ljs-w8IRM3XT1NDBbSBuQABhqCAAAAAAFyJdhiCPXms2vQHO7o81xZJn5r8_PAtro8Qpw48kdKrq4vt-551BCUbcErb_GyYRz8SVsu8hxTXvvKOn9QdewRGDfwx'
>>> token = _
>>> password_decrypt(token, password).decode()
'John Doe'
Including the salt in the output makes it possible to use a random salt value, which in turn ensures the encrypted output is guaranteed to be fully random regardless of password reuse or message repetition. Including the iteration count ensures that you can adjust for CPU performance increases over time without losing the ability to decrypt older messages.
A password alone can be as safe as a Fernet 32-byte random key, provided you generate a properly random password from a similar size pool. 32 bytes gives you 256 ^ 32 number of keys, so if you use an alphabet of 74 characters (26 upper, 26 lower, 10 digits and 12 possible symbols), then your password should be at least math.ceil(math.log(256 ** 32, 74)) == 42 characters long. However, a well-selected larger number of HMAC iterations can mitigate the lack of entropy somewhat as this makes it much more expensive for an attacker to brute force their way in.
Just know that choosing a shorter but still reasonably secure password won’t cripple this scheme, it just reduces the number of possible values a brute-force attacker would have to search through; make sure to pick a strong enough password for your security requirements.
Alternatives
Obscuring
An alternative is not to encrypt. Don't be tempted to just use a low-security cipher, or a home-spun implementation of, say Vignere. There is no security in these approaches, but may give an inexperienced developer that is given the task to maintain your code in future the illusion of security, which is worse than no security at all.
If all you need is obscurity, just base64 the data; for URL-safe requirements, the base64.urlsafe_b64encode() function is fine. Don't use a password here, just encode and you are done. At most, add some compression (like zlib):
import zlib
from base64 import urlsafe_b64encode as b64e, urlsafe_b64decode as b64d
def obscure(data: bytes) -> bytes:
return b64e(zlib.compress(data, 9))
def unobscure(obscured: bytes) -> bytes:
return zlib.decompress(b64d(obscured))
This turns b'Hello world!' into b'eNrzSM3JyVcozy_KSVEEAB0JBF4='.
Integrity only
If all you need is a way to make sure that the data can be trusted to be unaltered after having been sent to an untrusted client and received back, then you want to sign the data, you can use the hmac library for this with SHA1 (still considered secure for HMAC signing) or better:
import hmac
import hashlib
def sign(data: bytes, key: bytes, algorithm=hashlib.sha256) -> bytes:
assert len(key) >= algorithm().digest_size, (
"Key must be at least as long as the digest size of the "
"hashing algorithm"
)
return hmac.new(key, data, algorithm).digest()
def verify(signature: bytes, data: bytes, key: bytes, algorithm=hashlib.sha256) -> bytes:
expected = sign(data, key, algorithm)
return hmac.compare_digest(expected, signature)
Use this to sign data, then attach the signature with the data and send that to the client. When you receive the data back, split data and signature and verify. I've set the default algorithm to SHA256, so you'll need a 32-byte key:
key = secrets.token_bytes(32)
You may want to look at the itsdangerous library, which packages this all up with serialisation and de-serialisation in various formats.
Using AES-GCM encryption to provide encryption and integrity
Fernet builds on AEC-CBC with a HMAC signature to ensure integrity of the encrypted data; a malicious attacker can't feed your system nonsense data to keep your service busy running in circles with bad input, because the ciphertext is signed.
The Galois / Counter mode block cipher produces ciphertext and a tag to serve the same purpose, so can be used to serve the same purposes. The downside is that unlike Fernet there is no easy-to-use one-size-fits-all recipe to reuse on other platforms. AES-GCM also doesn't use padding, so this encryption ciphertext matches the length of the input message (whereas Fernet / AES-CBC encrypts messages to blocks of fixed length, obscuring the message length somewhat).
AES256-GCM takes the usual 32 byte secret as a key:
key = secrets.token_bytes(32)
then use
import binascii, time
from base64 import urlsafe_b64encode as b64e, urlsafe_b64decode as b64d
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from cryptography.hazmat.backends import default_backend
from cryptography.exceptions import InvalidTag
backend = default_backend()
def aes_gcm_encrypt(message: bytes, key: bytes) -> bytes:
current_time = int(time.time()).to_bytes(8, 'big')
algorithm = algorithms.AES(key)
iv = secrets.token_bytes(algorithm.block_size // 8)
cipher = Cipher(algorithm, modes.GCM(iv), backend=backend)
encryptor = cipher.encryptor()
encryptor.authenticate_additional_data(current_time)
ciphertext = encryptor.update(message) + encryptor.finalize()
return b64e(current_time + iv + ciphertext + encryptor.tag)
def aes_gcm_decrypt(token: bytes, key: bytes, ttl=None) -> bytes:
algorithm = algorithms.AES(key)
try:
data = b64d(token)
except (TypeError, binascii.Error):
raise InvalidToken
timestamp, iv, tag = data[:8], data[8:algorithm.block_size // 8 + 8], data[-16:]
if ttl is not None:
current_time = int(time.time())
time_encrypted, = int.from_bytes(data[:8], 'big')
if time_encrypted + ttl < current_time or current_time + 60 < time_encrypted:
# too old or created well before our current time + 1 h to account for clock skew
raise InvalidToken
cipher = Cipher(algorithm, modes.GCM(iv, tag), backend=backend)
decryptor = cipher.decryptor()
decryptor.authenticate_additional_data(timestamp)
ciphertext = data[8 + len(iv):-16]
return decryptor.update(ciphertext) + decryptor.finalize()
I've included a timestamp to support the same time-to-live use-cases that Fernet supports.
Other approaches on this page, in Python 3
AES CFB - like CBC but without the need to pad
This is the approach that All Іѕ Vаиітy follows, albeit incorrectly. This is the cryptography version, but note that I include the IV in the ciphertext, it should not be stored as a global (reusing an IV weakens the security of the key, and storing it as a module global means it'll be re-generated the next Python invocation, rendering all ciphertext undecryptable):
import secrets
from base64 import urlsafe_b64encode as b64e, urlsafe_b64decode as b64d
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from cryptography.hazmat.backends import default_backend
backend = default_backend()
def aes_cfb_encrypt(message, key):
algorithm = algorithms.AES(key)
iv = secrets.token_bytes(algorithm.block_size // 8)
cipher = Cipher(algorithm, modes.CFB(iv), backend=backend)
encryptor = cipher.encryptor()
ciphertext = encryptor.update(message) + encryptor.finalize()
return b64e(iv + ciphertext)
def aes_cfb_decrypt(ciphertext, key):
iv_ciphertext = b64d(ciphertext)
algorithm = algorithms.AES(key)
size = algorithm.block_size // 8
iv, encrypted = iv_ciphertext[:size], iv_ciphertext[size:]
cipher = Cipher(algorithm, modes.CFB(iv), backend=backend)
decryptor = cipher.decryptor()
return decryptor.update(encrypted) + decryptor.finalize()
This lacks the added armoring of an HMAC signature and there is no timestamp; you’d have to add those yourself.
The above also illustrates how easy it is to combine basic cryptography building blocks incorrectly; All Іѕ Vаиітy‘s incorrect handling of the IV value can lead to a data breach or all encrypted messages being unreadable because the IV is lost. Using Fernet instead protects you from such mistakes.
AES ECB – not secure
If you previously implemented AES ECB encryption and need to still support this in Python 3, you can do so still with cryptography too. The same caveats apply, ECB is not secure enough for real-life applications. Re-implementing that answer for Python 3, adding automatic handling of padding:
from base64 import urlsafe_b64encode as b64e, urlsafe_b64decode as b64d
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from cryptography.hazmat.primitives import padding
from cryptography.hazmat.backends import default_backend
backend = default_backend()
def aes_ecb_encrypt(message, key):
cipher = Cipher(algorithms.AES(key), modes.ECB(), backend=backend)
encryptor = cipher.encryptor()
padder = padding.PKCS7(cipher.algorithm.block_size).padder()
padded = padder.update(msg_text.encode()) + padder.finalize()
return b64e(encryptor.update(padded) + encryptor.finalize())
def aes_ecb_decrypt(ciphertext, key):
cipher = Cipher(algorithms.AES(key), modes.ECB(), backend=backend)
decryptor = cipher.decryptor()
unpadder = padding.PKCS7(cipher.algorithm.block_size).unpadder()
padded = decryptor.update(b64d(ciphertext)) + decryptor.finalize()
return unpadder.update(padded) + unpadder.finalize()
Again, this lacks the HMAC signature, and you shouldn’t use ECB anyway. The above is there merely to illustrate that cryptography can handle the common cryptographic building blocks, even the ones you shouldn’t actually use.
Assuming you are only looking for simple obfuscation that will obscure things from the very casual observer, and you aren't looking to use third party libraries. I'd recommend something like the Vigenere cipher. It is one of the strongest of the simple ancient ciphers.
Vigenère cipher
It's quick and easy to implement. Something like:
import base64
def encode(key, string):
encoded_chars = []
for i in xrange(len(string)):
key_c = key[i % len(key)]
encoded_c = chr(ord(string[i]) + ord(key_c) % 256)
encoded_chars.append(encoded_c)
encoded_string = "".join(encoded_chars)
return base64.urlsafe_b64encode(encoded_string)
Decode is pretty much the same, except you subtract the key.
It is much harder to break if the strings you are encoding are short, and/or if it is hard to guess the length of the passphrase used.
If you are looking for something cryptographic, PyCrypto is probably your best bet, though previous answers overlook some details: ECB mode in PyCrypto requires your message to be a multiple of 16 characters in length. So, you must pad. Also, if you want to use them as URL parameters, use base64.urlsafe_b64_encode(), rather than the standard one. This replaces a few of the characters in the base64 alphabet with URL-safe characters (as it's name suggests).
However, you should be ABSOLUTELY certain that this very thin layer of obfuscation suffices for your needs before using this. The Wikipedia article I linked to provides detailed instructions for breaking the cipher, so anyone with a moderate amount of determination could easily break it.
As you explicitly state that you want obscurity not security, we'll avoid reprimanding you for the weakness of what you suggest :)
So, using PyCrypto:
import base64
from Crypto.Cipher import AES
msg_text = b'test some plain text here'.rjust(32)
secret_key = b'1234567890123456'
cipher = AES.new(secret_key,AES.MODE_ECB) # never use ECB in strong systems obviously
encoded = base64.b64encode(cipher.encrypt(msg_text))
print(encoded)
decoded = cipher.decrypt(base64.b64decode(encoded))
print(decoded)
If someone gets a hold of your database and your code base, they will be able to decode the encrypted data. Keep your secret_key safe!
Here's a Python 3 version of the functions from #qneill 's answer:
import base64
def encode(key, clear):
enc = []
for i in range(len(clear)):
key_c = key[i % len(key)]
enc_c = chr((ord(clear[i]) + ord(key_c)) % 256)
enc.append(enc_c)
return base64.urlsafe_b64encode("".join(enc).encode()).decode()
def decode(key, enc):
dec = []
enc = base64.urlsafe_b64decode(enc).decode()
for i in range(len(enc)):
key_c = key[i % len(key)]
dec_c = chr((256 + ord(enc[i]) - ord(key_c)) % 256)
dec.append(dec_c)
return "".join(dec)
The extra encode/decodes are needed because Python 3 has split strings/byte arrays into two different concepts, and updated their APIs to reflect that..
The "encoded_c" mentioned in the #smehmood's Vigenere cipher answer should be "key_c".
Here are working encode/decode functions.
import base64
def encode(key, clear):
enc = []
for i in range(len(clear)):
key_c = key[i % len(key)]
enc_c = chr((ord(clear[i]) + ord(key_c)) % 256)
enc.append(enc_c)
return base64.urlsafe_b64encode("".join(enc))
def decode(key, enc):
dec = []
enc = base64.urlsafe_b64decode(enc)
for i in range(len(enc)):
key_c = key[i % len(key)]
dec_c = chr((256 + ord(enc[i]) - ord(key_c)) % 256)
dec.append(dec_c)
return "".join(dec)
Disclaimer: As implied by the comments, this should not be used to protect data in a real application, unless you read this and don't mind talking with lawyers:
What's wrong with XOR encryption?
Disclaimer: As mentioned in the comments, this should not be used to protect data in a real application.
What's wrong with XOR encryption?
https://crypto.stackexchange.com/questions/56281/breaking-a-xor-cipher-of-known-key-length
https://github.com/hellman/xortool
As has been mentioned the PyCrypto library contains a suite of ciphers. The XOR "cipher" can be used to do the dirty work if you don't want to do it yourself:
from Crypto.Cipher import XOR
import base64
def encrypt(key, plaintext):
cipher = XOR.new(key)
return base64.b64encode(cipher.encrypt(plaintext))
def decrypt(key, ciphertext):
cipher = XOR.new(key)
return cipher.decrypt(base64.b64decode(ciphertext))
The cipher works as follows without having to pad the plaintext:
>>> encrypt('notsosecretkey', 'Attack at dawn!')
'LxsAEgwYRQIGRRAKEhdP'
>>> decrypt('notsosecretkey', encrypt('notsosecretkey', 'Attack at dawn!'))
'Attack at dawn!'
Credit to https://stackoverflow.com/a/2490376/241294 for the base64 encode/decode functions (I'm a python newbie).
Here's an implementation of URL Safe encryption and Decryption using AES(PyCrypto) and base64.
import base64
from Crypto import Random
from Crypto.Cipher import AES
AKEY = b'mysixteenbytekey' # AES key must be either 16, 24, or 32 bytes long
iv = Random.new().read(AES.block_size)
def encode(message):
obj = AES.new(AKEY, AES.MODE_CFB, iv)
return base64.urlsafe_b64encode(obj.encrypt(message))
def decode(cipher):
obj2 = AES.new(AKEY, AES.MODE_CFB, iv)
return obj2.decrypt(base64.urlsafe_b64decode(cipher))
If you face some issue like this https://bugs.python.org/issue4329 (TypeError: character mapping must return integer, None or unicode) use str(cipher) while decoding as follows:
return obj2.decrypt(base64.urlsafe_b64decode(str(cipher)))
Test:
In [13]: encode(b"Hello World")
Out[13]: b'67jjg-8_RyaJ-28='
In [14]: %timeit encode("Hello World")
100000 loops, best of 3: 13.9 µs per loop
In [15]: decode(b'67jjg-8_RyaJ-28=')
Out[15]: b'Hello World'
In [16]: %timeit decode(b'67jjg-8_RyaJ-28=')
100000 loops, best of 3: 15.2 µs per loop
The library cryptocode provides a simple way to encode and decode strings with a password.
Here is how you install:
pip install cryptocode
Encrypting a message (example code):
import cryptocode
encoded = cryptocode.encrypt("mystring","mypassword")
## And then to decode it:
decoded = cryptocode.decrypt(encoded, "mypassword")
Documentation can be found here
Working encode/decode functions in python3 (adapted very little from qneill's answer):
def encode(key, clear):
enc = []
for i in range(len(clear)):
key_c = key[i % len(key)]
enc_c = (ord(clear[i]) + ord(key_c)) % 256
enc.append(enc_c)
return base64.urlsafe_b64encode(bytes(enc))
def decode(key, enc):
dec = []
enc = base64.urlsafe_b64decode(enc)
for i in range(len(enc)):
key_c = key[i % len(key)]
dec_c = chr((256 + enc[i] - ord(key_c)) % 256)
dec.append(dec_c)
return "".join(dec)
Thanks for some great answers. Nothing original to add, but here are some progressive rewrites of qneill's answer using some useful Python facilities. I hope you agree they simplify and clarify the code.
import base64
def qneill_encode(key, clear):
enc = []
for i in range(len(clear)):
key_c = key[i % len(key)]
enc_c = chr((ord(clear[i]) + ord(key_c)) % 256)
enc.append(enc_c)
return base64.urlsafe_b64encode("".join(enc))
def qneill_decode(key, enc):
dec = []
enc = base64.urlsafe_b64decode(enc)
for i in range(len(enc)):
key_c = key[i % len(key)]
dec_c = chr((256 + ord(enc[i]) - ord(key_c)) % 256)
dec.append(dec_c)
return "".join(dec)
enumerate()-- pair the items in a list with their index
iterate over the characters in a string
def encode_enumerate(key, clear):
enc = []
for i, ch in enumerate(clear):
key_c = key[i % len(key)]
enc_c = chr((ord(ch) + ord(key_c)) % 256)
enc.append(enc_c)
return base64.urlsafe_b64encode("".join(enc))
def decode_enumerate(key, enc):
dec = []
enc = base64.urlsafe_b64decode(enc)
for i, ch in enumerate(enc):
key_c = key[i % len(key)]
dec_c = chr((256 + ord(ch) - ord(key_c)) % 256)
dec.append(dec_c)
return "".join(dec)
build lists using a list comprehension
def encode_comprehension(key, clear):
enc = [chr((ord(clear_char) + ord(key[i % len(key)])) % 256)
for i, clear_char in enumerate(clear)]
return base64.urlsafe_b64encode("".join(enc))
def decode_comprehension(key, enc):
enc = base64.urlsafe_b64decode(enc)
dec = [chr((256 + ord(ch) - ord(key[i % len(key)])) % 256)
for i, ch in enumerate(enc)]
return "".join(dec)
Often in Python there's no need for list indexes at all. Eliminate loop index variables entirely using zip and cycle:
from itertools import cycle
def encode_zip_cycle(key, clear):
enc = [chr((ord(clear_char) + ord(key_char)) % 256)
for clear_char, key_char in zip(clear, cycle(key))]
return base64.urlsafe_b64encode("".join(enc))
def decode_zip_cycle(key, enc):
enc = base64.urlsafe_b64decode(enc)
dec = [chr((256 + ord(enc_char) - ord(key_char)) % 256)
for enc_char, key_char in zip(enc, cycle(key))]
return "".join(dec)
and some tests...
msg = 'The quick brown fox jumps over the lazy dog.'
key = 'jMG6JV3QdtRh3EhCHWUi'
print('cleartext: {0}'.format(msg))
print('ciphertext: {0}'.format(encode_zip_cycle(key, msg)))
encoders = [qneill_encode, encode_enumerate, encode_comprehension, encode_zip_cycle]
decoders = [qneill_decode, decode_enumerate, decode_comprehension, decode_zip_cycle]
# round-trip check for each pair of implementations
matched_pairs = zip(encoders, decoders)
assert all([decode(key, encode(key, msg)) == msg for encode, decode in matched_pairs])
print('Round-trips for encoder-decoder pairs: all tests passed')
# round-trip applying each kind of decode to each kind of encode to prove equivalent
from itertools import product
all_combinations = product(encoders, decoders)
assert all(decode(key, encode(key, msg)) == msg for encode, decode in all_combinations)
print('Each encoder and decoder can be swapped with any other: all tests passed')
>>> python crypt.py
cleartext: The quick brown fox jumps over the lazy dog.
ciphertext: vrWsVrvLnLTPlLTaorzWY67GzYnUwrSmvXaix8nmctybqoivqdHOic68rmQ=
Round-trips for encoder-decoder pairs: all tests passed
Each encoder and decoder can be swapped with any other: all tests passed
If you want to be safe, you can use Fernet, which is cryptographically sound. You can use a static "salt" if you don't want to store it separately - you will only lose dictionary and rainbow attack prevention. I chose it because I can pick long or short passwords´, which is not so easy with AES.
from cryptography.fernet import Fernet
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
import base64
#set password
password = "mysecretpassword"
#set message
message = "secretmessage"
kdf = PBKDF2HMAC(algorithm=hashes.SHA256(), length=32, salt="staticsalt", iterations=100000, backend=default_backend())
key = base64.urlsafe_b64encode(kdf.derive(password))
f = Fernet(key)
#encrypt
encrypted = f.encrypt(message)
print encrypted
#decrypt
decrypted = f.decrypt(encrypted)
print decrypted
If that's too complicated, someone suggested simplecrypt
from simplecrypt import encrypt, decrypt
ciphertext = encrypt('password', plaintext)
plaintext = decrypt('password', ciphertext)
I'll give 4 solutions:
1) Using Fernet encryption with cryptography library
Here is a solution using the package cryptography, that you can install as usual with pip install cryptography:
import base64
from cryptography.fernet import Fernet, InvalidToken
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
def cipherFernet(password):
key = PBKDF2HMAC(algorithm=hashes.SHA256(), length=32, salt=b'abcd', iterations=1000, backend=default_backend()).derive(password)
return Fernet(base64.urlsafe_b64encode(key))
def encrypt1(plaintext, password):
return cipherFernet(password).encrypt(plaintext)
def decrypt1(ciphertext, password):
return cipherFernet(password).decrypt(ciphertext)
# Example:
print(encrypt1(b'John Doe', b'mypass'))
# b'gAAAAABd53tHaISVxFO3MyUexUFBmE50DUV5AnIvc3LIgk5Qem1b3g_Y_hlI43DxH6CiK4YjYHCMNZ0V0ExdF10JvoDw8ejGjg=='
print(decrypt1(b'gAAAAABd53tHaISVxFO3MyUexUFBmE50DUV5AnIvc3LIgk5Qem1b3g_Y_hlI43DxH6CiK4YjYHCMNZ0V0ExdF10JvoDw8ejGjg==', b'mypass'))
# b'John Doe'
try: # test with a wrong password
print(decrypt1(b'gAAAAABd53tHaISVxFO3MyUexUFBmE50DUV5AnIvc3LIgk5Qem1b3g_Y_hlI43DxH6CiK4YjYHCMNZ0V0ExdF10JvoDw8ejGjg==', b'wrongpass'))
except InvalidToken:
print('Wrong password')
You can adapt with your own salt, iteration count, etc. This code is not very far from #HCLivess's answer but the goal is here to have ready-to-use encrypt and decrypt functions. Source: https://cryptography.io/en/latest/fernet/#using-passwords-with-fernet.
Note: use .encode() and .decode() everywhere if you want strings 'John Doe' instead of bytes like b'John Doe'.
2) Simple AES encryption with Crypto library
This works with Python 3:
import base64
from Crypto import Random
from Crypto.Hash import SHA256
from Crypto.Cipher import AES
def cipherAES(password, iv):
key = SHA256.new(password).digest()
return AES.new(key, AES.MODE_CFB, iv)
def encrypt2(plaintext, password):
iv = Random.new().read(AES.block_size)
return base64.b64encode(iv + cipherAES(password, iv).encrypt(plaintext))
def decrypt2(ciphertext, password):
d = base64.b64decode(ciphertext)
iv, ciphertext = d[:AES.block_size], d[AES.block_size:]
return cipherAES(password, iv).decrypt(ciphertext)
# Example:
print(encrypt2(b'John Doe', b'mypass'))
print(decrypt2(b'B/2dGPZTD8V22cIVKfp2gD2tTJG/UfP/', b'mypass'))
print(decrypt2(b'B/2dGPZTD8V22cIVKfp2gD2tTJG/UfP/', b'wrongpass')) # wrong password: no error, but garbled output
Note: you can remove base64.b64encode and .b64decode if you don't want text-readable output and/or if you want to save the ciphertext to disk as a binary file anyway.
3) AES using a better password key derivation function and the ability to test if "wrong password entered", with Crypto library
The solution 2) with AES "CFB mode" is ok, but has two drawbacks: the fact that SHA256(password) can be easily bruteforced with a lookup table, and that there is no way to test if a wrong password has been entered. This is solved here by the use of AES in "GCM mode", as discussed in AES: how to detect that a bad password has been entered? and Is this method to say “The password you entered is wrong” secure?:
import Crypto.Random, Crypto.Protocol.KDF, Crypto.Cipher.AES
def cipherAES_GCM(pwd, nonce):
key = Crypto.Protocol.KDF.PBKDF2(pwd, nonce, count=100000)
return Crypto.Cipher.AES.new(key, Crypto.Cipher.AES.MODE_GCM, nonce=nonce, mac_len=16)
def encrypt3(plaintext, password):
nonce = Crypto.Random.new().read(16)
return nonce + b''.join(cipherAES_GCM(password, nonce).encrypt_and_digest(plaintext)) # you case base64.b64encode it if needed
def decrypt3(ciphertext, password):
nonce, ciphertext, tag = ciphertext[:16], ciphertext[16:len(ciphertext)-16], ciphertext[-16:]
return cipherAES_GCM(password, nonce).decrypt_and_verify(ciphertext, tag)
# Example:
print(encrypt3(b'John Doe', b'mypass'))
print(decrypt3(b'\xbaN_\x90R\xdf\xa9\xc7\xd6\x16/\xbb!\xf5Q\xa9]\xe5\xa5\xaf\x81\xc3\n2e/("I\xb4\xab5\xa6ezu\x8c%\xa50', b'mypass'))
try:
print(decrypt3(b'\xbaN_\x90R\xdf\xa9\xc7\xd6\x16/\xbb!\xf5Q\xa9]\xe5\xa5\xaf\x81\xc3\n2e/("I\xb4\xab5\xa6ezu\x8c%\xa50', b'wrongpass'))
except ValueError:
print("Wrong password")
4) Using RC4 (no library needed)
Adapted from https://github.com/bozhu/RC4-Python/blob/master/rc4.py.
def PRGA(S):
i = 0
j = 0
while True:
i = (i + 1) % 256
j = (j + S[i]) % 256
S[i], S[j] = S[j], S[i]
yield S[(S[i] + S[j]) % 256]
def encryptRC4(plaintext, key, hexformat=False):
key, plaintext = bytearray(key), bytearray(plaintext) # necessary for py2, not for py3
S = list(range(256))
j = 0
for i in range(256):
j = (j + S[i] + key[i % len(key)]) % 256
S[i], S[j] = S[j], S[i]
keystream = PRGA(S)
return b''.join(b"%02X" % (c ^ next(keystream)) for c in plaintext) if hexformat else bytearray(c ^ next(keystream) for c in plaintext)
print(encryptRC4(b'John Doe', b'mypass')) # b'\x88\xaf\xc1\x04\x8b\x98\x18\x9a'
print(encryptRC4(b'\x88\xaf\xc1\x04\x8b\x98\x18\x9a', b'mypass')) # b'John Doe'
(Outdated since the latest edits, but kept for future reference): I had problems using Windows + Python 3.6 + all the answers involving pycrypto (not able to pip install pycrypto on Windows) or pycryptodome (the answers here with from Crypto.Cipher import XOR failed because XOR is not supported by this pycrypto fork ; and the solutions using ... AES failed too with TypeError: Object type <class 'str'> cannot be passed to C code). Also, the library simple-crypt has pycrypto as dependency, so it's not an option.
This works but password length should be exactly 8. This is simple and requires pyDes.
from pyDes import *
def encode(data,password):
k = des(password, CBC, "\0\0\0\0\0\0\0\0", pad=None, padmode=PAD_PKCS5)
d = k.encrypt(data)
return d
def decode(data,password):
k = des(password, CBC, "\0\0\0\0\0\0\0\0", pad=None, padmode=PAD_PKCS5)
d = k.decrypt(data)
return d
x = encode('John Doe', 'mypass12')
y = decode(x,'mypass12')
print x
print y
OUTPUT:
³.\Þ\åS¾+æÅ`;Ê
John Doe
You can use AES to encrypt your string with a password. Though, you'll want to chose a strong enough password so people can't easily guess what it is (sorry I can't help it. I'm a wannabe security weenie).
AES is strong with a good key size, but it's also easy to use with PyCrypto.
THIS ANSWER IS TERRIBLE FOR SECURITY. DO NOT USE FOR ANYTHING SENSITIVE
Whoever came here (and the bountier) seemed to be looking for one-liners with not much setup, which other answers don't provide. So I'm putting forward base64.
Now, keep in mind that this is basic obfuscation only, and is in NO WAY OK FOR SECURITY, but here are some one-liners:
from base64 import urlsafe_b64encode, urlsafe_b64decode
def encode(data, key): # the key DOES NOT make this safe
return urlsafe_b64encode(bytes(key+data, 'utf-8'))
def decode(enc, key):
return urlsafe_b64decode(enc)[len(key):].decode('utf-8')
print(encode('hi', 'there')) # b'dGhlcmVoaQ=='
print(decode(encode('hi', 'there'), 'there')) # 'hi'
A few things to note:
you will want to deal with more/less byte-to-string encoding/decoding on your own, depending on your I/O. Look into bytes() and bytes::decode()
base64 is easily recognizable by the types of characters used, and often ending with = characters. People like me absolutely go around decoding them in the javascript console when we see them on websites. It's as easy as btoa(string) (js)
the order is key+data, as in b64, what characters appear at the end depends on what characters are at the beginning (because of byte offsets. Wikipedia has some nice explanations). In this scenario, the beginning of the encoded string will be the same for everything encoded with that key. The plus is that the data will be more obfuscated. Doing it the other way around will result on the data part being exactly the same for everyone, regardless of key.
Now, if what you wanted didn't even need a key of any kind, but just some obfuscation, you can yet again just use base64, without any kinds of key:
from base64 import urlsafe_b64encode, urlsafe_b64decode
def encode(data):
return urlsafe_b64encode(bytes(data, 'utf-8'))
def decode(enc):
return urlsafe_b64decode(enc).decode()
print(encode('hi')) # b'aGk='
print(decode(encode('hi'))) # 'hi'
Adding one more code with decode and encode for reference
import base64
def encode(key, string):
encoded_chars = []
for i in range(len(string)):
key_c = key[i % len(key)]
encoded_c = chr(ord(string[i]) + ord(key_c) % 128)
encoded_chars.append(encoded_c)
encoded_string = "".join(encoded_chars)
arr2 = bytes(encoded_string, 'utf-8')
return base64.urlsafe_b64encode(arr2)
def decode(key, string):
encoded_chars = []
string = base64.urlsafe_b64decode(string)
string = string.decode('utf-8')
for i in range(len(string)):
key_c = key[i % len(key)]
encoded_c = chr(ord(string[i]) - ord(key_c) % 128)
encoded_chars.append(encoded_c)
encoded_string = "".join(encoded_chars)
return encoded_string
def main():
answer = str(input("EorD"))
if(answer in ['E']):
#ENCODE
file = open("D:\enc.txt")
line = file.read().replace("\n", " NEWLINEHERE ")
file.close()
text = encode("4114458",line)
fnew = open("D:\\new.txt","w+")
fnew.write(text.decode('utf-8'))
fnew.close()
else:
#DECODE
file = open("D:\\new.txt",'r+')
eline = file.read().replace("NEWLINEHERE","\n")
file.close()
print(eline)
eline = eline.encode('utf-8')
dtext=decode("4114458",eline)
print(dtext)
fnew = open("D:\\newde.txt","w+")
fnew.write(dtext)
fnew.close
if __name__ == '__main__':
main()
An other implementation of #qneill code which include CRC checksum of the original message, it throw an exception if the check fail:
import struct
import zlib
import base64
def vigenere_encode(text, key):
text = text.encode() + struct.pack('i', zlib.crc32(text.encode()))
enc = []
for i in range(len(text)):
key_c = key[i % len(key)]
enc_c = chr((text[i] + ord(key_c)) % 256)
enc.append(enc_c)
enc = ''.join(enc).encode()
enc = base64.urlsafe_b64encode(enc)
return enc.decode()
def vigenere_decode(encoded_text, key):
dec = []
encoded_text = base64.urlsafe_b64decode(encoded_text).decode()
for i in range(len(encoded_text)):
key_c = key[i % len(key)]
dec_c = chr((256 + ord(encoded_text[i]) - ord(key_c)) % 256)
dec.append(dec_c)
dec = "".join(dec)
checksum = dec[-4:]
dec = dec[:-4]
crc = struct.pack('i', zlib.crc32(dec.encode()))
assert [int(i) for i in crc] == [ord(i) for i in checksum], 'Decode Checksum Error'
return dec
External libraries provide secret-key encryption algorithms.
For example, the Cypher module in PyCrypto offers a selection of many encryption algorithms:
Crypto.Cipher.AES
Crypto.Cipher.ARC2
Crypto.Cipher.ARC4
Crypto.Cipher.Blowfish
Crypto.Cipher.CAST
Crypto.Cipher.DES
Crypto.Cipher.DES3
Crypto.Cipher.IDEA
Crypto.Cipher.RC5
Crypto.Cipher.XOR
MeTooCrypto is a Python wrapper for OpenSSL, and provides (among other functions) a full-strength general purpose cryptography library. Included are symmetric ciphers (like AES).
if you want secure encryption:
for python 2, you should use keyczar http://www.keyczar.org/
for python 3, until keyczar is available, i have written simple-crypt http://pypi.python.org/pypi/simple-crypt
both these will use key strengthening which makes them more secure than most other answers here. and since they're so easy to use you might want to use them even when security is not critical...
So, as nothing mission critical is being encoded, and you just want to encrypt for obsfuscation.
Let me present caeser's cipher
Caesar's cipher or Caesar shift, is one of the simplest and most widely known encryption techniques. It is a type of substitution cipher in which each letter in the plaintext is replaced by a letter some fixed number of positions down the alphabet. For example, with a left shift of 3, D would be replaced by A, E would become B, and so on.
Sample code for your reference :
def encrypt(text,s):
result = ""
# traverse text
for i in range(len(text)):
char = text[i]
# Encrypt uppercase characters
if (char.isupper()):
result += chr((ord(char) + s-65) % 26 + 65)
# Encrypt lowercase characters
else:
result += chr((ord(char) + s - 97) % 26 + 97)
return result
def decrypt(text,s):
result = ""
# traverse text
for i in range(len(text)):
char = text[i]
# Encrypt uppercase characters
if (char.isupper()):
result += chr((ord(char) - s-65) % 26 + 65)
# Encrypt lowercase characters
else:
result += chr((ord(char) - s - 97) % 26 + 97)
return result
#check the above function
text = "ATTACKATONCE"
s = 4
print("Text : " + text)
print("Shift : " + str(s))
print("Cipher: " + encrypt(text,s))
print("Original text: " + decrypt(encrypt(text,s),s))
Advantages : it meets your requirements and is simple and does the encoding thing'y'.
Disadvantage : can be cracked by simple brute force algorithms (highly unlikely anyone would attempt to go through all extra results).