I have this error:
Traceback (most recent call last):
File "python_md5_cracker.py", line 27, in <module>
m.update(line)
TypeError: Unicode-objects must be encoded before hashing
when I try to execute this code in Python 3.2.2:
import hashlib, sys
m = hashlib.md5()
hash = ""
hash_file = input("What is the file name in which the hash resides? ")
wordlist = input("What is your wordlist? (Enter the file name) ")
try:
hashdocument = open(hash_file, "r")
except IOError:
print("Invalid file.")
raw_input()
sys.exit()
else:
hash = hashdocument.readline()
hash = hash.replace("\n", "")
try:
wordlistfile = open(wordlist, "r")
except IOError:
print("Invalid file.")
raw_input()
sys.exit()
else:
pass
for line in wordlistfile:
# Flush the buffer (this caused a massive problem when placed
# at the beginning of the script, because the buffer kept getting
# overwritten, thus comparing incorrect hashes)
m = hashlib.md5()
line = line.replace("\n", "")
m.update(line)
word_hash = m.hexdigest()
if word_hash == hash:
print("Collision! The word corresponding to the given hash is", line)
input()
sys.exit()
print("The hash given does not correspond to any supplied word in the wordlist.")
input()
sys.exit()
It is probably looking for a character encoding from wordlistfile.
wordlistfile = open(wordlist,"r",encoding='utf-8')
Or, if you're working on a line-by-line basis:
line.encode('utf-8')
EDIT
Per the comment below and this answer.
My answer above assumes that the desired output is a str from the wordlist file. If you are comfortable in working in bytes, then you're better off using open(wordlist, "rb"). But it is important to remember that your hashfile should NOT use rb if you are comparing it to the output of hexdigest. hashlib.md5(value).hashdigest() outputs a str and that cannot be directly compared with a bytes object: 'abc' != b'abc'. (There's a lot more to this topic, but I don't have the time ATM).
It should also be noted that this line:
line.replace("\n", "")
Should probably be
line.strip()
That will work for both bytes and str's. But if you decide to simply convert to bytes, then you can change the line to:
line.replace(b"\n", b"")
You must have to define encoding format like utf-8,
Try this easy way,
This example generates a random number using the SHA256 algorithm:
>>> import hashlib
>>> hashlib.sha256(str(random.getrandbits(256)).encode('utf-8')).hexdigest()
'cd183a211ed2434eac4f31b317c573c50e6c24e3a28b82ddcb0bf8bedf387a9f'
import hashlib
string_to_hash = '123'
hash_object = hashlib.sha256(str(string_to_hash).encode('utf-8'))
print('Hash', hash_object.hexdigest())
The error already says what you have to do. MD5 operates on bytes, so you have to encode Unicode string into bytes, e.g. with line.encode('utf-8').
To store the password (PY3):
import hashlib, os
password_salt = os.urandom(32).hex()
password = '12345'
hash = hashlib.sha512()
hash.update(('%s%s' % (password_salt, password)).encode('utf-8'))
password_hash = hash.hexdigest()
encoding this line fixed it for me.
m.update(line.encode('utf-8'))
Please take a look first at that answer.
Now, the error message is clear: you can only use bytes, not Python strings (what used to be unicode in Python < 3), so you have to encode the strings with your preferred encoding: utf-32, utf-16, utf-8 or even one of the restricted 8-bit encodings (what some might call codepages).
The bytes in your wordlist file are being automatically decoded to Unicode by Python 3 as you read from the file. I suggest you do:
m.update(line.encode(wordlistfile.encoding))
so that the encoded data pushed to the md5 algorithm are encoded exactly like the underlying file.
You could open the file in binary mode:
import hashlib
with open(hash_file) as file:
control_hash = file.readline().rstrip("\n")
wordlistfile = open(wordlist, "rb")
# ...
for line in wordlistfile:
if hashlib.md5(line.rstrip(b'\n\r')).hexdigest() == control_hash:
# collision
If it's a single line string. wrapt it with b or B. e.g:
variable = b"This is a variable"
or
variable2 = B"This is also a variable"
This program is the bug free and enhanced version of the above MD5 cracker that reads the file containing list of hashed passwords and checks it against hashed word from the English dictionary word list. Hope it is helpful.
I downloaded the English dictionary from the following link
https://github.com/dwyl/english-words
# md5cracker.py
# English Dictionary https://github.com/dwyl/english-words
import hashlib, sys
hash_file = 'exercise\hashed.txt'
wordlist = 'data_sets\english_dictionary\words.txt'
try:
hashdocument = open(hash_file,'r')
except IOError:
print('Invalid file.')
sys.exit()
else:
count = 0
for hash in hashdocument:
hash = hash.rstrip('\n')
print(hash)
i = 0
with open(wordlist,'r') as wordlistfile:
for word in wordlistfile:
m = hashlib.md5()
word = word.rstrip('\n')
m.update(word.encode('utf-8'))
word_hash = m.hexdigest()
if word_hash==hash:
print('The word, hash combination is ' + word + ',' + hash)
count += 1
break
i += 1
print('Itiration is ' + str(i))
if count == 0:
print('The hash given does not correspond to any supplied word in the wordlist.')
else:
print('Total passwords identified is: ' + str(count))
sys.exit()
Related
I'm trying to encode a file and output the encode into a new file, but I got this error:
TypeError: ord() expected string of length 1, but int found
My code:
from sys import argv, exit
def encode(data):
encoded = ''
while data:
current = data[0]
count = 1
for i in data[1:]:
if i == current:
count += 1
else:
break
if count == 255:
break
encoded += '{}{}'.format(chr(ord(current) & 255), chr(count & 255)) #error occurs here.
data = data[count:]
return encoded
if __name__ == '__main__':
if len(argv) < 2:
print('Please specify input file!')
exit(0)
with open(argv[1], 'rb') as (f):
data = f.read()
with open(argv[1] + '.out', 'wb') as (f):
f.write(encode(data))
Additional question: How do I decode the encoded file?
You are reading bytes (open(..., 'rb')), so when you take one element of the byte string, you get a byte, ie. a number. This number already is the character code, so just leave out the ord. Alternatively, you could open the file without the b modifier (open(..., 'r')), which will return a string; I would advise to keep it as a byte string though (or you could run into encoding issues if you are parsing something non-ascii).
You will run into a similar problem saving your file: you cannot write a string into a file opened with the b modifier. Since you have characters outside the ascii range (>128), writing as a string is not a good idea, since python will try to encode your characters (eg. in UTF-8), and you will end up with completely different bytes. Therefore, the best solution probably is not to concat your data to a string in your loop (the part where you do '{}{}'.format(...), but to have a list (encoded = [], concat with encoded.append(current)) and convert that to a byte string using bytes(encoded) after your loop. You can then pass that to write without a problem.
As for how to decode your file, you can just open the file like you do for encoding, read two bytes b1 and b2, and append [b1]*b2 to your output (again, as a list), and convert that to a byte string with bytes().
I can't seem to get this program I'm supposed to do for a project to output the correct output, even though I have tried getting it to work multiple times. The project is:
Your program needs to decode an encrypted text file called "encrypted. txt". The person who wrote it used a cipher specified in "key. txt". This key file would look similar to the following:
A B
B C
C D
D E
E F
F G
G H
H I
I J
J K
K L
L M
M N
N O
O P
P Q
Q R
R S
S T
T U
U V
V W
W X
X Y
Y Z
Z A
The left column represents the plaintext letter, and the right column represents the corresponding ciphertext.
Your program should decode the "encrypted.txt" file using "key.txt" and write the plaintext to "decrypted.txt".
Your program should handle both upper and lower case letters in the encrypted without having two key files (or duplicating keys). You may have the decrypted text in all caps.
You should be able to handle characters in the encrypted text that are not in your key file. In that case, just have the decryption repeat the character. This will allow you to have spaces in your encrypted text that remain spaces when decrypted.
While you may write a program to create the key file - do NOT include that in the submission. You may manually create the encrypted and key text files. Use either the "new file" option in Python Shell (don't forget to save as txt) or an editor such as notepad. Do not use word.
Here is my code:
keyFile = open("key.txt", "r")
keylist1= []
keylist2 = []
for line in keyFile:
keylist1.append(line.split()[0])
keylist2.append(line.split()[1])
keyFile.close()
encryptedfile = open("encrypted.txt", "r")
lines = encryptedfile.readlines()
currentline = ""
decrypt = ""
for line in lines:
currentline = line
letter = list(currentline)
for i in range(len(letter)):
currentletter = letter[i]
if not letter[i].isalpha():
decrypt += letter[i]
else:
for o in range(len(keylist1)):
if currentletter == keylist1[o]:
decrypt += keylist2[o]
print(decrypt)
The only output I get is:
, ?
which is incorrect.
You forgot to handle lowercase letters. Use upper() to convert everything to a common case.
It would also be better to use a dictionary instead of a pair of lists.
mapping = {}
with open("key.txt", "r") as keyFile:
for line in keyFile:
l1, l2 = line.split()
mapping[upper(l1)] = upper(l2)
decrypt = ""
with open("encrypted.txt", "r") as encryptedFile:
for line in encryptedFile:
for char in line:
char = upper(char)
if char in mapping:
decrypt += mapping[char]
else:
decrypt += char
print(decrypt)
I've read a few postings regarding Python writing to text files but I could not find a solution to my problem. Here it is in a nutshell.
The requirement: to write values delimited by thorn characters (u00FE; and surronding the text values) and the pilcrow character (u00B6; after each column) to a UTF-16LE text file with BOM (FF FE).
The issue: The written-to text file has whitespace between each column that I did not script for. Also, it's not showing up right in UltraEdit. Only the first value ("mom") shows. I welcome any insight or advice.
The script (simplified to ease troubleshooting; the actual script uses a third-party API to obtain the list of values):
import os
import codecs
import shutil
import sys
import codecs
first = u''
textdel = u'\u00FE'.encode('utf_16_le') #thorn
fielddel = u'\u00B6'.encode('utf_16_le') #pilcrow
list1 = ['mom', 'dad', 'son']
num = len(list1) #pretend this is from the metadata profile
f = codecs.open('c:/myFile.txt', 'w', 'utf_16_le')
f.write(u'\uFEFF')
for item in list1:
mytext2 = u''
i = 0
i = i + 1
mytext2 = mytext2 + item + textdel
if i < (num - 1):
mytext2 = mytext2 + fielddel
f.write(mytext2 + u'\n')
f.close()
You're double-encoding your strings. You've already opened your file as UTF-16-LE, so leave your textdel and fielddel strings unencoded. They will get encoded at write time along with every line written to the file.
Or put another way, textdel = u'\u00FE' sets textdel to the "thorn" character, while textdel = u'\u00FE'.encode('utf-16-le') sets textdel to a particular serialized form of that character, a sequence of bytes according to that codec; it is no longer a sequence of characters:
textdel = u'\u00FE'
len(textdel) # -> 1
type(textdel) # -> unicode
len(textdel.encode('utf-16-le')) # -> 2
type(textdel.encode('utf-16-le')) # -> str
I want to determine which type of ROT encoding is used and based off that, do the correct decode.
Also, I have found the following code which will indeed decode rot13 "sbbone" to "foobart" correctly:
import codecs
codecs.decode('sbbone', 'rot_13')
The thing is I'd like to run this python file against an existing file which has rot13 encoding. (for example rot13.py encoded.txt).
Thank you!
To answer the second part of your first question, decode something in ROT-x, you can use the following code:
def encode(s, ROT_number=13):
"""Encodes a string (s) using ROT (ROT_number) encoding."""
ROT_number %= 26 # To avoid IndexErrors
alpha = "abcdefghijklmnopqrstuvwxyz" * 2
alpha += alpha.upper()
def get_i():
for i in range(26):
yield i # indexes of the lowercase letters
for i in range(53, 78):
yield i # indexes of the uppercase letters
ROT = {alpha[i]: alpha[i + ROT_number] for i in get_i()}
return "".join(ROT.get(i, i) for i in s)
def decode(s, ROT_number=13):
"""Decodes a string (s) using ROT (ROT_number) encoding."""
return encrypt(s, abs(ROT_number % 26 - 26))
To answer the first part of your first question, find the rot encoding of an arbitrarily encoded string, you probably want to brute-force. Uses all rot-encodings, and check which one makes the most sense. A quick(-ish) way to do this is to get a space-delimited (e.g. cat\ndog\nmouse\nsheep\nsay\nsaid\nquick\n... where \n is a newline) file containing most common words in the English language, and then check which encoding has the most words in it.
with open("words.txt") as f:
words = frozenset(f.read().lower().split("\n"))
# frozenset for speed
def get_most_likely_encoding(s, delimiter=" "):
alpha = "abcdefghijklmnopqrstuvwxyz" + delimiter
for punctuation in "\n\t,:; .()":
s.replace(punctuation, delimiter)
s = "".join(c for c in s if c.lower() in alpha)
word_count = [sum(w.lower() in words for w in encode(
s, enc).split(delimiter)) for enc in range(26)]
return word_count.index(max(word_count))
A file on Unix machines that you could use is /usr/dict/words, which can also be found here
Well, you can read the file line by line and decode it.
The output should go to an output file:
import codecs
import sys
def main(filename):
output_file = open('output_file.txt', 'w')
with open(filename) as f:
for line in f:
output_file.write(codecs.decode(line, 'rot_13'))
output_file.close()
if __name__ == "__main__":
_filename = sys.argv[1]
main(_filename)
Is there a good way to check if a string is encoded in base64 using Python?
I was looking for a solution to the same problem, then a very simple one just struck me in the head. All you need to do is decode, then re-encode. If the re-encoded string is equal to the encoded string, then it is base64 encoded.
Here is the code:
import base64
def isBase64(s):
try:
return base64.b64encode(base64.b64decode(s)) == s
except Exception:
return False
That's it!
Edit: Here's a version of the function that works with both the string and bytes objects in Python 3:
import base64
def isBase64(sb):
try:
if isinstance(sb, str):
# If there's any unicode here, an exception will be thrown and the function will return false
sb_bytes = bytes(sb, 'ascii')
elif isinstance(sb, bytes):
sb_bytes = sb
else:
raise ValueError("Argument must be string or bytes")
return base64.b64encode(base64.b64decode(sb_bytes)) == sb_bytes
except Exception:
return False
import base64
import binascii
try:
base64.decodestring("foo")
except binascii.Error:
print "no correct base64"
This isn't possible. The best you could do would be to verify that a string might be valid Base 64, although many strings consisting of only ASCII text can be decoded as if they were Base 64.
The solution I used is based on one of the prior answers, but uses more up to date calls.
In my code, the my_image_string is either the image data itself in raw form or it's a base64 string. If the decode fails, then I assume it's raw data.
Note the validate=True keyword argument to b64decode. This is required in order for the assert to be generated by the decoder. Without it there will be no complaints about an illegal string.
import base64, binascii
try:
image_data = base64.b64decode(my_image_string, validate=True)
except binascii.Error:
image_data = my_image_string
Using Python RegEx
import re
txt = "VGhpcyBpcyBlbmNvZGVkIHRleHQ="
x = re.search("^([A-Za-z0-9+/]{4})*([A-Za-z0-9+/]{3}=|[A-Za-z0-9+/]{2}==)?$", txt)
if (x):
print("Encoded")
else:
print("Non encoded")
Before trying to decode, I like to do a formatting check first as its the lightest weight check and does not return false positives thus following fail-fast coding principles.
Here is a utility function for this task:
RE_BASE64 = "^([A-Za-z0-9+/]{4})*([A-Za-z0-9+/]{3}=|[A-Za-z0-9+/]{2}==)?$"
def likeBase64(s:str) -> bool:
return False if s is None or not re.search(RE_BASE64, s) else True
if the length of the encoded string is the times of 4, it can be decoded
base64.encodestring("whatever you say").strip().__len__() % 4 == 0
so, you just need to check if the string can match something like above, then it won't throw any exception(I Guess =.=)
if len(the_base64string.strip()) % 4 == 0:
# then you can just decode it anyway
base64.decodestring(the_base64string)
#geoffspear is correct in that this is not 100% possible but you can get pretty close by checking the string header to see if it matches that of a base64 encoded string (re: How to check whether a string is base64 encoded or not).
# check if a string is base64 encoded.
def isBase64Encoded(s):
pattern = re.compile("^([A-Za-z0-9+/]{4})*([A-Za-z0-9+/]{4}|[A-Za-z0-9+/]{3}=|[A-Za-z0-9+/]{2}==)$")
if not s or len(s) < 1:
return False
else:
return pattern.match(s)
Also not that in my case I wanted to return false if the string is empty to avoid decoding as there's no use in decoding nothing.
I know I'm almost 8 years late but you can use a regex expression thus you can verify if a given input is BASE64.
import re
encoding_type = 'Encoding type: '
base64_encoding = 'Base64'
def is_base64():
element = input("Enter encoded element: ")
expression = "^([A-Za-z0-9+/]{4})*([A-Za-z0-9+/]{3}=|[A-Za-z0-9+/]{2}==)?$"
matches = re.match(expression, element)
if matches:
print(f"{encoding_type + base64_encoding}")
else:
print("Unknown encoding type.")
is_base64()
def is_base64(s):
s = ''.join([s.strip() for s in s.split("\n")])
try:
enc = base64.b64encode(base64.b64decode(s)).strip()
return enc == s
except TypeError:
return False
In my case, my input, s, had newlines which I had to strip before the comparison.
x = 'possibly base64 encoded string'
result = x
try:
decoded = x.decode('base64', 'strict')
if x == decoded.encode('base64').strip():
result = decoded
except:
pass
this code put in the result variable decoded string if x is really encoded, and just x if not. Just try to decode doesn't always work.