Develop Reference

MD5 Hash Cracker -- Unicode Objects must be encoded before hashing [duplicate]

I have this error:
Traceback (most recent call last):
File "python_md5_cracker.py", line 27, in <module>
m.update(line)
TypeError: Unicode-objects must be encoded before hashing
when I try to execute this code in Python 3.2.2:
import hashlib, sys
m = hashlib.md5()
hash = ""
hash_file = input("What is the file name in which the hash resides? ")
wordlist = input("What is your wordlist? (Enter the file name) ")
try:
hashdocument = open(hash_file, "r")
except IOError:
print("Invalid file.")
raw_input()
sys.exit()
else:
hash = hashdocument.readline()
hash = hash.replace("\n", "")
try:
wordlistfile = open(wordlist, "r")
except IOError:
print("Invalid file.")
raw_input()
sys.exit()
else:
pass
for line in wordlistfile:
# Flush the buffer (this caused a massive problem when placed
# at the beginning of the script, because the buffer kept getting
# overwritten, thus comparing incorrect hashes)
m = hashlib.md5()
line = line.replace("\n", "")
m.update(line)
word_hash = m.hexdigest()
if word_hash == hash:
print("Collision! The word corresponding to the given hash is", line)
input()
sys.exit()
print("The hash given does not correspond to any supplied word in the wordlist.")
input()
sys.exit()

It is probably looking for a character encoding from wordlistfile.
wordlistfile = open(wordlist,"r",encoding='utf-8')
Or, if you're working on a line-by-line basis:
line.encode('utf-8')
EDIT
Per the comment below and this answer.
My answer above assumes that the desired output is a str from the wordlist file. If you are comfortable in working in bytes, then you're better off using open(wordlist, "rb"). But it is important to remember that your hashfile should NOT use rb if you are comparing it to the output of hexdigest. hashlib.md5(value).hashdigest() outputs a str and that cannot be directly compared with a bytes object: 'abc' != b'abc'. (There's a lot more to this topic, but I don't have the time ATM).
It should also be noted that this line:
line.replace("\n", "")
Should probably be
line.strip()
That will work for both bytes and str's. But if you decide to simply convert to bytes, then you can change the line to:
line.replace(b"\n", b"")

You must have to define encoding format like utf-8,
Try this easy way,
This example generates a random number using the SHA256 algorithm:
>>> import hashlib
>>> hashlib.sha256(str(random.getrandbits(256)).encode('utf-8')).hexdigest()
'cd183a211ed2434eac4f31b317c573c50e6c24e3a28b82ddcb0bf8bedf387a9f'

import hashlib
string_to_hash = '123'
hash_object = hashlib.sha256(str(string_to_hash).encode('utf-8'))
print('Hash', hash_object.hexdigest())

The error already says what you have to do. MD5 operates on bytes, so you have to encode Unicode string into bytes, e.g. with line.encode('utf-8').

To store the password (PY3):
import hashlib, os
password_salt = os.urandom(32).hex()
password = '12345'
hash = hashlib.sha512()
hash.update(('%s%s' % (password_salt, password)).encode('utf-8'))
password_hash = hash.hexdigest()

encoding this line fixed it for me.
m.update(line.encode('utf-8'))

Please take a look first at that answer.
Now, the error message is clear: you can only use bytes, not Python strings (what used to be unicode in Python < 3), so you have to encode the strings with your preferred encoding: utf-32, utf-16, utf-8 or even one of the restricted 8-bit encodings (what some might call codepages).
The bytes in your wordlist file are being automatically decoded to Unicode by Python 3 as you read from the file. I suggest you do:
m.update(line.encode(wordlistfile.encoding))
so that the encoded data pushed to the md5 algorithm are encoded exactly like the underlying file.

You could open the file in binary mode:
import hashlib
with open(hash_file) as file:
control_hash = file.readline().rstrip("\n")
wordlistfile = open(wordlist, "rb")
# ...
for line in wordlistfile:
if hashlib.md5(line.rstrip(b'\n\r')).hexdigest() == control_hash:
# collision

If it's a single line string. wrapt it with b or B. e.g:
variable = b"This is a variable"
or
variable2 = B"This is also a variable"

This program is the bug free and enhanced version of the above MD5 cracker that reads the file containing list of hashed passwords and checks it against hashed word from the English dictionary word list. Hope it is helpful.
I downloaded the English dictionary from the following link
https://github.com/dwyl/english-words
# md5cracker.py
# English Dictionary https://github.com/dwyl/english-words
import hashlib, sys
hash_file = 'exercise\hashed.txt'
wordlist = 'data_sets\english_dictionary\words.txt'
try:
hashdocument = open(hash_file,'r')
except IOError:
print('Invalid file.')
sys.exit()
else:
count = 0
for hash in hashdocument:
hash = hash.rstrip('\n')
print(hash)
i = 0
with open(wordlist,'r') as wordlistfile:
for word in wordlistfile:
m = hashlib.md5()
word = word.rstrip('\n')
m.update(word.encode('utf-8'))
word_hash = m.hexdigest()
if word_hash==hash:
print('The word, hash combination is ' + word + ',' + hash)
count += 1
break
i += 1
print('Itiration is ' + str(i))
if count == 0:
print('The hash given does not correspond to any supplied word in the wordlist.')
else:
print('Total passwords identified is: ' + str(count))
sys.exit()

Python RSA Decrypt for class using vs2013

For my class assignment we need to decrypt a message that used RSA Encryption. We were given code that should help us with the decryption, but its not helping.
def block_decode(x):
output = ""
i = BLOCK_SIZE+1
while i > 0:
b1 = int(pow(95,i-1))
y = int(x/b1)
i = i - 1
x = x - y*b1
output = output + chr(y+32)
return output
I'm not great with python yet but it looks like it is doing something one character at a time. What really has me stuck is the data we were given. Can't figure out where or how to store it or if it is really decrypted data using RSA. below are just 3 lines of 38 lines some lines have ' or " or even multiple.
FWfk ?0oQ!#|eO Wgny 1>a^ 80*^!(l{4! 3lL qj'b!.9#'!/s2_
!BH+V YFKq _#:X &?A8 j_p< 7\[0 la.[ a%}b E`3# d3N? ;%FW
KyYM!"4Tz yuok J;b^!,V4) \JkT .E[i i-y* O~$? o*1u d3N?
How do I get this into a string list?

You are looking for the function ord which is a built-in function that
Returns the integer ordinal of a one-character string.
So for instance, you can do:
my_file = open("file_containing_encrypted_message")
data = my_file.read()
to read in the encrypted contents.
Then, you can iterate over each character doing
char_val = ord(each_character)
block_decode(char_val)

UnicodeEncodeError: 'ascii'

Sorry guys I'm really new at this.. Here is the full python script.
The purpose of the script is to read two different 1 wire temperature sensors and then use HTTP post to write those values into a mysql database.
#!/usr/bin/python
# -*- coding: utf-8 -*-
import requests
import hashlib
import time
#Dont forget to fill in PASSWORD and URL TO saveTemp (twice) in this file
sensorids = ["28-00000", "28-000004"]
avgtemperatures = []
for sensor in range(len(sensorids)):
temperatures = []
for polltime in range(0,3):
text = '';
while text.split("\n")[0].find("YES") == -1:
# Open the file that we viewed earlier so that python can see what is in it. Replace the serial number as before.
tfile = open("/sys/bus/w1/devices/"+ sensorids[sensor] +"/w1_slave")
# Read all of the text in the file.
text = tfile.read()
# Close the file now that the text has been read.
tfile.close()
time.sleep(1)
# Split the text with new lines (\n) and select the second line.
secondline = text.split("\n")[1]
# Split the line into words, referring to the spaces, and select the 10th word (counting from 0).
temperaturedata = secondline.split(" ")[9]
# The first two characters are "t=", so get rid of those and convert the temperature from a string to a number.
temperature = float(temperaturedata[2:])
# Put the decimal point in the right place and display it.
temperatures.append(temperature / 1000 * 9.0 / 5.0 + 32.0)
avgtemperatures.append(sum(temperatures) / float(len(temperatures)))
print avgtemperatures[0]
print avgtemperatures[1]
session = requests.Session()
# Getting a fresh nonce which we will use in the authentication step.
nonce = session.get(url='http://127.0.0.1/temp/saveTemp.php?step=nonce').text
# Hashing the nonce, the password and the temperature values (to provide some integrity).
response = hashlib.sha256('{}PASSWORD{}{}'.format(nonce.encode('utf8'), *avgtemperatures).hexdigest())
# Post data of the two temperature values and the authentication response.
post_data = {'response':response, 'temp1':avgtemperatures[0], 'temp2': avgtemperatures[1]}
post_request = session.post(url='http://127.0.0.1/temp/saveTemp.php', data=post_data)
if post_request.status_code == 200 :
print post_request.text
Below is the NEW error that I get.
Traceback (most recent call last):
File "/var/www/pollSensors.py", line 42, in <module>
response = hashlib.sha256('{}PASSWORD{}{}'.format(nonce.encode('utf8'), *avgtemperatures).hexdigest())
AttributeError: 'str' object has no attribute 'hexdigest'

nonce is a unicode value; session.get(..).text is always unicode.
You are trying to force that value into a string without explicitly providing an encoding. As a result Python is trying to encode it for you with the default ASCII codec. That encoding is failing.
Encode your Unicode values to strings explicitly instead. For a SHA 256 hash, UTF-8 is probably fine.
response = hashlib.sha256(nonce.encode('utf8') + 'PASSWORD' +
str(avgtemperatures[0]) +
str(avgtemperatures[1])).hexdigest()
or use string templating:
response = hashlib.sha256('{}PASSWORD{}{}'.format(
nonce.encode('utf8'), *avgtemperatures)).hexdigest()

I got the similar issue except that it was decimal instead of ascci
remove directory: your-profile.spyder2\spyder.lock

Convert PHP to Python

I need help converting the following PHP to python
$plaintext = "MyPassword";
$utf_text = mb_convert_encoding( $plaintext, 'UTF-16LE' );
$sha1_text = sha1( $utf_text, true );
$base64_text = base64_encode( $sha1_text );
echo $base64_text; //ouput = QEy4TXy9dNgleLq+IEcjsQDYm0A=
Convert the string to UTF16LE
Hash the output of 1. using SHA1
Encode the output of 2. using base64 encoding.
Im trying hashlib.sha1 but its not working. Maybe due to this, maybe encodings. Can anyone help

Your PHP code encodes the password to UTF16, little endian; if your password value is a unicode value, that works just fine in Python:
>>> import hashlib
>>> plaintext = u"MyPassword"
>>> utf_text = plaintext.encode('UTF-16LE')
>>> hashlib.sha1(utf_text).digest().encode('base64')
'QEy4TXy9dNgleLq+IEcjsQDYm0A=\n'
The above session was done in Python 2; in Python 3 the u prefix can be omitted as all string values are Unicode.

Got it, i was using the hashlib.sha1 incorrectly.
import hashlib
password = "MYPASSWORD"
result ='QEy4TXy9dNgleLq+IEcjsQDYm0A='
utftext = password.encode("utf-16LE")
m = hashlib.sha1()
m.update(utftext)
sha1text = m.digest()
output = sha1text.encode('base64','strict')
print "output: %s" % output
print "expect: %s" % result

I don't see why this wouldn't work. This works like a charm and prints out the desired result, QEy4TXy9dNgleLq+IEcjsQDYm0A=
from hashlib import sha1
from base64 import b64encode
print(b64encode(sha1("MyPassword".encode("utf-16le")).digest()))

appengine base64 encoded string is different from my local machine's

This is the code:
>>> import base64
>>> id = 1
>>> key = "secret key very long"
>>> enc = base64.urlsafe_b64encode(str(id)+key)
>>> enc
'MXNlY3JldCBrZXkgdmVyeSBsb25n'
>>> base64.urlsafe_b64decode(enc)
'1secret key very long'
Works as intended on my machine, but when I upload this code to google appengine, both encoded and decoded strings are totally different. How come?
EDIT 1:
this is the actual code:
import base64
id = 18005
key = "r-$b*8hglm+858&9t043hlm6-&6-3d3vfc4((7yd0dbrakhvi"
enc = base64.urlsafe_b64encode(str(id)+key)
print enc
# local machine: MTgwMDVyLSRiKjhoZ2xtKzg1OCY5dDA0M2hsbTYtJjYtM2QzdmZjNCgoN3lkMGRicmFraHZp
# appengine: PXItJGIqOGhnbG0rODU4Jjl0MDQzaGxtNi0mNi0zZDN2ZmM0KCg3eWQwZGJyYWtodmkxODAwNQ==

I can't explain why per se, but decoding the string you got from appengine shows it prepended an '=' to your key; and appended, rather than prepended, the ID.
>>> key='r-$b*8hglm+858&9t043hlm6-&6-3d3vfc4((7yd0dbrakhvi'
>>> base64.urlsafe_b64decode('PXItJGIqOGhnbG0rODU4Jjl0MDQzaGxtNi0mNi0zZDN2ZmM0KCg3eWQwZGJyYWtodmkxODAwNQ==')
'=r-$b*8hglm+858&9t043hlm6-&6-3d3vfc4((7yd0dbrakhvi18005'
>>> '=' + key + str(18005) == _
True
are you absolutely sure you used the same code on the server?