Develop Reference

How can I store a dictionary in a file which I can encrypt?

I'm pretty new to Python and programming in general and am currently working on a little password manager. Thus far I have a script which can encode a txt file using the cryptography library. I am now wondering if it is possible to store information on a website or an account with a corresponding Password in the txt file or if I need to use something else than a txt file. What would you recommend?

The easiest way to do that is convert that dict to json and encrypt the string result.
It's easy convert to dict too. What I advise you to do if you're using this only for auth, is save like a key value store. Example:
{
"<user>": "<password>",
"<user2>" : "<password2>"
}
On this datastruct the code will be always O(1).

Yes, you can use a text file. That is not a problem. The text file will contain encrypted text.
Depending upon how secure and how many unique passwords, you probably want a salted encryption techique.
I would recommend passlib.
I will give as an example some code I recently made, and explain it (although I am not a cryptography expert):
from passlib.context import CryptContext
CRYPTO_CTX = CryptContext(schemes=["bcrypt"], deprecated="auto") # This means only use `'bcrypt'`.
#app.post("/login")
def login_user(user, db):
db_user = crud.get_user_by_username(db, username=user.username)
if db_user is None:
raise HTTPException(
status_code=400, detail=f"No username {user.username} found."
)
if not CRYPTO_CTX.verify(user.password, db_user.hashed_password): ## This is the essential portion
raise HTTPException(status_code=400, detail=f"Incorrect password.")
db_user = crud.update_user(db, db_user, {"last_visit": datetime.now(timezone.utc)})
return {"user": db_user}
This code is, as I said, for a website, and it is using FastAPI. I have removed some stuff for clarity while hopefully keeping enough context. Here is the breakdown:
You need to make a "Cryptographic context", which will know which hashing scheme(s) to use.
In my case, I am only using and allowing 'bcrypt', a best practice choice for 2020 (but you can choose others for flexibility).
I then create a '/login' web route, which is a URI endpoint that triggers a function.
That function receives user information from an HTTP POST that an end user submitted (and a database session).
The user is then searched on the database backend. Presuming she is found, the submitted password in the POST is compared to the password in the database, but that password is cryptographically protected.
Let's zoom in on that comparison:
if not CRYPTO_CTX.verify(user.password, db_user.hashed_password):
We are providing a plain text string (user.password) and a hashed string (db_user.hashed_password) to our CRYPTO_CTXs verify method. That is all that is needed, since the passlib library is doing all the heavy lifting.
How was the password encrypted? That was just as easy:
hashed_password = CRYPTO_CTX.hash(user.password)
And this does not just encrypt the password, it also salts it. In short, this means that if somehow the encryption were hacked, it would only work for that one entry. With salting, it's a two-step encryption process:
Create a password. Eg, 'Elderberries'
Add a salt to it: 'Elderberries34(*&#arst##!'
Hash that entire thing: 'B4B6603ABC670967E99C7E7F1389E40CD16E78AD38EB1468EC2AA1E62B8BED3A'

Cracking a salted hash using a dictionary in python

im currently attempting a question where i have to crack a salted hash in python given the hash e77decd0e7c8a7b4688b010241bece45 and the salt "$goodluck$". I have tried downloading 10 million of the most popular passwords (https://github.com/danielmiessler/SecLists/blob/master/Passwords/Common-Credentials/10-million-password-list-top-1000000.txt), and using that as a dictionary. I then attach the salt to the word, encrypt it and then hash it. This is then compared to the given hash value. However i still have not been able to crack it. Here is my code:
import hashlib
from Crypto.Cipher import AES
from Crypto.Random import get_random_bytes
from Crypto.Hash import MD5
def dictionary_attack(password_hash):
dic = lines #extracted from file
pass_found = False
for word in dic:
word = word+"$goodluck$"
key = get_random_bytes(16)
cipher = AES.new(key, AES.MODE_EAX)
ciphertext, tag = cipher.encrypt_and_digest(word.encode('utf-8'))
hashed_val = MD5.new()
hashed_val.update(ciphertext)
hashed_val = hashed_val.hexdigest()
if hashed_val == password_hash:
pass_found = True
recovered_password = word
if pass_found:
print("Your password is: {}".format(recovered_password))
else:
print("Password not found")
dictionary_attack("e77decd0e7c8a7b4688b010241bece45")
Any help would be greatly appreciated. Thanks

Did you use this or this list?
Please try a reverse approach so see if you algorithm works or if there is something wrong:
pick a password
salt it
get the hash
generate some small list with that password and some others
feed that list and the hash to your function and see if it works
Ok, so that means that your code actually works and the password is not on the list. Instead of using that list, find the "The Top 500 Worst Passwords " and try them.
I don't know which library is faster, so just try it and get the duration for a reasonable amount of passwords to average out overhead, e.g. pick that many passwords that the duration is about one minute.

Zip Password functionality works wrong

I have a paradoxon, that I just cannot explain.
In short: I built a python script that is supposed to crack a zipped file, which is password protected.
This is what I've done:
(1) zip the text file:
zip --password bla zip3.zip myZip So the passphrase is "bla".
(2) Then I use the following Python Script:
import zipfile
import itertools
from itertools import *
import string
import time
That's the basic function, that is supposed to check, if a given password works or not:
def crack(File, pwd):
try:
File.extractall(pwd=str.encode(pwd))
print("\n---- SUCCESS! {0} ----".format(pwd))
except:
print("{0} did not work.".format(pwd))
pass
Here I specify, which characters I want to use for trying:
myLetters = string.ascii_letters
Here I specify, which zip-file I want to crack:
File = zipfile.ZipFile("PATH/TO/MY/zip3.zip", 'r')
Here I specify, how long the password-phrase is:
pwd_len = 3
here I specify, how many possible combinations of the charactes exist:
all_poss = (len(myLetters)**pwd_len)
Here is the procedure for concrete password cracking:
count = 0
start_time = time.time()
for i in range(0,pwd_len+1):
for j in map(''.join, itertools.product(myLetters, repeat=i)):
crack(File, j)
count += 1
print(round((count/all_poss)*100, 1), end='\r')
res_time = time.time() - start_time
print("\n--- {} ---".format(round(res_time,2)))
I use a nested loop, to try every password. If it works, I should get the Success-message. Else I should only see the "doesn't work message".
However...
If I type in my terminal: python3 pwdCracker.py >> out I get a long text file, which contains many many "does not work messages", BUT I also get a whole bunch of "Success-messages", although only ONE ("bla") should be correct.
Here is a little extract:
wN did not work.
---- SUCCESS! wO ----
wO did not work.
wP did not work.`
So apparently "wO" is working.. But why?? I set the password to "bla"! I really can open the file with "wO"... why can that happen??
Hope you can help!

The default zip encryption is known to be weak, and I think you are seeing hash collisions 1,2.
Most encryption methods (including those used in zip files) need a fixed length key, and so the password is hashed to give that key. The hash function used in zip is crc32 (specified here, although it details a different attack) which was designed for error checking rather than cryptographic hashing. Therefore it will be vulnerable to this type of attack.

The old zip format contains a check byte to quickly verify if your password is right or wrong. This check byte is verified against the last byte of the decrypted 'decryption header'.
Since the check byte is, well, only one byte, false positives happen quite frequently (1/256). What bruteforce crackers usually do in these cases is to check against multiple files from the same archive (hence using multiple check bytes).
From PkWare's APPNOTE.TXT:
After the header is decrypted, the last 1 or 2 bytes in Buffer SHOULD
be the high-order word/byte of the CRC for the file being decrypted,
stored in Intel low-byte/high-byte order. Versions of PKZIP prior to
2.0 used a 2 byte CRC check; a 1 byte CRC check is used on versions after 2.0.
This can be used to test if the password supplied is correct or not.
So what you are seeing is just that, false positives.

Python with etc/Shadow

so I'm writing this program that needs to check the password hash in etc/shadow and compare it to the password the user entered. I tried encrypting the password with hashlib.sha512, but the result was not the same. I think it's salted some how, but I don't know if it uses a universal salt or how I can get the salt each time.
tldr; I need a way for a user to enter a password, then have the program hash it and check it against the etc/shadow. Any ideas?

Try this https://pypi.python.org/pypi/pam . First link in google by python pam.
Look at distribution package manager for python-pam if exists. Else install with pip or easy_install.
Small example:
>>> import pam
>>> pam.authenticate('fred', 'fredspassword')
False

>>> import crypt
>>> line = 'bob:$1$qda8YAO9$rBiov9uVJlH1/97cbcyEt.:15965:0:99999:7:::'
>>> encript = line.split(':')[1]
>>> encript
--> '$1$qda8YAO9$rBiov9uVJlH1/97cbcyEt.'
>>> i = encript.rfind('$')
>>> salt = encript[:i]
>>> salt
--> '$1$qda8YAO9'
>>> crypt.crypt('bob_password',salt)
--> '$1$qda8YAO9$rBiov9uVJlH1/97cbcyEt.'
>>> encript
--> '$1$qda8YAO9$rBiov9uVJlH1/97cbcyEt.'

The passwd field is not just a SHA-512 hash of the password.*
This is explained in the crypt manpage. The format is $id$salt$hash, where id specifies the hash method (1 for MD5, 2a for Blowfish, 5 for SHA-256, 6 for SHA-512), salt specifies the salt to use with that algorithm, and hash specifies what the result should be.
As the manpage implies, you can actually pass the whole $id$salt$ to the crypt function in place of the salt, and it will automatically use the appropriate algorithm. This wouldn't be too hard to do via, say, ctypes.
At any rate, what you're doing is almost certainly a bad idea. You'll need to run as root in order to have access to /etc/shadow, and you'll need to simulate more than just password verification if you actually want to verify that the user can log in, and of course you'll need to handle secure input and make sure you don't end up saving the password in plaintext somewhere and so on. It's a lot simpler and safer to just let PAM do the work for you.
* I believe that in theory, it can be—if it doesn't start with a $ it's interpreted as some legacy format… presumably meaning it's interpreted as POSIX crypt using the DES algorithm.

Salt and hash a password in Python

This code is supposed to hash a password with a salt. The salt and hashed password are being saved in the database. The password itself is not.
Given the sensitive nature of the operation, I wanted to make sure everything was kosher.
import hashlib
import base64
import uuid
password = 'test_password'
salt = base64.urlsafe_b64encode(uuid.uuid4().bytes)
t_sha = hashlib.sha512()
t_sha.update(password+salt)
hashed_password = base64.urlsafe_b64encode(t_sha.digest())

Based on the other answers to this question, I've implemented a new approach using bcrypt.
Why use bcrypt
If I understand correctly, the argument to use bcrypt over SHA512 is that bcrypt is designed to be slow. bcrypt also has an option to adjust how slow you want it to be when generating the hashed password for the first time:
# The '12' is the number that dictates the 'slowness'
bcrypt.hashpw(password, bcrypt.gensalt( 12 ))
Slow is desirable because if a malicious party gets their hands on the table containing hashed passwords, then it is much more difficult to brute force them.
Implementation
def get_hashed_password(plain_text_password):
# Hash a password for the first time
# (Using bcrypt, the salt is saved into the hash itself)
return bcrypt.hashpw(plain_text_password, bcrypt.gensalt())
def check_password(plain_text_password, hashed_password):
# Check hashed password. Using bcrypt, the salt is saved into the hash itself
return bcrypt.checkpw(plain_text_password, hashed_password)
Notes
I was able to install the library pretty easily in a linux system using:
pip install py-bcrypt
However, I had more trouble installing it on my windows systems. It appears to need a patch. See this Stack Overflow question: py-bcrypt installing on win 7 64bit python

EDIT: This answer is wrong. A single iteration of SHA512 is fast, which makes it inappropriate for use as a password hashing function. Use one of the other answers here instead.
Looks fine by me. However, I'm pretty sure you don't actually need base64. You could just do this:
import hashlib, uuid
salt = uuid.uuid4().hex
hashed_password = hashlib.sha512(password + salt).hexdigest()
If it doesn't create difficulties, you can get slightly more efficient storage in your database by storing the salt and hashed password as raw bytes rather than hex strings. To do so, replace hex with bytes and hexdigest with digest.

Edit:
The library suggested in this answer is now outdated, and the hashlib key derivation functionality mentioned in this answer: https://stackoverflow.com/a/56915300/893857 is a good suggestion to use nowadays.
Original Answer
The smart thing is not to write the crypto yourself but to use something like passlib: https://passlib.readthedocs.io/en/stable/#
It is easy to mess up writing your crypto code in a secure way. The nasty thing is that with non crypto code you often immediately notice it when it is not working since your program crashes. While with crypto code you often only find out after it is to late and your data has been compromised. Therefore I think it is better to use a package written by someone else who is knowledgeable about the subject and which is based on battle tested protocols.
Also passlib has some nice features which make it easy to use and also easy to upgrade to a newer password hashing protocol if an old protocol turns out to be broken.
Also just a single round of sha512 is more vulnerable to dictionary attacks. sha512 is designed to be fast and this is actually a bad thing when trying to store passwords securely. Other people have thought long and hard about all this sort issues so you better take advantage of this.

As of Python 3.4, the hashlib module in the standard library contains key derivation functions which are "designed for secure password hashing".
So use one of those, like hashlib.pbkdf2_hmac, with a salt generated using os.urandom:
from typing import Tuple
import os
import hashlib
import hmac
def hash_new_password(password: str) -> Tuple[bytes, bytes]:
"""
Hash the provided password with a randomly-generated salt and return the
salt and hash to store in the database.
"""
salt = os.urandom(16)
pw_hash = hashlib.pbkdf2_hmac('sha256', password.encode(), salt, 100000)
return salt, pw_hash
def is_correct_password(salt: bytes, pw_hash: bytes, password: str) -> bool:
"""
Given a previously-stored salt and hash, and a password provided by a user
trying to log in, check whether the password is correct.
"""
return hmac.compare_digest(
pw_hash,
hashlib.pbkdf2_hmac('sha256', password.encode(), salt, 100000)
)
# Example usage:
salt, pw_hash = hash_new_password('correct horse battery staple')
assert is_correct_password(salt, pw_hash, 'correct horse battery staple')
assert not is_correct_password(salt, pw_hash, 'Tr0ub4dor&3')
assert not is_correct_password(salt, pw_hash, 'rosebud')
Note that:
The use of a 16-byte salt and 100000 iterations of PBKDF2 match the minimum numbers recommended in the Python docs. Further increasing the number of iterations will make your hashes slower to compute, and therefore more secure.
os.urandom always uses a cryptographically secure source of randomness
hmac.compare_digest, used in is_correct_password, is basically just the == operator for strings but without the ability to short-circuit, which makes it immune to timing attacks. That probably doesn't really provide any extra security value, but it doesn't hurt, either, so I've gone ahead and used it.
For theory on what makes a good password hash and a list of other functions appropriate for hashing passwords with, see https://security.stackexchange.com/q/211/29805.

For this to work in Python 3 you'll need to UTF-8 encode for example:
hashed_password = hashlib.sha512(password.encode('utf-8') + salt.encode('utf-8')).hexdigest()
Otherwise you'll get:
Traceback (most recent call last):
File "", line 1, in
hashed_password = hashlib.sha512(password + salt).hexdigest()
TypeError: Unicode-objects must be encoded before hashing

passlib seems to be useful if you need to use hashes stored by an existing system. If you have control of the format, use a modern hash like bcrypt or scrypt. At this time, bcrypt seems to be much easier to use from python.
passlib supports bcrypt, and it recommends installing py-bcrypt as a backend: http://pythonhosted.org/passlib/lib/passlib.hash.bcrypt.html
You could also use py-bcrypt directly if you don't want to install passlib. The readme has examples of basic use.
see also: How to use scrypt to generate hash for password and salt in Python

I don' want to resurrect an old thread, but... anyone who wants to use a modern up to date secure solution, use argon2.
https://pypi.python.org/pypi/argon2_cffi
It won the the password hashing competition. ( https://password-hashing.net/ ) It is easier to use than bcrypt, and it is more secure than bcrypt.

I did the same thing in NodeJs before:
echo "console.log(require('crypto').createHmac('sha256', 'salt').update('password').digest('hex'))" | node
it's equivalent in python is:
python3 -c 'import hashlib;import base64;import hmac;print(hmac.new(b"salt", "password".encode(), hashlib.sha256).hexdigest())'
And the equivalent shell command is:
echo -n "password" | openssl sha256 -hmac "salt"