Develop Reference

How do I store Credentials in Djnago [duplicate]

I'm working on a python/django app which, among other things, syncs data to a variety of other services, including samba shares, ssh(scp) servers, Google apps, and others. As such, it needs to store the credentials to access these services. Storing them as unencrypted fields would be, I presume, a Bad Idea, as an SQL injection attack could retrieve the credentials. So I would need to encrypt the creds before storage - are there any reliable libraries to achieve this?
Once the creds are encrypted, they would need to be decrypted before being usable. There are two use cases for my app:
One is interactive - in this case the user would provide the password to unlock the credentials.
The other is an automated sync - this is started by a cron job or similar. Where would I keep the password in order to minimise risk of exploits here?
Or is there a different approach to this problem I should be taking?

I have the same problem and have been researching this the past few days. The solution presented by #Rostislav is pretty good, but it's incomplete and a bit out dated.
On the Algorithm Layer
First, there's a new library for cryptography called, appropriately enough, Cryptography. There are a good number of reasons to use this library instead of PyCrypto, but the main ones that attracted me are:
A core goal is for you to be unable to shoot yourself in the foot. For example, it doesn't have severely outdated hash algos like MD2.
It has strong institutional support
500,000 tests with continuous integration on various platforms!
Their documentation website has a better SSL configuration (near-perfect A+ score instead of a mediocre B rating)
They have a disclosure policy for vulnerabilities.
You can read more about the reasons for creating the new library on LWN.
Second, the other answer recommends using SHA1 as the encryption key. SHA1 is dangerously weak and getting weaker. The replacement for SHA1 is SHA2, and on top of that, you should really being salting your hash and stretching it using either bcrypt or PBKDF2. Salting is important as a protection against rainbow tables and stretching is an important protection against brute forcing.
(Bcrypt is less tested, but is designed to use lots of memory and PBKDF2 is designed to be slow and is recommended by NIST. In my implementation, I use PBKDF2. If you want more on the differences, read this.)
For encryption AES in CBC mode with a 128-bit key should be used, as mentioned above – that hasn't changed, although it's now rolled up into a spec called Fernet. The initialization vector will be generated for you automatically in this library, so you can safely forget about that.
On the Key Generation and Storage Layer
The other answers are quite right to suggest that you need to carefully consider key handling and opt for something like OAuth, if you can. But assuming that's not possible (it isn't in my implementation), you have two use cases: Cron jobs and Interactive.
The cron job use case boils down to the fact that you need to keep a key somewhere safe and use it to run cron jobs. I haven't studied this, so I won't opine here. I think there are a lot of good ways to do this, but I don't know the easiest way.
For the Interactive use case, what you need to do is collect a user's password, use that to generate a key, and then use that key to decrypt the stored credentials.
Bringing it home
Here's how I would do all of the above, using the Cryptography library:
from cryptography.fernet import Fernet
from cryptography.hazmat.primitives.hashes import SHA256
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
from cryptography.hazmat.backends import default_backend
secret = "Some secret"
# Generate a salt for use in the PBKDF2 hash
salt = base64.b64encode(os.urandom(12)) # Recommended method from cryptography.io
# Set up the hashing algo
kdf = PBKDF2HMAC(
algorithm=SHA256(),
length=32,
salt=str(salt),
iterations=100000, # This stretches the hash against brute forcing
backend=default_backend(), # Typically this is OpenSSL
)
# Derive a binary hash and encode it with base 64 encoding
hashed_pwd = base64.b64encode(kdf.derive(user_pwd))
# Set up AES in CBC mode using the hash as the key
f = Fernet(hashed_pwd)
encrypted_secret = f.encrypt(secret)
# Store the safe inputs in the DB, but do NOT include a hash of the
# user's password, as that is the key to the encryption! Only store
# the salt, the algo and the number of iterations.
db.store(
user='some-user',
secret=encrypted_secret,
algo='pbkdf2_sha256',
iterations='100000',
salt=salt
)
Decryption then looks like:
# Get the data back from your database
encrypted_secret, algo, iterations, salt = db.get('some-user')
# Set up the Key Derivation Formula (PBKDF2)
kdf = PBKDF2HMAC(
algorithm=SHA256(),
length=32,
salt=str(salt),
iterations=int(iterations),
backend=default_backend(),
)
# Generate the key from the user's password
key = base64.b64encode(kdf.derive(user_pwd))
# Set up the AES encryption again, using the key
f = Fernet(key)
# Decrypt the secret!
secret = f.decrypt(encrypted_secret)
print(" Your secret is: %s" % secret)
Attacks?
Let's assume your DB is leaked to the Internet. What can an attacker do? Well, the key we used for encryption took the 100,000th SHA256 hash of your user's salted password. We stored the salt and our encryption algo in your database. An attacker must therefore either:
Attempt brute force of the hash: Combine the salt with every possible password and hash it 100,000 times. Take that hash and try it as the decryption key. The attacker will have to do 100,000 hashes just to try one password. This is basically impossible.
Try every possible hash directly as the decryption key. This is basically impossible.
Try a rainbow table with pre-computed hashes? Nope, not when random salts are involved.
I think this is pretty much solid.
There is, however, one other thing to think about. PBKDF2 is designed to be slow. It requires a lot of CPU time. This means that you are opening yourself up to DDOS attacks if there's a way for users to generate PBKDF2 hashes. Be prepared for this.
Postscript
All of this said, I think there are libraries that will do some of this for you. Google around for things like django encrypted field. I can't make any promises about those implementations, but perhaps you'll learn something about how others have done this.

First storing on a server credentials enough to login to a multitude of systems looks like a nightmare. Compromising code on your server will leak them all whatever the encryption.
You should store only the credentials that would be necessary to perform your task (i.e. files sync). For servers you should consider using synchronization server like RSync, for Google the protocols like OAuth etc. This way if your server is compromised this will only leak the data not the access to systems.
Next thing is encrypting these credentials. For cryptography I advise you to use PYCrypto.
For all random numbers you would use in your cryptography generate them by Crypto.Random (or some other strong method) to be sure they are strong enough.
You should not encrypt different credentials with the same key. The method I would recommend is this:
Your server should have it's master secret M (derived from /dev/random). Store it in the file owned by root and readable by root only.
When your server starts with root privileges it reads the file into memory and before serving clients drops it's privileges. That's normal practice for web servers and other demons.
When you are to write a new credential (or update existing one) generate a random block S. Take the first half and calculate hash K=H(S1,M). That would be your encryption key.
Use CBC mode to encrypt your data. Take the initialization vector (IV) from S2.
Store S alongside with encrypted data.
When you need to decrypt just take out S create the K and decrypt with the same IV.
For hash I would advise SHA1, for encryption — AES. Hashes and symmetric cyphers are fast enough so going for larger key sizes wouldn't hurt.
This scheme is a bit overshot in some places but again this wouldn't hurt.
But remember again, best way to store credentials is not to store credentials, and when you have to, use the least privileged ones that will allow you to accomplish the task.

Maybe you can rely on a multi-user scheme, by creating :
A user running Django (e.g. django) who does not have the permission to access the credentials
A user having those permissions (e.g. sync).
Both of them can be in the django group, to allow them to access the app. After that, make a script (a Django command, such as manage.py sync-external, for instance) that syncs what you want.
That way, the django user will have access to the app and the sync script, but not the credentials, because only the sync user does. If anyone tries to run that script without the credentials, it will of course result in an error.
Relying on Linux permission model is in my opinion a "Good Idea", but I'm not a security expert, so bear that in mind. If anyone has anything to say about what's above, don't hesitate!

Decrypting From OpenSSL PHP Library Using Python

I'm implementing an RSA encryption scheme where the MySQL database data is encrypted using a public key, and only the private key can be used to decrypt it.
I've managed to do encryption and decryption using the PHP openssl library, and even added a passphrase to protect the private key.
// $sRawText is the text string to encrypt.
// $sPublicKey is the public key stored on the server.
openssl_public_encrypt($sRawText, $sResult, $sPublicKey, OPENSSL_PKCS1_OAEP_PADDING);
// $sResult is the encrypted result which can then be stored in the database.
It works great for all the purposes I need to be publicly available, which is to add new entries if a user signs up or, in the future, verify details if they log in. This is sensitive data like contact information, which I only ever use very occasionally. Data that's verify only like passwords can be hashed still, and anything that needs to be read doesn't get encrypted.
Here's how I'm presently decrypting with PHP:
// $sPrivateKey is the matching private key.
// $sPassPhrase is the pass phrase (required to decrypt the result).
// $sRawBytes is the encrypted data from the database to decrypt.
$kRsaKey = openssl_pkey_get_private($sPrivateKey, $sPassPhrase);
openssl_private_decrypt($sRawBytes, $sResult, $kRsaKey, OPENSSL_PKCS1_OAEP_PADDING);
// $sResult will be the decrypted data.
The problem with this decryption approach is that it's all taking place live on the server. Obviously, storing the private key on that server would defeat most of the security benefits, as anyone with enough access to the server could pretty easily take it. The best I can come up with using PHP on the server is to pass the private key and/or pass phrase at each time of decryption. However, that still has the key being live there, and also opens up a new attack avenue for the key and pass phrase to be intercepted in those requests.
What I'm trying to do is perform all the decryption offline. So an offline software will be provided the encrypted data in a text file, which it translates into a list of instructions based on the data and intended operation. The private key never leaves that isolated environment and the pass phrase is never stored between sessions.
I'm running into all kinds of hurdles setting up an offline PHP environment, so I thought it might be easier to try to do the decryption in Python. Everything should be standard, or so I thought.
I'm currently trying to use the Python RSA library here:
https://stuvel.eu/python-rsa-doc/reference.html#exceptions
The thing I can't figure out is how to decrypt using the passphrase-protected private key. Would anyone be able to post a really simple example, or is it fundamentally not compatible without major modifications to the Python RSA library? What's the easiest approach to be able to do this decryption offline?

Okay I'll post the answer now that I figured out.
(1) We need to get the cryptography library in Python.
On Windows, enter into the command prompt (not Python interpreter):
py -m pip install cryptography
If the above doesn't work, try replacing 'py' with 'python'.
(2) The private key needs to be in a text file "privatekey.txt".
(3) Use this code for decryption:
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import padding
#privatekey.txt must contain the private key.
#passphrase_bytes must be the passphrase, and it needs to be in bytes (ie b'My Pass Phrase')
with open("privatekey.txt", "rb") as key_file:
private_key = serialization.load_pem_private_key(
key_file.read(),
password=passphrase_bytes,
)
#rawtext_bytes must have the bytes of the ciphertext to decrypt
plaintext = private_key.decrypt(
rawtext_bytes,
padding.OAEP(
mgf=padding.MGF1(algorithm=hashes.SHA1()),
algorithm=hashes.SHA1(),
label=None
)
)
#plaintext will now contain the result

python rsa private-encrypt to comunicate with node.js

First things first,
SECURITY DISCLAIMER:
I'm fully aware that if I use private-encryption -> public-decryption the "encrypted" message is is readable for anyone who has access to the so called "public key".
In a normal use case of rsa this would be fatal!But in my case the "public key" is meant to be private too.
I'm trying to establish an encrypted connection where the full control stays on one party.
One of the partners is untrusted and if he would got compromised I want to be sure no one else can use the interface (to prevent privilege escalation).
As you can see in this use case encryption itself is in fact not the main reason of asymmetric "encrytion".
You actually get an encrypted data stream which can't be manipulated from a third party even if the key is leaked.
My actual communication should run between a node.js and a python environment.
The URSA node lib already has a feature for the private encryption.
Sadly pyCrypto doesn't support that for reasons mentioned above earlier.
I already tried to cheese the lib and used the private exponent to instantiate a public key object but nodejs can't decrypt the result(not a big surprise).
Does anyone know how to tackle this problem?
Is there another way to get python to do actual private key encryption?
Maybe there is another python lib capable of doing so i couldn't find?
Or maybe its possible to get the nodejs lib to decrypt my cheesie private key encryption?

I have found a working solution for me.
I simply hacked an extra method in the signing object of pyCrypto which does exactly what i want.
This works with python 2.7.9 and 3.4 as far as I tested it.
In the file "[...]/Crypto/Signature/PKCS1_v1_5.py" is the class "PKCS115_SigScheme"
I simply added following method:
def private_encrypt(self, msg):
modBits = Crypto.Util.number.size(self._key.n)
k = ceil_div(modBits,8) # Convert from bits to bytes
PS = bchr(0xFF) * (k - len(msg) - 3)
m = self._key.decrypt(b("\x00\x01") + PS + bchr(0x00) + msg)
S = bchr(0x00)*(k-len(m)) + m
return S
It's just a manipulated copy of the "sign" function.
Usage looks like:
from Crypto.Signature import PKCS1_v1_5
from Crypto.PublicKey import RSA
plain_message="Hi, I'm going to be encrypted with a private key."
key = RSA.generate(4096)
signer = PKCS1_v1_5.new(key)
bin_enc_msg = signer.private_encrypt(plain_message.encode())
The decryption after the base64 encoded transfer in nodejs with URSA publicDecrypt works fine.
It should be clear from the imports but I want to mention this nonetheless:
This actually encrypts with PKCS1.5 padded rsa.
Another solution would be to combine symmetric encryption with asymmetric signing to be able to verify it but that would require additional effort and complexity I don't want or need.

If your goal is to connect a python environment and Node.js with encryption and don't require the use of third party verification, are you required to use RSA Public key encryption? It really sounds like what you're trying to accomplish is Symmetric-key Encryption and you could use AES. The pyCrypto Library already supports this, and some googling provided this library for Node.js on git hub.

AES - Encryption with Crypto (node-js) / decryption with Pycrypto (python)

I'm writing this question + answer because I struggled a lot (maybe because of a lack of experience), got lost in many different ways of encrypting/decrypting things with node or python.
I thought maybe my case could help people in the future.
What I needed to do:
Get data from a form, encrypt them using Crypto (node-js)
Pass the encrypted data in Python and decrypt it using PyCrypto.
I chose to use the AES encryption.
Here is how I started (I'm not gonna go through everything I tried):
I followed the example at the end of this page
Which gave in my case:
(this might be a very bad mix between javascript and coffeescript)
crypto = require "crypto"
[...]
key = "mykeywhatever"
cipher = crypto.createCipher('aes192', key)
cipher.update('string i want to encode', 'binary', 'hex')
encoded_string = cipher.final('hex')
[...]
This worked pretty fine to encode my string.
Then I wrote my python script to decrypt this string, using the readme on PyCrypto's github's page:
from Crypto.Cipher import AES
[...]
my_string = data_coming_from_rabbitmq
obj = AES.new('mykeywhatever', AES.MODE_CBC)
obj.decrypt(ciphertext)
[...]
This obviously didn't work: in the readme there is an IV but since I didn't gave one in the node script, why would I give one in the python one?
After more googling, I learnt that node's Crypto uses OpenSSL, while PyCrypto apparently doesn't. So I looked into that and found those pages:
How can I decrypt something with PyCrypto that was encrypted using OpenSSL?
Is AES the same in libraries PyCrypto & Node.JS Crypto
and a lot more...
So things got complicated, no one is doing the same thing to decrypt data, I got lost, and asked for help.
The answer is what my coworker and I came up with (well, mostly my corworker).

So we started from the "How can i decrypt... OpenSSL" 's answer.
We needed to modify the encryption script which gave:
crypto = require "crypto"
[...]
var iv = new Buffer('asdfasdfasdfasdf')
var key = new Buffer('asdfasdfasdfasdfasdfasdfasdfasdf')
var cipher = crypto.createCipheriv('aes-256-cbc', key, iv);
cipher.update(new Buffer("mystring"));
var enc = cipher.final('base64');
[...]
iv needs to be 16bytes long, key is 32bytes. And we changed createCipher to createCipheriv.
Back to the python decryption script:
Process was simply reading PyCrypto's documentation, and compare with the code we started from.
Then we decided to just stick to the API, and start from scratch. And it gave:
from base64 import b64decode
from Crypto.Cipher import AES
[...]
iv = 'asdfasdfasdfasdf'
key = 'asdfasdfasdfasdfasdfasdfasdfasdf'
encoded = b64decode('my_encrypted_string')
dec = AES.new(key=key, mode=AES.MODE_CBC, IV=iv)
value = dec.decrypt(encoded)
And it was as simple as that...
Hope it'll help some of you!
Update:
As Perseids wrote in the comments of his answer, the IV has to be random and different for every message

The system you are building is probably insecure
Except for storage you basically never want to just encrypt your data, but also authenticate it. Authentication in this context means that a valid message can only be generated by someone who knows the key. A widely used authentication scheme is HMAC.
If you do not authenticate your messages anyone can feed data into your service. An attacker might not be able to fully control the outcome after decryption but he/she might still be very dangerous. For example, if you use CBC (which you do) and the most common paddings schemes (AES is a block cipher and can only encrypt 128bit Blocks of data) and an attacker can differentiate between a padding error and any other error then all your messages can be decrypted by an attacker. This is called a padding oracle attack and is far too common.
To protect from this class of attacks you can use an authenticated encryption scheme, for example the GCM blockcipher mode.
Also you have to protect against replay attacks. Consider a banking application and the data you are transmitting is a bank transfer order. Barring any TAN an attacker might record a previous transaction and replay this transaction to your service again and again thus transferring a multiple of the money the customer originally wanted to.
Is the form you are getting the data from transmitted over HTTPS? If not: Can the key be eavesdropped by an attacker? How does a user know he got the form from you and not anybody else (SSL/TLS is as much about authentication as it is about confidentiality).
Probably I have forgotten some other attack vectors simple CBC-encryption offers.
Alternatives
Probably the easiest way to protect against these attacks is to transmit the form data over HTTPS. SSL/TLS was designed to prevent all of the above attacks and the client and server side implementations had a long time to mature.

How to encrypt and send data via ECDH, with only a publickey (peerkey) provided?

I'm coding a program that needs asymmetric cryptography. My approach is using Python's M2Crypto and I have chosen ECDH as the algorithm. However, I also want the procedure to be as simple as RSA, which requires only the public key of our receiver. ECDH is not. Its public key seems to have to be combined with a sender's own public key first, thus shared secret calculated and everything done.
Regardless of efficiency, I do want such a solution, so that if anyone wants to send me a message, my EC public key is published and the sender has to:
get his/her own EC private key
combine this private key with my public one and calculate a shared secret
use the secret and AES or other symmetric cipher encrypt the message
send me ciphertext and the derived public key(from this private key).
But here is a problem. I observed that a successful key exchange exists only when either side generate their keys basing on the same curve parameters. But unless I provide additional information of what curve my public key has based on, is there any other way of figuring this out? A public key is loaded with:
M2Crypto.EC.load_pub_key([filename])
And an EC.EC_pub instance is returned. I think information about the curve should hide somewhere in this instance, but I've not found where.
Please point out what I have missed, or if my solution was wrong or can be simpler, or any other suggestions. Thanks.