I'm trying to send username and password data from a web form to my server.
The password is sent as plain text over a https connection, then properly encrypted on the server (using python hashlib.sha224) before being stored, however I'm not sure how to transmit the password text to the server in an encrypted format.
My web client is written in javascript, and the server is written in python.
You'd have to encrypt it in the JavaScript. Unfortunately JavaScript does not offer any cryptographic tools out of the box, so you'd have to either use a third-party library or write something on your own.
As others have said, HTTPS is meant to encrypt the whole traffic, so perhaps you don't need anything extra? If you do however, you may want to look at this Article, which might shed some light on the problem. Good luck ! :)
Actually you transmit the password encrypted, because you use SSL.
Furthermore you do not encrypt the password, you hash the password on the server.
Anyway, you can use something like jCryption for it. crypt-js could also fit your purpose.
For python there is a Crypto Library called PyCrypto. But I have a Problem with the communication between Javascript and Python. I try to do something similar, but have a problem with it. I think my question will help you with yours.
Include nonce and block count in PyCrypto AES MODE_CTR
But in general, you have already solved the problem on your own, by using https.
https is an encrypted format. You're good.
If you want to do it clientside anyway I recommend hashing it with sha1. This guy seems to have some libs for that: http://pajhome.org.uk/crypt/md5/ - SHA1, sha256, md5, etc.
The HTTPS channel over which you send the password to the server provides encryption that is good enough.
However, you need a more secure storage mechanism for the password. Use an algorithm like "bcrypt" with many thousands of hash iterations (bcrypt calls this the cost factor, and it should be at least 16, meaning 216 iterations), and a random "salt". This works by deriving an encryption key from the password, which is a computationally expensive process, then using that key to encrypt some known cipher text, which is saved for comparison on future login attempts.
Also, using HTTPS on the login only is not sufficient. You should use it for any requests that require an authenticated user, or that carry an authentication cookie.
On the contrary to http, https is an encrypted protocol. You don't need additional encryption between the client and the server.
SHA224, SHA1 or MD5 are not a encryption, but a hashing function, which means they are irreversible.
Some answers suggest hashing passwords client-side.
However, irreversible doesn't mean uncrackable. Having plain password hashed, it is relatively easy to obtain the matching password from hash (see Rainbow tables for example).
Therefore you should not hash the password on the client side, but concatenate it with a some arbitrary string selected on the server side (usually called a salt) and hash the result.
Related
I'm working on a python/django app which, among other things, syncs data to a variety of other services, including samba shares, ssh(scp) servers, Google apps, and others. As such, it needs to store the credentials to access these services. Storing them as unencrypted fields would be, I presume, a Bad Idea, as an SQL injection attack could retrieve the credentials. So I would need to encrypt the creds before storage - are there any reliable libraries to achieve this?
Once the creds are encrypted, they would need to be decrypted before being usable. There are two use cases for my app:
One is interactive - in this case the user would provide the password to unlock the credentials.
The other is an automated sync - this is started by a cron job or similar. Where would I keep the password in order to minimise risk of exploits here?
Or is there a different approach to this problem I should be taking?
I have the same problem and have been researching this the past few days. The solution presented by #Rostislav is pretty good, but it's incomplete and a bit out dated.
On the Algorithm Layer
First, there's a new library for cryptography called, appropriately enough, Cryptography. There are a good number of reasons to use this library instead of PyCrypto, but the main ones that attracted me are:
A core goal is for you to be unable to shoot yourself in the foot. For example, it doesn't have severely outdated hash algos like MD2.
It has strong institutional support
500,000 tests with continuous integration on various platforms!
Their documentation website has a better SSL configuration (near-perfect A+ score instead of a mediocre B rating)
They have a disclosure policy for vulnerabilities.
You can read more about the reasons for creating the new library on LWN.
Second, the other answer recommends using SHA1 as the encryption key. SHA1 is dangerously weak and getting weaker. The replacement for SHA1 is SHA2, and on top of that, you should really being salting your hash and stretching it using either bcrypt or PBKDF2. Salting is important as a protection against rainbow tables and stretching is an important protection against brute forcing.
(Bcrypt is less tested, but is designed to use lots of memory and PBKDF2 is designed to be slow and is recommended by NIST. In my implementation, I use PBKDF2. If you want more on the differences, read this.)
For encryption AES in CBC mode with a 128-bit key should be used, as mentioned above – that hasn't changed, although it's now rolled up into a spec called Fernet. The initialization vector will be generated for you automatically in this library, so you can safely forget about that.
On the Key Generation and Storage Layer
The other answers are quite right to suggest that you need to carefully consider key handling and opt for something like OAuth, if you can. But assuming that's not possible (it isn't in my implementation), you have two use cases: Cron jobs and Interactive.
The cron job use case boils down to the fact that you need to keep a key somewhere safe and use it to run cron jobs. I haven't studied this, so I won't opine here. I think there are a lot of good ways to do this, but I don't know the easiest way.
For the Interactive use case, what you need to do is collect a user's password, use that to generate a key, and then use that key to decrypt the stored credentials.
Bringing it home
Here's how I would do all of the above, using the Cryptography library:
from cryptography.fernet import Fernet
from cryptography.hazmat.primitives.hashes import SHA256
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
from cryptography.hazmat.backends import default_backend
secret = "Some secret"
# Generate a salt for use in the PBKDF2 hash
salt = base64.b64encode(os.urandom(12)) # Recommended method from cryptography.io
# Set up the hashing algo
kdf = PBKDF2HMAC(
algorithm=SHA256(),
length=32,
salt=str(salt),
iterations=100000, # This stretches the hash against brute forcing
backend=default_backend(), # Typically this is OpenSSL
)
# Derive a binary hash and encode it with base 64 encoding
hashed_pwd = base64.b64encode(kdf.derive(user_pwd))
# Set up AES in CBC mode using the hash as the key
f = Fernet(hashed_pwd)
encrypted_secret = f.encrypt(secret)
# Store the safe inputs in the DB, but do NOT include a hash of the
# user's password, as that is the key to the encryption! Only store
# the salt, the algo and the number of iterations.
db.store(
user='some-user',
secret=encrypted_secret,
algo='pbkdf2_sha256',
iterations='100000',
salt=salt
)
Decryption then looks like:
# Get the data back from your database
encrypted_secret, algo, iterations, salt = db.get('some-user')
# Set up the Key Derivation Formula (PBKDF2)
kdf = PBKDF2HMAC(
algorithm=SHA256(),
length=32,
salt=str(salt),
iterations=int(iterations),
backend=default_backend(),
)
# Generate the key from the user's password
key = base64.b64encode(kdf.derive(user_pwd))
# Set up the AES encryption again, using the key
f = Fernet(key)
# Decrypt the secret!
secret = f.decrypt(encrypted_secret)
print(" Your secret is: %s" % secret)
Attacks?
Let's assume your DB is leaked to the Internet. What can an attacker do? Well, the key we used for encryption took the 100,000th SHA256 hash of your user's salted password. We stored the salt and our encryption algo in your database. An attacker must therefore either:
Attempt brute force of the hash: Combine the salt with every possible password and hash it 100,000 times. Take that hash and try it as the decryption key. The attacker will have to do 100,000 hashes just to try one password. This is basically impossible.
Try every possible hash directly as the decryption key. This is basically impossible.
Try a rainbow table with pre-computed hashes? Nope, not when random salts are involved.
I think this is pretty much solid.
There is, however, one other thing to think about. PBKDF2 is designed to be slow. It requires a lot of CPU time. This means that you are opening yourself up to DDOS attacks if there's a way for users to generate PBKDF2 hashes. Be prepared for this.
Postscript
All of this said, I think there are libraries that will do some of this for you. Google around for things like django encrypted field. I can't make any promises about those implementations, but perhaps you'll learn something about how others have done this.
First storing on a server credentials enough to login to a multitude of systems looks like a nightmare. Compromising code on your server will leak them all whatever the encryption.
You should store only the credentials that would be necessary to perform your task (i.e. files sync). For servers you should consider using synchronization server like RSync, for Google the protocols like OAuth etc. This way if your server is compromised this will only leak the data not the access to systems.
Next thing is encrypting these credentials. For cryptography I advise you to use PYCrypto.
For all random numbers you would use in your cryptography generate them by Crypto.Random (or some other strong method) to be sure they are strong enough.
You should not encrypt different credentials with the same key. The method I would recommend is this:
Your server should have it's master secret M (derived from /dev/random). Store it in the file owned by root and readable by root only.
When your server starts with root privileges it reads the file into memory and before serving clients drops it's privileges. That's normal practice for web servers and other demons.
When you are to write a new credential (or update existing one) generate a random block S. Take the first half and calculate hash K=H(S1,M). That would be your encryption key.
Use CBC mode to encrypt your data. Take the initialization vector (IV) from S2.
Store S alongside with encrypted data.
When you need to decrypt just take out S create the K and decrypt with the same IV.
For hash I would advise SHA1, for encryption — AES. Hashes and symmetric cyphers are fast enough so going for larger key sizes wouldn't hurt.
This scheme is a bit overshot in some places but again this wouldn't hurt.
But remember again, best way to store credentials is not to store credentials, and when you have to, use the least privileged ones that will allow you to accomplish the task.
Maybe you can rely on a multi-user scheme, by creating :
A user running Django (e.g. django) who does not have the permission to access the credentials
A user having those permissions (e.g. sync).
Both of them can be in the django group, to allow them to access the app. After that, make a script (a Django command, such as manage.py sync-external, for instance) that syncs what you want.
That way, the django user will have access to the app and the sync script, but not the credentials, because only the sync user does. If anyone tries to run that script without the credentials, it will of course result in an error.
Relying on Linux permission model is in my opinion a "Good Idea", but I'm not a security expert, so bear that in mind. If anyone has anything to say about what's above, don't hesitate!
I am learning python so I figured I would try to make a script that automatically sends gmail emails for me.
It needs to send a login and password to my gmail account to automatically send some emails for me.
I read some of the guides here and it seems passlib is a good choice. But I am still a bit confused about how to set it up.
I use passlib to encrypt my password
The result will be stored in a hidden file in my linux vm.
When script runs, it needs to parse that hidden file on my vm for the hased password.
Here is where I am confused, I think I need to decrypt it and before sending it out? Isn't that rather insecure? Or can I send it without decryption and hopefully gmail can decrypt it?
I am using the SMTP library.
server = smtplib.SMTP('smtp.gmail.com', 587)
server.ehlo()
server.starttls()
server.login(gmail_sender, gmail_passwd)
Thanks
Passlib probably isn't actually the right choice here. It's primarily concerned with one-way hashing of passwords. You'd be able to use the resulting hash to verify if a user-provided password matches your hash, but not decrypt the hash. (Older Passlib releases did have an incorrectly named .encrypt() method, but Passlib 1.7 renamed that to .hash()).
You probably want one of two things, depending on your use case:
keyring is a python interface for saving & storing passwords in an OS-controlled "wallet". It works cross-platform on windows, linux, etc.
However, it may prove awkward to use if the password is needing to be run from a script that's triggered outside of a logged-in desktop session.
In that case, you may want to encrypt the credentials you're using for storage, and pick a single password (provided at call time to your script, e.g. from command line prompt or env var) to decrypt those credentials.
cryptography is a good choice for doing something like that; it provides a number of high-level functions that don't require (too much) crypto experience, such as their fernet encryption helpers.
The starttls() call sets up an encrypted SSL/TLS connection, so the password is not being sent in cleartext. Yes you will need decrypt the password before sending it to server.login(). Alternatively you could store the password in an environment variable and read it into python, which is not as secure, but much easier to implement.
Since the mechanism of bcrypt is:
>>> myhash = bcrypt.hashpw('testpassword', bcrypt.gensalt(12))
>>> myhash
'$2a$12$K1hnCm5z74QtXaynv4.S8.i1FK9xjRr7JSPCRCyB9zpv8xZznZGFi'
>>> bcrypt.hashpw('testpassword', myhash)
'$2a$12$K1hnCm5z74QtXaynv4.S8.i1FK9xjRr7JSPCRCyB9zpv8xZznZGFi'
I want to use it for auth. The problem is that I want to make it from the client, so I need the salt part in the client.
I thought, if I use my own gensalt(username) — which generates a salt from a user name — it could be good for the client to always use the same salt, different from other users.
Is that a good approximation to bcrypt and for my project, or am I breaking the security in bcrypt mechanism?
I’m thinking that if someone wants to decrypt the password, it can’t be possible using rainbow tables because (s)he must use one for each user. I’m not experienced enough in security issues to know if that would be good. Maybe the hashpw is fast enough to do brute force on a PC.
The short answer is: No, what you are describing isn't secure at all.
First of all, bcrypt is not an encryption function and there for its results of this function cannot be "decrypted". bcrypt is a message digest function built using blowfish. Hashes produced by a message digest function are cracked.
It is very problematic for a client to authenticate using a message digest function. Microsoft's NTLM uses a message digest function for authentication and it has been broken many times. I think that this approach to authentication is flawed and should be avoided.
The reason why message digest functions are used is as a defense in depth measure in security in layers. If an attacker is able to find a sql injection vulnerability you want to force them to spend resources to break the hash before they can login. If i can pull the hash out of the database, and use this to login, then your system is totally worthless. Replay attacks are a huge concern when a client authenticates with a hash. If I can sniff the network and replay the login sequence, then this system is totally worthless.
Generate a random salt, bcrypt.gensalt(12) is probably fine. Store the hash and the salt in your database. You must authenticate using a secure transport layer. Make sure you read owasp a9.
I've implemented a HTTP server (CherryPy and Python) that receives an encrypted file from a client (Android). I'm using OpenSSL to decrypt the uploaded file. Currently I'm using openssl -enc -pass file:password.txt -in encryptedfile -out decryptedfile to perform to decryption on the server side. As you can see the password used by openssl is stored in a plain text file (password.txt).
Is there a more secure way to store this OpenSSL password?
Thanks.
Pass it through a higher FD, and use that FD in the command line. Note that you'll need to use the preexec_fn argument to set up the FD before the process gets run.
subprocess.Popen(['openssl', ..., 'file:/dev/fd/12', ...], ...,
preexec_fn=passtofd12(password), ...)
For the sake of privacy for a user and other reasons passwords are generally not stored by servers. Typically users choose a password which is stored as a hash of some sort on the server.
Users then authenticate with the web application by checking the stored hash against a hash supplied based on user input. Once the client is authenticated a session identifier is provided allowing use of server resource(s). During this time a user can for instance upload the file. Encryption of the file on the server should be un-necessary assuming the hosting server is secured properly and and absent of other issues.
In this case, the authentication mechanism is not made clear, neither are the threats that pose a danger, or the life cycle of that uploaded file.
It seems that a server is receiving an encrypted file, plus some type of password. Is the protection of the password being considered during the transmission phase, or as storage on the server? The HTTPS protocol can help guard against threats concerning the transmission of the file/data. As I see from your description the concern seems to be storage on the server side.
Encrypting the passwords once they have been received by the server (either individually or by using a master password) adds another layer of security, but this approach is not without fault as the passphrase either (1) needs to be stored on the server in cleartext for accessing the files (2) or needs to be entered manually by an administrator when needed as part of any processing requiring the password - note that any resources encrypted with the password become un-useable to users.
While I am not completely aware of what is going on, the most secure thing to do would be to re-work the web application and carefully think through the design and its requirements.
I have a webapp with some functionality that I'd like to be made accessible via an API or webservice. My problem is that I want to control where my API can be accessed from, that is, I only want the apps that I create or approve to have access to my API. The API would be a web-based REST service. My users do not login, so there is no authentication of the user. The most likely use case, and the one to work with now, is that the app will be an iOS app. The API will be coded with django/python.
Given that it is not possible to view the source-code of an iOS app (I think, correct me if I'm wrong), my initial thinking is that I could just have some secret key that is passed in as a parameter to the API. However, anyone listening in on the connection would be able to see this key and just use it from anywhere else in the world.
My next though is that I could add a prior step. Before the app gets to use API it must pass a challenge. On first request, my API will create a random phrase and encrypt it with some secret key (RSA?). The original, unencrypted phrase will be sent to the app, which must also encrypt the phrase with the same secret key and send back the encrypted text with their request. If the encryptions match up, the app gets access but if not they don't.
My question is: Does this sound like a good methodology and, if so, are there any existing libraries out there that can do these types of things? I'll be working in python server-side and objective-c client side for now.
The easiest solution would be IP whitelisting if you expect the API consumer to be requesting from the same IP all the time.
If you want to support the ability to 'authenticate' from anywhere, then you're on the right track; it would be a lot easier to share an encryption method and then requesting users send a request with an encrypted api consumer handle / password / request date. Your server decodes the encrypted value, checks the handle / password against a whitelist you control, and then verifies that the request date is within some timeframe that is valid; aka, if the request date wasnt within 1 minute ago, deny the request (that way, someone intercepts the encrypted value, it's only valid for 1 minute). The encrypted value keeps changing because the request time is changing, so the key for authentication keeps changing.
That's my take anyways.
In addition to Tejs' answer, one known way is to bind the Product ID of the OS (or another unique ID of the client machine) with a specific password that is known to the user, but not stored in the application, and use those to encrypt/decrypt messages. So for example, when you get the unique no. of the machine from the user, you supply him with password, such that they complete each other to create a seed X for RC4 for example and use it for encryption / decryption. this seed X is known to the server as well, and it also use it for encryption / decryption. I won't tell you this is the best way of course, but assuming you trust the end-user (but not necessarily any one who has access to this computer), it seems sufficient to me.
Also, a good python library for cryptography is pycrypto
On first request, my API will create a random phrase and encrypt it with some secret key (RSA?)
Read up on http://en.wikipedia.org/wiki/Digital_signature to see the whole story behind this kind of handshake.
Then read up on
http://en.wikipedia.org/wiki/Lamport_signature
And it's cousin
http://en.wikipedia.org/wiki/Hash_tree
The idea is that a signature can be used once. Compromise of the signature in your iOS code doesn't matter since it's a one-use-only key.
If you use a hash tree, you can get a number of valid signatures by building a hash tree over the iOS binary file itself. The server and the iOS app both have access to the same
file being used to generate the signatures.