I am creating a Python script which reads a spreadsheet and issues Rest requests to an external service. A token must be obtained to issue requests to the Rest service, so the script needs a username and password to obtain the oauth2 token.
What's the best or standard way to hide or not have visible information in the Python script?
I recommend using a config file. Let's create a config file and name it config.cfg. The file structure should look more or less like this:
[whatever]
key=qwerertyertywert2345
secret=sadfgwertgrtujdfgh
Then in python you can load it this way:
from configparser import ConfigParser
config = ConfigParser()
config.read('config.cfg')
my_key = config['whatever']['key']
my_secret = config['whatever']['secret']
In general, the most standard way to handle secrets in Python is by putting them in runtime configuration.
You can do that by reading explicitly from external files or using os.getenv to read from environment variables.
Another way is to use a tool like python-decouple, which lets you use the environment (variables), a .env file, and an .ini file in a cascade, so that developers and operations can control the environment in local, dev, and production systems.
Related
I'm currently using the azure-cosmos module in Python to connect to a database on Azure. I want to fetch the data, make a few transformations, and then push it to a new container.
You need the key and client ID to connect to the database, which I've used as variables in my code for now, as follows:
url = 'https://xyz.azure.com:443/'
key ='randomlettersandnumbers=='
client = CosmosClient(url, credential=key)
This seems to be a bad practice intuitively, and especially once I push this to Git, anyone could gain access to my database. So what's the most secure way to do this?
I'm coming from a non-SWE background, so apologies if this question is dumb.
Thanks!
The way I deal with this kind of problem is by using environment variables
import os
url = os.environ.get("url-endpoint")
key = os.environ.get("api-key")
client = CosmosClient(url, credential=key)
You can set them in your ssh shell like that:
export url-endpoint="https://xyz.azure.com:443/"
export api-key="randomlettersandnumbers=="
Or you can put them in a bash script envs.sh
export url-endpoint="https://xyz.azure.com:443/"
export api-key="randomlettersandnumbers=="
And then you can use source command.
source envs.sh
You have a good article about storing sensitive data using environment variables here
I have a python app that makes a call to a web API to collect some data, processes the data displays it on a front end. Unfortunately, the credentials that are needed to access the API also give access to a bunch of sensitive information.
Therefore, I was wondering if there was any way to connect to the API in a way that doesn't let people with access to the python code reconstruct the credentials. I have found some posts about encrypting the data outside the app and decrypting it inside the app, but it seems to me that this would require revealing the decryption method in the python code, which would mean that everyone could simply reconstruct it.
By using .env file with e.g. python-dotenv library.
mycode.py file:
from dotenv import load_dotenv
load_dotenv()
my_var=os.getenv("MYHIDDEN_VAR")
.env file:
MYHIDDEN_VAR='REAL VALUE'
.env.example file:
MYHIDDEN_VAR='OBSCURED VALUE'
At present my application related variables like external server IP / default pattern etc.. These variables are specific to my application. It includes my external server username and password.
Now how can I have a single common place so that I can externalise this from my application.
The options which I thought are below:
Have one conf.ini file and use configparser and read this during the start of the django app
But I am not sure from where I should have method to read this so that it will be done in the startup.
Other option is to save this in a py file itself and import this file in my modules
Please suggests me the good and standard approach for above issue.
Save the important secret details in an ini file and place it in etc/project_name/my_settings.ini. Read these settings from settings.py. This way it will be secure. Can be read directly in the settings file itself
Or better way is to set them in bashrc, read the env vars from it.
Check this: Setting envs
I am using Redshift and have to write some custom scripts to generate reports. I am using AWS datapipeline CustomShellActivity for running my custom logic. I am using python and boto3. I am wondering what is the safest way and in fact, best practice to provide database password in python script. I am sure that hardcoding password in script is not good practice. What other options do I have or should I explore?
A pretty standard approach is to store credentials in a secure S3 bucket and download them as part of the deployment/launch process using an IAM role with access to the secure bucket. For limited runtime cases like lambda or datapipeline you could download from S3 to an in memory buffer using boto.Key.get_contents_as_string() at startup, parse the file and set up your credentials.
For increased security you can incorporate KMS secret management. Here is an example that combines the two.
I usually store them as an environment variables. I am not sure about the AWS data pipeline deployment, but on a standard Linux box (EC2), you could do:
# ~/.profile or /etc/profile
export MY_VAR="my_value"
And then you can access them in Python like this:
# python script
import os
my_var_value = os.environ['MY_VAR'] if 'MY_VAR' in os.environ else 'default'
I am building simple app which is using Twitter API. What I have to do to hide my Twitter app keys? For example, if I will put my program to the internet and somebody who look up to the code will know my consumer key, access token etc. And if I not include this information into my program, that it won't be work!
I'm assuming by putting on the internet you mean publishing your code on github or such.
In that case you should always separate code and configuration. Put your API keys in an .ini file, i.e. config.ini, then load that file from python program using configparser
Add configuration file to your .gitignore so it would not get added to the source control.
Assuming you're running on a Unix like system, one way to handle this is environment variables.
In your shell you can do this:
export TWITTER_API_KEY=yoursecretapikey
Note that you don't use quotes of any kind for this.
Then in your script:
import os
twitter_key = os.environ.get('TWITTER_API_KEY')