I'm using AWS for the first time and have just installed boto for python. I'm stuck at the step where it advices to:
"You can place this file either at /etc/boto.cfg for system-wide use or in the home directory of the user executing the commands as ~/.boto."
Honestly, I have no idea what to do. First, I can't find the boto.cfg and second I'm not sure which command to execute for the second option.
Also, when I deploy the application to my server, I'm assuming I need to do the same thing there too...
"You can place this file either at /etc/boto.cfg for system-wide use
or in the home directory of the user executing the commands as
~/.boto."
The former simply means that you might create a configuration file named boto.cfg within directory /etc (i.e. it won't necessarily be there already, depending on how boto has been installed on your particular system).
The latter is indeed phrased a bit unfortunate - ~/.boto means that boto will look for a configuration file named .boto within the home directory of the user executing the commands (i.e. Python scripts) which are facilitating the boto library.
You can read more about this in the boto wiki article BotoConfig, e.g. regarding the question at hand:
A boto config file is simply a .ini format configuration file that
specifies values for options that control the behavior of the boto
library. Upon startup, the boto library looks for configuration files
in the following locations and in the following order:
/etc/boto.cfg - for site-wide settings that all users on this machine
will use
~/.boto - for user-specific settings
You'll indeed need to prepare a respective configuration file on the server your application is deployed to as well.
Good luck!
For those who want to configure the credentials in Windows:
1-Create your file with the name you want(e.g boto_config.cfg) and place it in a location of your choice(e.g C:\Users\\configs).
2- Create an environment variable with the Name='BOTO_CONFIG' and Value= file_location/file_name
3- Boto is now ready to work with credentials automatically configured!
To create environment variables in Windows follow this tutorial: http://www.onlinehowto.net/Tutorials/Windows-7/Creating-System-Environment-Variables-in-Windows-7/1705
For anyone looking for information on the now-current boto3, it does not use a separate configuration file but rather respects the default one created by the aws cli when running aws configure (Ie, it will look at ~/.aws/config)
Related
I am trying to set up ezsheets for the use with Google Sheets. I followed the instructions from here https://ezsheets.readthedocs.io/en/latest/ and here https://automatetheboringstuff.com/2e/chapter14/
The set up process works quite differently on my computer: Somehow I could download the credentials-sheets.json. I need to download the token-sheets.pickle and token-drive.pickle files. When I run import ezsheets, no browser window is opended as described in the set up instructions. Nothing happens.
Is there another way to download both files?
I followed the steps you referenced and managed to generate the files, but I also encountered the same issue before figuring out the cause. The problem is that there are a few possible causes and the script silently fails without telling you exactly what happened.
Here are a few suggestions:
First off you need to configure your OAuth Consent Screen. You won't be able to create the credentials without it.
Make sure that you have the right credentials file. To generate it you have to go to the Credentials page in the Cloud Console. The docs say that you need an OAuth Client ID. Make sure that you have chosen the correct app at the top.
Then you will be prompted to choose an application type. According to the docs you shared the type should be "Other", but this is no longer available so "Desktop app" is the best equivalent if you're just running a local script.
After that you can just choose a name and create the credentials. You will be prompted to download the file afterwards.
Check that the credentials-sheets.json file has that exact name.
Make sure that the credentials-sheets.json file is located in the same directory where you're running your python script file or console commands.
Check that you've enabled both the Sheets and Drive API in your GCP Project.
Python will try to setup a temporary server on http://localhost:8080/ to retrieve the pickle files. If another application is using port 8080 then it will also fail. In my case a previously failed Python script was hanging on to that port.
To find and close the processes using port 8080 you can refer to this answer for Linux/Mac or this other answer for Windows. Just make sure that the process is not something you're currently using.
I just used the single import ezsheets command to get the files so after getting the token-sheets.pickle I had to run it again to get the token-drive.pickle, but after that the library should detect that you already have the files.
I want to provide a config ini file with my python package that will hold sensitive database credentials. How do I provide default config file that will be installed let's say under /etc/config folder? what is the best practice in terms of deploying such config files with packages, installed with pip? I do not like to keep this config file with the rest of the package files.
Thanks!
I just checked how it is done with popular python projects.
django creates a config file that allows pointing to another config file with database connection details, stored in a different place:
https://docs.djangoproject.com/en/2.0/ref/databases/#connecting-to-the-database
flask and airflow create default configs under user home folder but then allow to use an environment variable to overwrite location of a config file. Config file stores sensitive data such as database connections or secret keys:
http://flask.pocoo.org/docs/0.12/config/#configuring-from-files
https://airflow.apache.org/configuration.html
Both methods allow to:
1) distribute default config file and deploy easily with the package install
2) allow storing sensitive config values in a secured folder
airflow also allows encrypting database connection string using a secret key.
Also refer to this post below and some ideas on how to deploy/store your config file:
https://stackoverflow.com/a/22554594/473725
Our client has a web application running a Django instance from virtualenv on a Ubuntu server. We did a security audit for that service and found a path traversal vulnerability in a file upload form that could allow the attacker to write arbitrary files in the django user owned paths. Example:
A parameter "Import Name" is supplied with value
../some/path/to/create
Then the form file field is supplied with arbitary filename and the correct file contents
The application then does
try:
path = os.path.join(DEFAULT_UPLOAD_DIR, <Import Name>)
os.mkdir(path)
...
with open(os.path.join(path, <Filename From Form>)) as upload_file:
upload_file.write(<File Contents>)
...
The unsafe os.path.join allows the attacker to walk up in the directory tree and upload to other directories than the DEFAULT_UPLOAD_DIR. So basically if the attacker is able to find a path that doesn't yet exist on the server he's able to create that folder avoiding the failure of os.mkdir() in the try...except and the file is uploaded there.
Now this translates to a real exploit if the attacker is able to write to
../virtualenvs/<env name>/lib/python2.7/
Since e.g Django modules are loaded from the subdirectory site-packages within the virtualenv python directory and pythonpath tells us whatever is directly under lib/python2.7 gets loaded first, essentially the module loading order allows the attacker to 'overwrite' a module and ensure their code is run on import.
We did a proof-of-concept penetration test and wrote to
../virtualenvs/somepath/__init__.py
Which succeeded but for some reason we are unable to write to
../virtualenvs/<actual env name>/
Which is strange cause the permissions are exactly the same as with somepath and owner / group is in both cases the Django user. Enabling the virtualenv for the Django user and going to the python shell it allows me to do the write so it seems weird that it can't when called from the vulnerable form view.
The question is: Is there something special about the virtualenv path from which the Django instance is running that makes it unable to write to that path? Or am I missing something?
Let's say I have some code running on a Heroku dyno (such as this autoscaling script), that needs access to the Platform API. To access the API, I have to authenticate using my app's API Key.
What's the right way to do this?
That script I referenced hardcoded the API Key in the script itself.
A better practice generally seems to put secrets in environment variables, which is what Heroku normally recommends. However, they say they say:
Setting the HEROKU_API_KEY environment variable on your machine will
interfere with normal functioning of auth commands from Toolbelt.
Clearly I could store the API key with under a different key name.
What's the right way? I couldn't find this in the documentation, but seems like a common issue.
Yes, storing this token into a config var is the right way to go.
As for HEROKU_API_KEY, this will happen because locally, the toolbelt will look for the environment variable as one solution to try to fetch your token.
This won't impact your production environment (the heroku toolbelt isn't available within dynos).
Locally, you can also set it easily with a tool like python-dotenv, which will allow you to have a local .env file (don't check it into source control, or your token could be corrupted), with all of it's values available as env vars in your dev app.
I'm trying to run a site with Django on an IIS-based server. I followed all the instructions on the main site (http://code.djangoproject.com/wiki/DjangoOnWindowsWithIISAndSQLServer), and double checked it with a very good article (http://www.messwithsilverlight.com/2009/11/django-on-windows-server-2003-and-iis6/).
I successfully got as far as setting up IIS to read .py files. Following the main instructions, I can get the server to render Info.py. However, I can't seem to get IIS and Django to play nice. If, for instance, my Virtual directory is "abc", then if I go to "localhost/abc/", the browser simply shows me the content directory for that folder. Furthermore, if I have my urls set up so that "/dashboard/1" should bring me to a certain page, entering "localhost/abc/dashboard/1" gives me a "page cannot be displayed" error.
I'm fairly certain IIS simply isn't referencing or interacting with Django at all. Does anyone have any ideas how to fix this?
Thanks
Here are the original instructions I followed,
basics instructions: https://code.djangoproject.com/wiki/DjangoOnWindowsWithIISAndSQLServer
additional tips: http://whelkaholism.blogspot.ca/
The first thing you should do is install Python 2.5 or 2.6, for 2.7 you need to recompile PyISAPIe, which I have not done. http://www.python.org/ftp/python/2.6/python-2.6.msi
You need to install the version of PyISAPIe that will match your Python Interpreter version, if they do not match, it will fail. Get it there : http://sourceforge.net/projects/pyisapie/files/pyisapie/
Move the extracted folder from the last step at a decent location (i.e. C:)
You need to change the security settings of the PyISAPIe.dll, they suggest Network Service read, but I set everyone, to be sure there are no problems with this
You then have to CUT AND PASTE (Important) the Http folder of PyISAPIe to Lib\Site-Packages of your Python installation directory
Next, you setup IIS (open the manager with inetmgr in run (winkey+r):
Add a new virtual directory and allow executing ISAPI extensions when prompted by the wizard
Add a new wildcard extension in the property of your virtual directory, untick file exist setting
Add Web Service Extension to IIS Manager pointing to the dll, ensure it is allowed
From the PyISAPIe folder, copy examples\django\Isapi.py and paste it in Lib\Site-Packages\Http
In Isapi.py, set the path (i.e. c:\inetpub\wwwroot\ web_site\ django_project ) and DJANGO_SETTINGS_MODULE (i.e. django_app .settings)
When any change is done to your files, use iisreset in your command prompt to apply the changes
Here are some other things you might do
Ensure the path of your db file (if sqlite used) is okay
Do the same with template location settings
In your urls and html files, ensure the path start with the name you gave to your virtual directory alias (i.e. web_site in our example)
Finally, you may encounter difficulties with serving your CSS. If you have any troubles, tell me and I will update my post.
Serving Django with any webserver basically involves three key details:
Telling the webserver, "I want you
to serve content that is provided by
this module that invokes python"
Telling the python module, "I want you to execute python code
using the details in this file"
Telling the file, "I want you to use Django"
If you're getting a directory listing back for your Virtual Directory then it would seem that you should investigate the VD settings to make sure PyISAPIe is configured for that directory (key details #1).
From the article you mentioned:
Open the IIS Management Console, and create a new virtual directory, and
allow executing ISAPI extensions when
prompted by the wizard.
View the properties of the new folder and click on the
"configuration" button (if it's greyed
out, click 'create' first), then add a
new wildcard extension (the lower
box), locate the pyisapie.dll file and
untick the "check that file exists"
box.
In the IIS Manager, go to the "Web Service Extensions" section, and
right click -> add new web service
extension.
Give it a name (it doesn't matter what), add the pyisapie.dll
fill as a required file and check the
box to set the extension status to
allowed.