I want to be able to have a file that contains variables that I can access in all my scripts within my project? I am not sure how to do this, I was also looking at environmental variables based on this article. Not sure how to google to find what i want to find. Is there a package out there that lets you toggle between the global variables you want to use?
I have a remote server that I want to use but have to change my directories, I want to be able to refer to those specific remote paths for the script but then switch back/toggle to another set of environmental variables that contain the directory paths that are correct to my local. What is the best way for me to do this?
In my head it would look something like one variable with two paths and I switch to one or the other with just one change "local", "remote". Any resources that someone can link me to or just the name of what they use or how they do it would help.
You should be able to modify os.environ
Let`s try it:
>>> import os
>>> from invoke import run
>>> os.environ['remote'] = 'server'
>>> os.environ.get('remote')
'server'
>>> run("env | grep LANG")
LANG=en_US.UTF-8
<invoke.runners.Result object at 0x1a10310>
>>> run("env | grep remote")
remote=server
<invoke.runners.Result object at 0x1a19f10>
>>>
And visit these links as resources:
os.environ os.putenv os.execve
I'm trying to learn how to use variables from Jenkins in Python scripts. I've already learned that I need to call the variables, but I'm not sure how to implement them in the case of using os.path.join().
I'm not a developer; I'm a technical writer. This code was written by somebody else. I'm just trying to adapt the Jenkins scripts so they are parameterized so we don't have to modify the Python scripts for every release.
I'm using inline Jenkins python scripts inside a Jenkins job. The Jenkins string parameters are "BranchID" and "BranchIDShort". I've looked through many questions that talk about how you have to establish the variables in the Python script, but with the case of os.path.join(),I'm not sure what to do.
Here is the original code. I added the part where we establish the variables from the Jenkins parameters, but I don't know how to use them in the os.path.join() function.
# Delete previous builds.
import os
import shutil
BranchID = os.getenv("BranchID")
BranchIDshort = os.getenv("BranchIDshort")
print "Delete any output from a previous build."
if os.path.exists(os.path.join("C:\\Doc192CS", "Output")):
shutil.rmtree(os.path.join("C:\\Doc192CS", "Output"))
I expect output like: c:\Doc192CS\Output
I am afraid that if I do the following code:
if os.path.exists(os.path.join("C:\\Doc",BranchIDshort,"CS", "Output")):
shutil.rmtree(os.path.join("C:\\Doc",BranchIDshort,"CS", "Output"))
I'll get: c:\Doc\192\CS\Output.
Is there a way to use the BranchIDshort variable in this context to get the output c:\Doc192CS\Output?
User #Adonis gave the correct solution as a comment. Here is what he said:
Indeed you're right. What you would want to do is rather:
os.path.exists(os.path.join("C:\\","Doc{}CS".format(BranchIDshort),"Output"))
(in short use a format string for the 2nd argument)
So the complete corrected code is:
import os
import shutil
BranchID = os.getenv("BranchID")
BranchIDshort = os.getenv("BranchIDshort")
print "Delete any output from a previous build."
if os.path.exists(os.path.join("C:\\Doc{}CS".format(BranchIDshort), "Output")):
shutil.rmtree(os.path.join("C:\\Doc{}CS".format(BranchIDshort), "Output"))
Thank you, #Adonis!
I'm writing a Python script to update maven passwords. To find the path to mvn and the file-path I use M2_HOME and M2_REPOS like this
mvn = os.environ['M2_HOME'] + "/bin/mvn %s %s"
real_files_path = os.environ['M2_REPOS']
The problem is that some of my coworkers have M3_HOME or M3_REPOS set in their .bash files. So I need a more generic solution to get mvn the file-path.
Well if the only problem you face is different environment variable names, you could use (and chain) the dict.get() method like this :
mvn = os.environ.get("M2_HOME", os.environ.get("M3_HOME"))
real_files_path = os.environ.get('M2_REPOS', os.environ.get("M3_REPOS"))
This way, python will first try to find the M2...variables first, and if not defined will fall back on the M3... one.
I need to set a environment variables in the python, and I try the commands bellow
import os
basepath = os.putenv('CPATH','/Users/cat/doc')
basepath = os.getenv('CPATH','/Users/cat/doc')
And when I print the varible, they are not set:
print basepath
None
What i'm doing wrong?
Rephrasing the question, I would like to create a base path based on a evironment variable. I'm testing this:
os.environ["CPATH"] = "/Users/cat/doc"
print os.environ["CPATH"]
base_path=os.getenv('C_PATH')
And when I try to print the basepath:
print basepath
it always return
None
Try this one.
os.environ["CPATH"] = "/Users/cat/doc"
Python documentation is quite clear about it.
Calling putenv() directly does not change os.environ, so it’s better to modify os.environ.
Based on the documentation, the os.getenv() is available on most flavors of Unix and Windows. OS X is not listed.
Use rather the snippet below to retrieve the value.
value = os.environ.get('CPATH')
Use os.environ:
os.environ['CPATH'] = '/Users/cat/doc'
print os.environ['CPATH'] # /Users/cat/doc
print os.environ.get('CPATH') # /Users/cat/doc
See the above link for more details:
If the platform supports the putenv() function, this mapping may be
used to modify the environment as well as query the environment.
putenv() will be called automatically when the mapping is modified.
Note Calling putenv() directly does not change os.environ, so it’s
better to modify os.environ.
It's a good practice to restore the environment variables at function completion.
You may need something like the modified_environ context manager describe in this question to restore the environment variables.
Usage example:
with modified_environ(CPATH='/Users/cat/doc'):
call_my_function()
Django uses real Python files for settings, Trac uses a .ini file, and some other pieces of software uses XML files to hold this information.
Are one of these approaches blessed by Guido and/or the Python community more than another?
Depends on the predominant intended audience.
If it is programmers who change the file anyway, just use python files like settings.py
If it is end users then, think about ini files.
As many have said, there is no "offical" way. There are, however, many choices. There was a talk at PyCon this year about many of the available options.
Don't know if this can be considered "official", but it is in standard library: 14.2. ConfigParser — Configuration file parser.
This is, obviously, not an universal solution, though. Just use whatever feels most appropriate to the task, without any necessary complexity (and — especially — Turing-completeness! Think about automatic or GUI configurators).
I use a shelf ( http://docs.python.org/library/shelve.html ):
shelf = shelve.open(filename)
shelf["users"] = ["David", "Abraham"]
shelf.sync() # Save
Just one more option, PyQt. Qt has a platform independent way of storing settings with the QSettings class. Underneath the hood, on windows it uses the registry and in linux it stores the settings in a hidden conf file. QSettings works very well and is pretty seemless.
There is no blessed solution as far as I know. There is no right or wrong way to storing app settings neither, xml, json or all types of files are fine as long as you are confortable with. For python I personally use pypref it's very easy, cross platform and straightforward.
pypref is very useful as one can store static and dynamic settings and preferences ...
from pypref import Preferences
# create singleton preferences instance
pref = Preferences(filename="preferences_test.py")
# create preferences dict
pdict = {'preference 1': 1, 12345: 'I am a number'}
# set preferences. This would automatically create preferences_test.py
# in your home directory. Go and check it.
pref.set_preferences(pdict)
# lets update the preferences. This would automatically update
# preferences_test.py file, you can verify that.
pref.update_preferences({'preference 1': 2})
# lets get some preferences. This would return the value of the preference if
# it is defined or default value if it is not.
print pref.get('preference 1')
# In some cases we must use raw strings. This is most likely needed when
# working with paths in a windows systems or when a preference includes
# especial characters. That's how to do it ...
pref.update_preferences({'my path': " r'C:\Users\Me\Desktop' "})
# Sometimes preferences to change dynamically or to be evaluated real time.
# This also can be done by using dynamic property. In this example password
# generator preference is set using uuid module. dynamic dictionary
# must include all modules name that must be imported upon evaluating
# a dynamic preference
pre = {'password generator': "str(uuid.uuid1())"}
dyn = {'password generator': ['uuid',]}
pref.update_preferences(preferences=pre, dynamic=dyn)
# lets pull 'password generator' preferences twice and notice how
# passwords are different at every pull
print pref.get('password generator')
print pref.get('password generator')
# those preferences can be accessed later. Let's simulate that by creating
# another preferences instances which will automatically detect the
# existance of a preferences file and connect to it
newPref = Preferences(filename="preferences_test.py")
# let's print 'my path' preference
print newPref.get('my path')
I am not sure that there is an 'official' way (it is not mentioned in the Zen of Python :) )- I tend to use the Config Parser module myself and I think that you will find that pretty common. I prefer that over the python file approach because you can write back to it and dynamically reload if you want.
One of the easiest ways which is use is using the json module.
Save the file in config.json with the details as shown below.
Saving data in the json file:
{
"john" : {
"number" : "948075049" ,
"password":"thisisit"
}
}
Reading from json file:
import json
#open the config.json file
with open('config.json') as f:
mydata = json.load(f) ;
#Now mydata is a python dictionary
print("username is " , mydata.get('john').get('number') , " password is " , mydata.get('john').get('password')) ;
It depends largely on how complicated your configuration is. If you're doing a simple key-value mapping and you want the capability to edit the settings with a text editor, I think ConfigParser is the way to go.
If your settings are complicated and include lists and nested data structures, I'd use XML or JSON and create a configuration editor.
For really complicated things where the end user isn't expected to change the settings much, or is more trusted, just create a set of Python classes and evaluate a Python script to get the configuration.
For web applications I like using OS environment variables: os.environ.get('CONFIG_OPTION')
This works especially well for settings that vary between deploys. You can read more about the rationale behind using env vars here: http://www.12factor.net/config
Of course, this only works for read-only values because changes to the environment are usually not persistent. But if you don't need write access they are a very good solution.
It is more of convenience. There is no official way per say. But using XML files would make sense as they can be manipulated by various other applications/libraries.
Not an official one but this way works well for all my Python projects.
pip install python-settings
Docs here: https://github.com/charlsagente/python-settings
You need a settings.py file with all your defined constants like:
# settings.py
DATABASE_HOST = '10.0.0.1'
Then you need to either set an env variable (export SETTINGS_MODULE=settings) or manually calling the configure method:
# something_else.py
from python_settings import settings
from . import settings as my_local_settings
settings.configure(my_local_settings) # configure() receives a python module
The utility also supports Lazy initialization for heavy to load objects, so when you run your python project it loads faster since it only evaluates the settings variable when its needed
# settings.py
from python_settings import LazySetting
from my_awesome_library import HeavyInitializationClass # Heavy to initialize object
LAZY_INITIALIZATION = LazySetting(HeavyInitializationClass, "127.0.0.1:4222")
# LazySetting(Class, *args, **kwargs)
Just configure once and now call your variables where is needed:
# my_awesome_file.py
from python_settings import settings
print(settings.DATABASE_HOST) # Will print '10.0.0.1'
why would Guido blessed something that is out of his scope? No there is nothing particular blessed.