Which is better for creating a settings file for Python programs, the built-in module (ConfigParser), or the independent project (ConfigObj)?
I recently switched from configparser to configobj, and I'm thrilled to have done so.
For me, the big difference is configobj's validator. It lets me very easily/succinctly (1) define the type and acceptable values for each entry, and (2) set defaults.
Those two features save me a lot of code and prevent a lot from going wrong. Plus, there's really no extra overhead to using configobj in terms of the complexity of my code, and the library is quite small.
Depending on your Python version, it may be contentious whether this answers your question, but after a short look at ConfigParser and ConfigObj, I settled for configparser, the Python 3 version of ConfigParser. There’s also a backported version on PyPI.
ConfigParser seemed cumbersome to me, maybe even—dare I say it—unpythonic, and with ConfigObj I encountered an esoteric problem with lists (I’d prefer to be able to align them vertically, since mine get really long), and it didn’t help that the latest version was published more than two years ago. The API of configparser, however, looked as spiffy as that of ConfigObj (albeit not as feature-rich), and when we move to Python 3, I can probably switch to the built-in version easily.
By the way, it works great so far.
Thus far, I found ConfigParser sufficient any time I used it. Plus, it's an included battery and not a third party library.
Though I have to admit, the code samples in the ConfigObj documentation make ConfigParser look really sucksy in comparision. For a script which heavily manipulates .ini files, I'd propably prefer it.
Related
I often include this, or something close to it, in Python scripts and IPython notebooks.
import cPickle
def unpickle(filename):
with open(filename) as f:
obj = cPickle.load(f)
return obj
This seems like a common enough use case that the standard library should provide a function that does the same thing. Is there such a function? If there isn't, how come?
Most of the serialization libraries in the stdlib and on PyPI have a similar API. I'm pretty sure it was marshal that set the standard,* and pickle, json, PyYAML, etc. have just followed in its footsteps.
So, the question is, why was marshal designed that way?
Well, you obviously need loads/dumps; you couldn't build those on top of a filename-based function, and to build them on top of a file-object-based function you'd need StringIO, which didn't come until later.
You don't necessarily need load/dump, because those could be built on top of loads/dumps—but doing so could have major performance implications: you can't save anything to the file until you've built the whole thing in memory, and vice-versa, which could be a problem for huge objects.
You definitely don't need a loadf/dumpf function based on filenames, because those can be built trivially on top of load/dump, with no performance implications, and no tricky considerations that a user is likely to get wrong.
On the one hand, it would be convenient to have them anyway—and there are some libraries, like ElementTree, that do have analogous functions. It may only save a few seconds and a few lines per project, but multiply that by thousands of projects…
On the other hand, it would make Python larger. Not so much the extra 1K to download and install it if you added these two functions to every module (although that did mean a lot more back in the 1.x days than nowadays…), but more to document, more to learn, more to remember. And of course more code to maintain—every time you need to fix a bug in marshal.dumpf you have to remember to go check pickle.dumpf and json.dumpf to make sure they don't need the change, and sometimes you won't remember.
Balancing those two considerations is really a judgment call. One someone made decades ago and probably nobody has really discussed since. If you think there's a good case for changing it today, you can always post a feature request on the issue tracker or start a thread on python-ideas.
* Not in the original 1991 version of marshal.c; that just had load and dump. Guido added loads and dumps in 1993 as part of a change whose main description was "Add separate main program for the Mac: macmain.c". Presumably because something inside the Python interpreter needed to dump and load to strings.**
** marshal is used as the underpinnings for things like importing .pyc files. This also means (at least in CPython) it's not just implemented in C, but statically built into the core of the interpreter itself. While I think it actually could be turned into a regular module since the 3.4 import changes, but it definitely couldn't have back in the early days. So, that's extra motivation to keep it small and simple.
Long story short, a piece of code that I'm working with at work has the line:
from System import System
with a later bit of code of:
desc_ = System()
xmlParser = Parser(desc_.getDocument())
# xmlParser.setEntityBase(self.dtdBase)
for featureXMLfile in featureXmlList.split(","):
print featureXMLfile
xmlParser.parse(featureXMLfile)
feat = desc_.get(featureName)
return feat
Parser is an XML parser in Java (it's included in a different import), but I don't get what the desc_ bit is doing. I mean obviously, it somehow holds the feature that we're trying to pull out, but I don't entirely see where. Is System a standard library in Python or Java, or am I looking at something custom?
Unfortunately, everyone else in my group is out for Christmas Eve vacation, so I can't ask them directly. Thank you for your help. I'm still not horribly familiar with Python.
This isn't from the standard library, so you'll need to check your system (Python has plenty of introspection to help you with that).
You can tell as Python modules in the standard library use lowercase names as per PEP-8, or by searching the library reference.
Note as well that Python has it's own XML parsing tools that will be much nicer to work with in Python than Java's.
Edit: As you have noted in the comments you are using Jython, it seems likely this is Java's System package.
millimoose indicated the correct answer in his comment, but neglected to submit it as an answer, so I'm posting to indicate the correct answer. It was indeed a custom module built by my company. I was able to determine this by typing import System; print(System) into the interpreter.
I have this core python module we use in our facility called mfxLib. I need to be able to keep different version of this module without breaking all the other modules/plugin that are importing this module.
My solution was keep a duplicate of my module by renaming them mfxLib01 and mfxLib02 then
to replace the original mfxLib module with an empty module containing only a __init__.py file that import the latest version.
# content of mfxLib.__init__.py
from mfxLib02 import *
This seems logical and seems to work but I was wondering if there was a common practice for doing this? guidelines to follow? etc
Thanks
You can import a module as another name. Commonly people use this to save typing in a long module name, for example:
import numpy as np
np.array([1,2,3,4])
Hence you could do:
import mfxLib01 as mfxLib
or
import mfxLib02 as mfxLib
then your code uses mfxLib everywhere.
That might help...
If you have different scripts requiring different versions, your current approach should be the the best, but I'd suggest using a version control system like Git or SVN. That would allow you to commit and revert to earlier versions easily, as well as share the module with other users.
Version control will almost certainly make your life easier. In addition to Petterson's recommendations, consider Mercurial. Like git and SVN, it's free. It's also written in Python and should run without difficulty on any of your systems.
Spacedman's recommendations are also useful, especially if the differences between the versions represent customizations for particular systems and the customizations are relatively stable. Note that you can use that approach in combination with a version control system.
Finally, it's always worthwhile to make a strong effort to write your module so that it can work without modification everywhere. Often, you can accomplish this by adding some optional arguments to a few key functions to handle the different requirements. Python is really convenient in that regard because keyword arguments at the end of the arg list are always optional, so you can easily arrange to provide the existing behavior by giving them suitable default values.
def foo(oldarg1, oldarg2, newarg1=None):
if newarg1 != None:
## behave differently
else:
## behave as usual
i am learning python.. i want to do certain kind of scripting in python.. like,
i want to communicate 'wmic' commands through dos promt.. store the result a file..
access some sqlite database, take the data it has and compare with the result i stored..
now, what i dont get is that, how should i proceed? is there any specific frameworks or modules/libraries? like, win32api / com or what else?
pls guide me what things i should follow/learn to accomplish what i intend to do..
thanks
for your project look here
http://tgolden.sc.sabren.com/python/wmi/index.html
http://docs.python.org/library/sqlite3.html
here is list of generic python modules
http://docs.python.org/modindex.html
http://docs.python.org/ - here you can find all information you need with examples.
One of the particularly attractive features of Python is the "batteries included" philosophy: The standard library is huge, and extremely well thought out in 90% of the modules I've ever used. Conversely, this means that a good approach is to learn it first before branching out and installing third-party libraries that may be much less well supported and, ultimately, have no advantage.
In practice, this means I'd recommend keeping https://docs.python.org/release/2.6.5/library/index.html close to your heart, and delve into the documentation for the sqlite3 and probably the subprocess modules. You may or may not want win32api later (I've never worked with wmic, so I'm not sure what you'd need), or come back here when you have concrete questions and already explored the standard library offering.
Though there can be many but as i am very new to python so which modules or classes within standard libraries i should know when programming in python, especially when i am practicing programming challenges from a C++ book? Libraries which can make my life easier? Since there can be no single correct answer, i am making this question a wiki.
Check out the excellent Python Module of the Week blog series.
The standard libraries, i.e. the ones considered more or less part of Python. Start with those, there is plenty to learn before starting on 3rd party stuff.
Things like:
os
re
subprocess
struct
The re module is a must. itertools also often comes handy.
Generally speaking: Take a deep look at Standard library. Then you might think about wxPython for GUI, numPy for computations, Django for web and Amara for XML, and... there's plenty of Python libs and modules out there. Just suit your needs.
Actually, to work problems from a C++ book using Python, you mainly just need to master Python's built-in types, especially the data structures tuple, list, set, and dict; and the built-in functions, like max, min, sorted, and reversed.
These builtins have a lot of features that aren't obvious at first, such as the in keyword, the optional key= argument to list.sort, list slicing, sequence multiplication, the dict(list_of_pairs) constructor, del, tuple unpacking, and so on. It's fun to learn these, and they make Python a real joy to use.
Also see collections.defaultdict. If you need file I/O, read about open and file objects.
math
Seems too fundamental, but when getting started with python (lets face it, I'm still learning it) I missed some functions in the math module that would have been helpful. I ended up writing my own versions which worked but I could have saved time...
Since you ask about libraries, not specific modules in them, the standard library that comes with Python is the first and most fundamental answer; the programming challenges from a C++ book are unlikely to require anything beyond that (such as GUI toolkits) -- perhaps numpy/scipy if the book is heavily slanted to scientific programming.
The standard library, especially the built-in functions. They seem trivial but can yield impressive results!
It really pays to know the basics of a default python installation. If you doubt that just follow the Stack Overflow python questions. Some answers are just amazing :)