I would like to provide database for my program that uses elixir for ORM. Right now the database file (I am using SQLite) must be hardcoded in metadata, but I would like to be able to pass this in argv. Is there any way to do this nice?
The only thing I thought of is to:
from sys import argv
metadata.bind = argv[1]
Can I set this in the main script and it would be used in all modules, that define any Entities?
I have some code that does this in a slightly nicer fashion than just using argv
from optparse import OptionParser
parser = OptionParser()
parser.add_option("-u", "--user", dest="user",
help="Database username")
parser.add_option("-p", "--password", dest="password",
help="Database password")
parser.add_option("-D", "--database", dest="database", default="myDatabase",
help="Database name")
parser.add_option("-e", "--engine", dest="engine", default="mysql",
help="Database engine")
parser.add_option("-H", "--host", dest="host", default="localhost",
help="Database host")
(options, args) = parser.parse_args()
def opt_hash(name):
global options
return getattr(options, name)
options.__getitem__ = opt_hash
metadata.bind = '%(engine)s://%(user)s:%(password)s#%(host)s/%(database)s' % options
Note that the part using opt_hash is a bit of a hack. I use it because OptionParser doesn't return a normal hash, which is what is really needed for the niceness of the bind string I use in the last line.
Your question seems to be more related to general argument parsing in python than with elixir.
Anyway, I had a similar problem, and I have solved it by using different configuration files and parsing them with the configparse module in python.
For example, I have two config files, and each of them describes the db url, username, password, etc.. of one database. When I want to switch to another configuration, I pass an option like --configfile guest to the script (I use argparse for the command line interface), then the script looks for a config file called guest.txt, and reads all the information there.
This is a lot safer, because if you pass a metadata string as a command line argument you can have some security issues, and moreover it is a lot longer to type.
By the way, you can also find useful to write a Makefile to store the most common options.
e.g. cat >Makefile
debug_db:
ipython connect_db.py -config guest -i
connect_root:
ipython connect_db.py -config db1_root -i
connect_db1:
ipython connect_db.py -config db1 -i
and on the command line, you only have to type 'make debug_db' or 'make connect_db1' to execute a rule.
Related
I am new to argparser in python . I am trying to create argparser for file which contains two functions download and upload file on/from box. It will do only one functionality at once according according to that i am trying to create parser for that file as follows but it's not working for me:
parser = argparse.ArgumentParser(description='Download or Upload file on box.')
parser.add_argument('-df', '--download', required=True,
help='download file box')
parser.add_argument('-uf', '--upload', nargs='+', required=True,
help='upload file of box')
parser.add_argument('-fp', '--filepath', required=True,
help='file path to upload(which file to upload) or download(where to download file)')
parser.add_argument('-fn', '--filename', required=True,
help='what should be the name of file on box')
parser.add_argument('-fi', '--fileid', required=True,
help='file id of file to download from box')
args = vars(parser.parse_args())
NOTE :- every time only -df or -uf options will be there, -fp is mandatory for both and if it is -df then -fi is only option and it is mandatory and if -uf then -fn is only option and it's mandatory.
How do I achieve this, following are example how will i pass argument to file
pyhton abc.py -df -fp 'Download/boxfile/xyz.txt' -fi 123
python abc.py -uf -fp 'Download/boxfile/xyz.txt' -fn 'qwe.txt'
As written all 5 of the arguments are required - you made that explicit. If that's what you really want, all the rest of the question is irrelevant. You'll have to provide all 5 regardless.
But the comments indicate that you want to use either -df or -uf, but probably not both (though that bit's unclear).
While there is a mutually_exclusive_mechanism in argparse, there isn't a inclusive equivalent - something that says if -f is present, then -g must also be given.
But subparsers mechanism can be used that way. You could define an download subparser, with a required -fi argument. And an upload with its own argument.
Another option is to set -df to take 2 arguments (nargs=2), both its box and its file.
If -df and -uf are mutually exclusive, why not use the same argument for the name of the file? That is, replace -fn and -fi with one file argument.
Another option is to make all (or most) of the arguments not-required, and check for the correct combinations after parsing. It's easier to implement complicated logic in your own code than to force argparse to do it for you.
e.g.
if args.download is not None: # not default
<check for `args.filename`> etc
It can also be a good idea to provide defaults for optional arguments. That way the code can run even if the user doesn't provide all items.
On a matter of style. Short option flags, with on - are usually a single character. -d, -u, etc. The ability to combine several into one string only works in that case, e.g. -df foo. In this case it probably doesn't matter since none of your arguments are store_true.
I'm not incredibly familiar with argparse, however from reading the documentation I don't believe there's a way to use mutually_exclusive_group and required to force this. It seems conditional statements or equivalent are necessary to confirm that valid parameter combinations were passed.
See: Python Argparse conditionally required arguments
I'd suggest that you get rid of most of these arguments, maybe all, and think about standard Unix ways of doing things.
Consider scp, which has a syntax of: scp source destination
For instance: scp Download/boxfile/xyz.txt qwe.txt
If you don't supply a destination, it infers that you want the file to be called the same thing, and land right here, so these two things are equivalent:
scp Download/boxfile/xyz.txt xyz.txt
scp Download/boxfile/xyz.txt
Of course, scp can talk to machines across the internet, so there is a format for that:
scp hostname:Download/boxfile/xyz.txt xyz.txt
And if you are uploading, you simple switch the order:
scp xyz.txt hostname:Download/boxfile/xyz.txt
I have a python script that I'd like to supply with an argument (usually) containing wildcards, referring to a series of files that I'd like to do stuff with. Example here:
#!/usr/bin/env python
import argparse
import glob
parser = argparse.ArgumentParser()
parser.add_argument('-i', action="store", dest="i")
results = parser.parse_args()
print 'argument i is: ', results.i
list_of_matched_files = glob.glob(results.i)
In this case, everything works great if the user adds quotes to the passed argument like so:
./test_script.py -i "foo*.txt"
...but often times the users forget to add quotes to the argument and are stumped when the list only contains the first match because UNIX already expanded the list and argparse only then gets the first list element.
Is there a way (within the script) to prevent UNIX from expanding the list before passing it to python? Or maybe even just to test if the argument doesn't contain quotes and then warn the user?
No. Wildcards are expanded by the shell (Bash, zsh, csh, fish, whatever) before the script even runs, and the script can't do anything about them. Testing whether the argument contains quotes also won't work, as the shell similarly strips the quotes from "foo*.txt" before passing the argument to the script, so all Python sees is foo*.txt.
Its not UNIX that is doing the expansion, it is the shell.
Bash has an option set -o noglob (or -f) which turns off globbing (filename expansion), but that is non-standard.
If you give an end-user access to the command-line then they really should know about quoting. For example, the commonly used find command has a -name parameter which can take glob constructs but they have to be quoted in a similar manner. Your program is no different to any other.
If users can't handle that then maybe you should give them a different interface. You could go to the extreme of writing a GUI or a web/HTML front-end, but that's probably over the top.
Or why not prompt for the filename pattern? You could, for example, use a -p option to indicate prompting, e.g:
import argparse
import glob
parser = argparse.ArgumentParser()
parser.add_argument('-i', action="store", dest="i")
parser.add_argument('-p', action="store_true", default=False)
results = parser.parse_args()
if results.p:
pattern = raw_input("Enter filename pattern: ")
else:
pattern = results.i
list_of_matched_files = glob.glob(pattern)
print list_of_matched_files
(I have assumed Python 2 because of your print statement)
Here the input is not read by the shell but by python, which will not expand glob constructs unless you ask it to.
You can disable the expansion using set -f from the command line. (re-enable with set +f).
As jwodder correctly says though, this happens before the script is run, so the only way I can think of to do this is to wrap it with a shell script that disables expansion temporarily, runs the python script, and re-enables expansion. Preventing UNIX from expanding the list before passing it to python is not possible.
Here is an example for the Bash shell that shows what #Tom Wyllie is talking about:
alias sea='set -f; search_function'
search_function() { perl /home/scripts/search.pl $# ; set +f; }
This defines an alias called "sea" that:
Turns off expansion ("set -f")
Runs the search_function function which is a perl script
Turns expansion back on ("set +f")
The problem with this is that if a user stops execution with ^C or some such then the expansion may not be turned back on leaving the user puzzling why "ls *" is not working. So I'm not necessarily advocating using this. :).
This worked for me:
files = sys.argv[1:]
Even though only one string is on the command line, the shell expands the wildcards and fills sys.argv[] with the list.
So I wrote a Python 3 library, which serves as an application 'backend'. Now I can sit down with the interpreter, import the source file(s), and hack around using the lib - I know how to do this.
But I would also like to build a command line 'frontent' application using the library. My library defines a few objects which have high-level commands, which should be visible by the application. Such commands may return some data structures and the high-level commands would print them nicely. In other words, the command line app would be a thin wrapper around the lib, passing her input to the library commands, and presenting results to the user.
The best example of what I'm trying to achieve would probably be Mercurial SCM, as it is written in Python and the 'hg' command does what I'm looking for - for instance, 'hg commit -m message' will find the code responsible for the 'commit' command implementation, pass the arguments from the user and do its work. On the way back, it might get some results and print them out nicely.
Is there a general way of doing it in Python? Like exposing classes/methods/functions as 'high level' commands with an annotation? Does anybody know of any tutorials?
You can do this with argparse. For example here is the start of my deploy script.
def main(argv):
"""
Entry point for the deploy script.
Arguments:
argv: All command line arguments save the name of the script.
"""
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('-v', '--verbose', action='store_true',
help='also report if files are the same')
parser.add_argument('-V', '--version', action='version',
version=__version__)
parser.add_argument('command', choices=['check', 'diff', 'install'])
fname = '.'.join(['filelist', pwd.getpwuid(os.getuid())[0]])
args = parser.parse_args(argv)
It uses an argument with choices to pick a function. You could define a dictionary mapping choices to functions;
cmds = {'check': do_check, 'diff': do_diff, 'install': do_install}
fn = cmds[args.command]
If you make sure that all the dict keys are in the command choices, you don't need to catch KeyError.
I know how to detect if my Python script's stdout is being redirected (>) using sys.stdout.isatty() but is there a way to discover what it's being redirected to?
For example:
python my.py > somefile.txt
Is there a way to discover the name somefile.txt on both Windows and Linux?
I doubt you can do that in a system-independent way. On Linux, the following works:
import os
my_output_file = os.readlink('/proc/%d/fd/1' % os.getpid())
If you need a platform-independent way to get the name of the file, pass it as an argument and use argparse (or optparse) to read your arguments, don't rely on shell redirection at all.
Use python my.py --output somefile.txt with code such as:
parser = argparse.ArgumentParser()
parser.add_argument('--output', # nargs='?', default=sys.stdout,
type=argparse.FileType('w'),
help="write the output to FILE",
metavar="FILE")
args = parser.parse_args()
filename = args.output.name
If knowing the name is optional and used for some weird optimization, then use Igor Nazarenko's solution and check that sys.platform is 'linux2', otherwise assume that you don't have the name and treat it as a normal pipe.
if __name__=='__main__':
parser = OptionParser()
parser.add_option("-i", "--input_file",
dest="input_filename",
help="Read input from FILE", metavar="FILE")
(options, args) = parser.parse_args()
print options
result is
$ python convert.py -i video_*
{'input_filename': 'video_1.wmv'}
there are video_[1-6].wmv in the current folder.
Question is why video_* become video_1.wmv. What i'm doing wrong?
Python has nothing to do with this -- it's the shell.
Call
$ python convert.py -i 'video_*'
and it will pass in that wildcard.
The other six values were passed in as args, not attached to the -i, exactly as if you'd run python convert.py -i video_1 video_2 video_3 video_4 video_5 video_6, and the -i only attaches to the immediate next parameter.
That said, your best bet might to be just read your input filenames from args, rather than using options.input.
Print out args and you'll see where the other files are going...
They are being converted to separate arguments in argv, and optparse only takes the first one as the value for the input_filename option.
To clarify:
aprogram -e *.wmv
on a Linux shell, all wildcards (*.wmv) are expanded by the shell. So aprogram actually recieves the arguments:
sys.argv == ['aprogram', '-e', '1.wmv', '2.wmv', '3.wmv']
Like Charles said, you can quote the argument to get it to pass in literally:
aprogram -e "*.wmv"
This will pass in:
sys.argv == ['aprogram', '-e', '*.wmv']
It isn't obvious, even if you read some of the standards (like this or this).
The args part of a command line are -- almost universally -- the input files.
There are only very rare odd-ball cases where an input file is specified as an option. It does happen, but it's very rare.
Also, the output files are never named as args. They almost always are provided as named options.
The idea is that
Most programs can (and should) read from stdin. The command-line argument of - is a code for "stdin". If no arguments are given, stdin is the fallback plan.
If your program opens any files, it may as well open an unlimited number of files specified on the command line. The shell facilitates this by expanding wild-cards for you. [Windows doesn't do this for you, however.]
You program should never overwrite a file without an explicit command-line options, like '-o somefile' to write to a file.
Note that cp, mv, rm are the big examples of programs that don't follow these standards.