I would like to get a arg.pics which returns something like ['pic1.png', 'pic2.png', 'pic3.png'] (to arbitrarily parse all files of .png format) after running the following (test.py):
import argparse
import os
def parser_arg():
par = argparse.ArgumentParser()
parser = par.add_argument_group('pictures')
parser.add_argument("-p", "--pics", nargs="+", help="picture files", required=True)
arguments = par.parse_args()
return arguments
args = parser_arg()
And upon running the script via command line, and inputting
python test.py -p ../User/Desktop/Data/*.png
then args.pics returns ['../User/Desktop/Data/*.png'] instead..
Am I using the right approach? I heard using *.png will be expanded into the .png files once inputted but it doesn't seem to be the case on my end.
Edits: I'm using Anaconda Prompt on Windows 10 if it helps.
There are a couple of things that could be going on. One possibility is that ../User/Desktop/Data/*.png does not match any files, so does not get expanded. This would happen on a UNIX-like shell only (or PowerShell I suppose). The other possibility is that you are using cmd.exe on Windows, which simply does not do wildcard expansion at all. Given that you are using Anaconda prompt on Windows, I would lean towards the latter possibility as the explanation.
Since you are looking for a list of all the PNGs in a folder, you don't need to rely on the shell at all. There are lots of ways of doing the same thing in Python, with and without integrating in argparse.
Let's start by implementing the listing functionality. Given a directory, here are some ways to get a list of all the PNGs in it:
Use glob.glob (recommended option). This can either recurse into subdirectories or not, depending on what you want:
mydir = '../User/Desktop/Data/'
pngs = glob.glob(os.path.join(mydir, '*.png'))
To recurse into subfolders, just add the recursive=True keyword-only option.
Use os.walk. This is much more flexible (and therefore requires more work), but also lets you have recursive or non-recursive solutions:
mydir = '../User/Desktop/Data/'
pngs = []
for path, dirs, files in os.walk(mydir):
pngs.extend(f for f in files if f.lower().endswith('.png'))
del dirs[:]
To enable recursion, just delete the line del dirs[:], which suppresses subdirectory search.
A related method that is always non-recursive, is to use os.listdir, which is Pythons rough equivalent to ls or dir commands:
mydir = '../User/Desktop/Data/'
pngs = [f for f in os.listdir(mydir) if f.lower().endswith('.png')]
This version does not check if something is actually a file. It assumes you don't have folder names ending in .png.
Let's say you've picked one of these methods and created a function that accepts a folder and returns a file list:
def list_pngs(directory):
return glob.glob(os.path.join(directory, '*.png'))
Now that you know how to list files in a folder, you can easily plug this into argparse at any level. Here are a couple of examples:
Just get all your directories from the argument and list them out afterwards:
parser.add_argument("-p", "--pics", action='store', help="picture files", required=True)
Once you've processed the arguments:
print(list_pngs(args.pics))
Integrate directly into argparse with the type argument:
parser.add_argument("-p", "--pics", action='store', type=list_pngs, help="picture files", required=True)
Now you can use the argument directly, since it will be converted into a list directly:
print(args.pics)
Your approach is correct. However, your script will only receive the expanded list of files as parameters if your shell supports globbing and the pattern actually matches any files. Otherwise, it will be the pattern itself in most cases.
The Anaconda Command Prompt uses cmd.exe by default, which doesn't support wildcard expansion. You could use PowerShell instead, which does understand wildcards. Alternatively, you can do the expansion in your application as described in Mad Physicist's answer.
Related
lately I started working with the Os module in python . And I finally arrived to this Os.path method . So here is my question . I ran this method in one of my kivy project just for testing and it actually didn't returned the correct output.The method consisted of finding if any directory exist and return a list of folders in the directory . otherwise print Invalid Path and return -1 . I passed in an existing directory and it returned -1 but the weird path is that when I run similar program out of my kivy project using the same path present in thesame folder as my python file it return the desired output .here is the image with the python file and the directory name image I have tested which returns invalid path.
and here is my code snippet
def get_imgs(self, img_path):
if not os.path.exists(img_path):
print("Invalid Path...")
return -1
else:
all_files = os.listdir(img_path)
imgs = []
for f in all_files:
if (
f.endswith(".png")
or f.endswith(".PNG")
or f.endswith(".jpg")
or f.endswith(".JPG")
or f.endswith(".jpeg")
or f.endswith(".JPEG")
):
imgs.append("/".join([img_path, f]))
return imgs
It's tough to tell without seeing the code with your function call. Whatever argument you're passing must not be a valid path. I use the os module regularly and have slowly learned a lot of useful methods. I always print out paths that I'm reading or where I'm writing before doing it in case anything unexpected happens, I can see that img_path variable, for example. Copy and paste the path in file explorer up to the directory and make sure that's all good.
Some other useful os.path methods you will find useful, based on your code:
os.join(<directory>, <file_name.ext>) is much more intuitive than imgs.append("/".join([img_path, f]))
os.getcwd() gets your working directory (which I print at the start of scripts in dev to quickly address issues before debugging). I typically use full paths to play it safe because Python pathing can cause differences/issues when running from cmd vs. PyCharm
os.path.basename(f) gives you the file, while os.path.dirname(f) gives you the directory.
It seems like a better approach to this is to use pathlib and glob. You can iterate over directories and use wild cards.
Look at these:
iterating over directories: How can I iterate over files in a given directory?
different file types: Python glob multiple filetypes
Then you don't even need to check whether os.path.exists(img_path) because this will read the files directly from your file system. There's also more wild cards in the glob library such as * for anything/any length, ? for any character, [0-9] for any number, found here: https://docs.python.org/3/library/glob.html
I would like to know how to get the path where the script is stored with argparse, if possible, because if I run the script from another path (I have the path of the script in the %PATH% variable) it uses by default the relative path.
I know that I can obtain it using:
import sys
sys.argv[0]
but I would like to know if it is possible to acess it directly from the argparse module.
Thanks
Edit: I have my reply and I am satisfied.
To explain better the question: I have a script called mdv.py that I use to transform markdown files into html. I would like to call it from any location in my computer.
The script is in:
c:\Python27\markdown
in this path there are other files and a folder templates that I use to generate my HTML (a default stylesheet and files for header, body and footer).
These files are in:
C:\Python\27\markdown\markdown\templates
When I call the script from a non standard path, for example c:\dropbox\public it looks in c:\dropbox\public\templates for these files and not in c:\python27\markdown\templates where they are saved.
Ihope to have better explained. Sorry I'm not a native english speaker.
I think you are looking for the prog parameter; you can interpolate sys.argv[0] into your help strings with %(prog)s.
The value for prog can be set when creating the ArgumentParser() instance; it is the first parameter:
parser = argparse.ArgumentParser('some_other_name')
and can be retrieved with the .prog attribute:
print(parser.prog) # prints "some_other_name"
However, argparsecalls os.path.basename() on this name, and does not store the directory of the program anywhere.
I'm doing a program in which Chimera needs to be opened, I'm opening it with:
def generate_files_bat(filename):
f = open(filename, 'w')
text = """echo off SET PATH=%PATH%;"C:\\Program Files (x86)\\Chimera 1.6.1\\bin" chimera colpeps.cmd"""
print >>f, text
f.close()
But I need to find Chimera apart from the computer the python program is running. Is there any way the path can be searched by the python program in any computer?
Generally speaking, I don't think it is such a good idea to search the path for a program. Imagine, for example that two different versions were installed on the machine. Are-you sure to find the right one? Maybe a configuraition file parsed with the standard module ConfigParser would be a better option?
Anyway, to go back to your question, in order to find a file or directory, you could try to use os.walk which recursively walks trough a directory tree.
Here is an example invoking os.walk from a generator, allowing you to collect either the first or all matching file names. Please note that the generator result is only based on file name. If you require more advanced filtering (say, to only keep executable files), you will probably use something like os.stat() to extend the test.
import os
def fileInPath(name, root):
for base, dirs, files in os.walk(root):
if name in files:
yield os.path.join(base, name)
print("Search for only one result:")
print(next(fileInPath("python", "/home/sylvain")))
print("Display all matching files:")
print([i for i in fileInPath("python", "/home/sylvain")])
There is which for Linux and where for Windows. They both give you the path to the executable, provided it lies in a directory that is 'searched' by the console (so it has to be in %PATH% in case of Windows)
There is a package called Unipath, that does elegant, clean path calculations.
Have look here for the AbstractPath constructor
Example:
from unipath import Path
prom_dir = Path(__file__)
So, I want to create a simple script to create directories based upon the file names contained within a certain folder.
My method looks like this:
def make_new_folders(filenames, destination):
"""
Take a list of presets and create new directories using mkdir
"""
for filename in filenames:
path = '"%s/%s/"' % (destination, filename)
subprocess.call(["mkdir", path])
For some reason I can't get the command to work.
If I pass in a file named "Test Folder", i get an error such as:
mkdir: "/Users/soundteam/Desktop/PlayGround/Test Folder: No such file or directory
Printing the 'path' variable results in:
"/Users/soundteam/Desktop/PlayGround/Test Folder/"
Can anyone point me in the right direction?
First of all, you should use os.path.join() to glue your path parts together because it works cross-platform.
Furthermore, there are built-in commands like os.mkdir or os.makedirs (which is really cool because it's recursive) to create folders. Creating a subprocess is expensive and, in this case, not a good idea.
In your example you're passing double-quotes ("destination/filename") to subprocess, which you don't have to do. Terminals need double-quotes if you use whitespaces in file or folder names, subprocess takes care of that for you.
You don't need the double quotes. subprocess passes the parameters directly to the process, so you don't need to prepare them for parsing by a shell. You also don't need the trailing slash, and should use os.path.join to combine path components:
path = os.path.join(destination, filename)
EDIT: You should accept #Fabian's answer, which explains that you don't need subprocess at all (I knew that).
I have a file, for example "something.exe" and I want to find path to this file
How can I do this in python?
Perhaps os.path.abspath() would do it:
import os
print os.path.abspath("something.exe")
If your something.exe is not in the current directory, you can pass any relative path and abspath() will resolve it.
use os.path.abspath to get a normalized absolutized version of the pathname
use os.walk to get it's location
import os
exe = 'something.exe'
#if the exe just in current dir
print os.path.abspath(exe)
# output
# D:\python\note\something.exe
#if we need find it first
for root, dirs, files in os.walk(r'D:\python'):
for name in files:
if name == exe:
print os.path.abspath(os.path.join(root, name))
# output
# D:\python\note\something.exe
if you absolutely do not know where it is, the only way is to find it starting from root c:\
import os
for r,d,f in os.walk("c:\\"):
for files in f:
if files == "something.exe":
print os.path.join(r,files)
else, if you know that there are only few places you store you exe, like your system32, then start finding it from there. you can also make use of os.environ["PATH"] if you always put your .exe in one of those directories in your PATH variable.
for p in os.environ["PATH"].split(";"):
for r,d,f in os.walk(p):
for files in f:
if files == "something.exe":
print os.path.join(r,files)
Just to mention, another option to achieve this task could be the subprocess module, to help us execute a command in terminal, like this:
import subprocess
command = "find"
directory = "/Possible/path/"
flag = "-iname"
file = "something.foo"
args = [command, directory, flag, file]
process = subprocess.run(args, stdout=subprocess.PIPE)
path = process.stdout.decode().strip("\n")
print(path)
With this we emulate passing the following command to the Terminal:
find /Posible/path -iname "something.foo".
After that, given that the attribute stdout is binary string, we need to decode it, and remove the trailing "\n" character.
I tested it with the %timeit magic in spyder, and the performance is 0.3 seconds slower than the os.walk() option.
I noted that you are in Windows, so you may search for a command that behaves similar to find in Unix.
Finally, if you have several files with the same name in different directories, the resulting string will contain all of them. In consequence, you need to deal with that appropriately, maybe using regular expressions.
This is really old thread, but might be useful to someone who stumbles across this. In python 3, there is a module called "glob" which takes "egrep" style search strings and returns system appropriate pathing (i.e. Unix\Linux and Windows).
https://docs.python.org/3/library/glob.html
Example usage would be:
results = glob.glob('./**/FILE_NAME')
Then you get a list of matches in the result variable.
Uh... This question is a bit unclear.
What do you mean "have"? Do you have the name of the file? Have you opened it? Is it a file object? Is it a file descriptor? What???
If it's a name, what do you mean with "find"? Do you want to search for the file in a bunch of directories? Or do you know which directory it's in?
If it is a file object, then you must have opened it, reasonably, and then you know the path already, although you can get the filename from fileob.name too.