Develop Reference

Why os.path.realpath doesn't work properly

i just downloaded a file called "N_PR_8705_004A_.doc" in my "Downloads" folder and i want to put it into my "Stage NLP" folder using os. I know how to do it without os but i'd like that shit to work it's faster and it simply doesnt. First i tried to get the path of my file doing this:
import os
os.path.dirname(os.path.abspath("N_PR_8705_004A_.doc"))
# or os.path.realpath it's the same
and the result i get is:
'C:\\Users\\f002722\\Stage NLP'
whereas when i do list all the files in this folder doing:
os.listdir("C:\\Users\\f002722\\Stage NLP")
you clearly see it is simply not there:
['.ipynb_checkpoints',
'ADR service study - D2 (1st part).pdf',
'basetal.py',
'Codes test',
'Cours NLP.ipynb',
'e Deorbit',
'edot CDF study.pdf',
'edot_v5.pdf',
'Entrainement.ipynb',
'ESA edot workshop May 6th 2014 - Summary.msg',
'ESA_edotWorkshop-_Envisat_attitude-Copy1',
'ESA_edotWorkshop-_Envisat_attitude.pdf',
'ESA_edotWorkshop_GNC_.pdf',
'ESA_INNOCENTI_Challenges.pdf',
'ESA_Robin_Biesbroek_edot.pdf',
'GMV_edot_Symposium.pdf',
'JOP_edotWorkshop.pdf',
'KT_HAARMANN_Edot.pdf',
'MDA_edot_Symposium_-_Robotic_Capture.pdf',
'MDA_eDot_Symposium_-_Robotic_Capture.pdf.kx2zd5w.partial',
'Note_Ariane_NLP.ipynb',
'Note_Ariane_NLP_2.ipynb',
'Note_Ariane_NLP_3.ipynb',
'OHB_eDotWorkshop_ADRM.pdf',
'OHB_Sweden_eDotWorkshop_PRISMA_and_IRIDES.pdf',
'SKA_Polska_eDotWorkshop_Net_Simulator.pdf',
'TAS_Carole_Billot_edot.pdf',
'Test.ipynb',
'Text_clustering_v3_2.py',
'Webinar_OOSandADR_7May2020.pdf',
'__pycache__']
So what the hell is going on i'm out of ideas here.
Thx in advance

I think I have a possible answer to your question. Neither realpath nor abspath require their arguments to name existing files. In particular, the documentation for abspath() says: "On most platforms, this is equivalent to calling the function normpath() as follows: normpath(join(os.getcwd(), path))."
This means that if you have a Python script that has a line like,
foo = os.path.dirname(os.path.abspath("doesnotexist"))
then the value of foo will be the current working directory of the script. Since "doesnotexist" isn't the name of a file in this directory, it won't show up if you do os.listdir(foo).
I notice that you wrote that "N_PR_8705_004A_.doc" was in your "Downloads" directory, which is obviously not the same as 'C:\\Users\\f002722\\Stage NLP'. If 'C:\\Users\\f002722\\Stage NLP' is the working directory for your Python script, then running os.path.dirname(os.path.abspath("N_PR_8705_004A_.doc")) is just like writing os.path.dirname(os.path.abspath("doesnotexist")), for the reasons that I just gave.
Python can't automatically figure out the path of a file just by giving it a relative file name. For example, there could be many files named README.txt on a system, each in different directories, so there's no way for os.path.abspath('README.txt') to know which of those directories you want.
To move the file "N_PR_8705_004A_.doc" from the "Downloads" directory to 'C:\\Users\\f002722\\Stage NLP', you'd probably need to do something like this:
import shutil
shutil.move('C:\\Users\\f002722\\Downloads\\N_PR_8705_004A_.doc',
'C:\\Users\\f002722\\Stage NLP')
presuming, of course, that the "Downloads" directory was inside 'C:\\Users\\f002722'.

How to pass all files (with given name patterns) to python program in PyCharm using the parameters field in run/debug configurations?

I have a bunch of .html files in a directory that I am reading into a python program using PyCharm. I am using the (*) star operator in the following way in the parameters field of the run/debug configuration dialog box in PyCharm:
*.html
, but this doesn't work. I get the following error:
IOError: [Errno 2] No such file or directory: '*.html'
at the line where I open the file to read into my program. I think its reading the "*.html" literally as a file name. I'd appreciate your help in teaching me how to use the star operator in this case.
Addendum:
I'm pretty new to Python and Pycharm. I'm running my script using the following configuration options:
Now, I've tried different variations of parameters here, like '*.html', "*.html", and just *.html. I also tried glob.glob('*.html'), but the code takes it literally and thinks that the file name itself is "glob.glob('*.html')" and throws an error. I think this is more of a Pycharm thing than understanding bash or python. I guess what I want is to make Pycharm pass all the files of the directory through that parameters field in the picture. Is there some way for me to specify to Pycharm NOT to consider the string of parameters literally?
The way the files are being handled is by running a for loop through the sys.argv list and calling a function on each file. The function simply uses the open() method to read the contents of the file into a string so I can pull patterns out of the text. Hope that fleshes out the problem a bit better.

Filename expansion is a feature of bash. So if you call your python script from the linux command line, it will work, just like if you would have typed out all of the filenames as arguments to your script. Pycharm doesn't have this feature, so you will have to do that by yourself in your python script using a glob.
import glob
import sys
files = glob.glob(sys.argv[-1])
To keep compatibility between bash and pycharm, you can use something like this:
import glob
globs = ['*.html', '*.css', 'script.js']
files = []
for g in globs:
files.extend(glob.glob(g))

I have multiple arguments so this is what I did to allow for compatibility:
I have an argparse argument that returns an array of image file names. I check it as follows.
images = args["images"]
if len(images) == 1 and '*' in images[0]:
import glob
images = glob.glob(images[0])

parse_args all .png files from a parser argument

I would like to get a arg.pics which returns something like ['pic1.png', 'pic2.png', 'pic3.png'] (to arbitrarily parse all files of .png format) after running the following (test.py):
import argparse
import os
def parser_arg():
par = argparse.ArgumentParser()
parser = par.add_argument_group('pictures')
parser.add_argument("-p", "--pics", nargs="+", help="picture files", required=True)
arguments = par.parse_args()
return arguments
args = parser_arg()
And upon running the script via command line, and inputting
python test.py -p ../User/Desktop/Data/*.png
then args.pics returns ['../User/Desktop/Data/*.png'] instead..
Am I using the right approach? I heard using *.png will be expanded into the .png files once inputted but it doesn't seem to be the case on my end.
Edits: I'm using Anaconda Prompt on Windows 10 if it helps.

There are a couple of things that could be going on. One possibility is that ../User/Desktop/Data/*.png does not match any files, so does not get expanded. This would happen on a UNIX-like shell only (or PowerShell I suppose). The other possibility is that you are using cmd.exe on Windows, which simply does not do wildcard expansion at all. Given that you are using Anaconda prompt on Windows, I would lean towards the latter possibility as the explanation.
Since you are looking for a list of all the PNGs in a folder, you don't need to rely on the shell at all. There are lots of ways of doing the same thing in Python, with and without integrating in argparse.
Let's start by implementing the listing functionality. Given a directory, here are some ways to get a list of all the PNGs in it:
Use glob.glob (recommended option). This can either recurse into subdirectories or not, depending on what you want:
mydir = '../User/Desktop/Data/'
pngs = glob.glob(os.path.join(mydir, '*.png'))
To recurse into subfolders, just add the recursive=True keyword-only option.
Use os.walk. This is much more flexible (and therefore requires more work), but also lets you have recursive or non-recursive solutions:
mydir = '../User/Desktop/Data/'
pngs = []
for path, dirs, files in os.walk(mydir):
pngs.extend(f for f in files if f.lower().endswith('.png'))
del dirs[:]
To enable recursion, just delete the line del dirs[:], which suppresses subdirectory search.
A related method that is always non-recursive, is to use os.listdir, which is Pythons rough equivalent to ls or dir commands:
mydir = '../User/Desktop/Data/'
pngs = [f for f in os.listdir(mydir) if f.lower().endswith('.png')]
This version does not check if something is actually a file. It assumes you don't have folder names ending in .png.
Let's say you've picked one of these methods and created a function that accepts a folder and returns a file list:
def list_pngs(directory):
return glob.glob(os.path.join(directory, '*.png'))
Now that you know how to list files in a folder, you can easily plug this into argparse at any level. Here are a couple of examples:
Just get all your directories from the argument and list them out afterwards:
parser.add_argument("-p", "--pics", action='store', help="picture files", required=True)
Once you've processed the arguments:
print(list_pngs(args.pics))
Integrate directly into argparse with the type argument:
parser.add_argument("-p", "--pics", action='store', type=list_pngs, help="picture files", required=True)
Now you can use the argument directly, since it will be converted into a list directly:
print(args.pics)

Your approach is correct. However, your script will only receive the expanded list of files as parameters if your shell supports globbing and the pattern actually matches any files. Otherwise, it will be the pattern itself in most cases.
The Anaconda Command Prompt uses cmd.exe by default, which doesn't support wildcard expansion. You could use PowerShell instead, which does understand wildcards. Alternatively, you can do the expansion in your application as described in Mad Physicist's answer.

What is 'subdir' defined var in python 3 for all OS?

I have a script in Python which i get 3 arguments from the user
One of the arguments is a folder path in which there are some files i need to use
Since my program is designed for all OS, i would like to know how to use the path from the argument correctly to get to my required files
I.E.
if i get the following path :
c:\windows
i would like to be able to get 1.exe in this folder,
In windows it will be slash or backslash but in unix systems it will be probably different,
As i understand there is a const or defined var from 'os' module in which i can use as this subdir sign, where can i find it ?
Thank you

Just use os.path.join and Python will take care of the slash for you:
path = os.path.join(sys.argv[1], '1.exe')
The platform-specific path separator is stored as os.sep.

Use os.path.join(). For example:
os.path.join(dirname, "1.exe")

Passing commands to OS: What is wrong here?

So, I want to create a simple script to create directories based upon the file names contained within a certain folder.
My method looks like this:
def make_new_folders(filenames, destination):
"""
Take a list of presets and create new directories using mkdir
"""
for filename in filenames:
path = '"%s/%s/"' % (destination, filename)
subprocess.call(["mkdir", path])
For some reason I can't get the command to work.
If I pass in a file named "Test Folder", i get an error such as:
mkdir: "/Users/soundteam/Desktop/PlayGround/Test Folder: No such file or directory
Printing the 'path' variable results in:
"/Users/soundteam/Desktop/PlayGround/Test Folder/"
Can anyone point me in the right direction?

First of all, you should use os.path.join() to glue your path parts together because it works cross-platform.
Furthermore, there are built-in commands like os.mkdir or os.makedirs (which is really cool because it's recursive) to create folders. Creating a subprocess is expensive and, in this case, not a good idea.
In your example you're passing double-quotes ("destination/filename") to subprocess, which you don't have to do. Terminals need double-quotes if you use whitespaces in file or folder names, subprocess takes care of that for you.

You don't need the double quotes. subprocess passes the parameters directly to the process, so you don't need to prepare them for parsing by a shell. You also don't need the trailing slash, and should use os.path.join to combine path components:
path = os.path.join(destination, filename)
EDIT: You should accept #Fabian's answer, which explains that you don't need subprocess at all (I knew that).