Develop Reference

remove file command not working on spaces name

my file name is
file_name = '19-00165_my-test - Copy (7)_Basic_sample_data'
my function is like
call("rm -rf /tmp/" + file_name + '.csv', shell=True)
but getting this error
/bin/sh: -c: line 0: syntax error near unexpected token `('

My response always is: Don't use space in files.
But if you really want this, than you should place the files in quotes as such:
call("rm -f '/tmp/{0}.csv'".format(file_name), shell=True)

Why are you using shell=True? That means the command will be passed to a shell for parsing, which is what's causing all the trouble. With shell=False, you pass a list consisting of the commands followed by its arguments, each as a separate list element (rather than all mashed together as a single string). Since the filename never goes through shell parsing, it can't get mis-parsed.
call(["rm", "-rf", "/tmp/" + file_name + '.csv'], shell=False)

In order to avoid having problems with unescaped characters, one way is to use the shlex module:
You can use the quote() function to escape the string, it returns a shell-escaped version of the string:
import shlex
file_name = "19-00165_my-test - Copy (7)_Basic_sample_'data"
call(f"rm -f /tmp/{shlex.quote(file_name)}.csv", shell=True)
# rm -rf /tmp/'19-00165_my-test - Copy (7)_Basic_sample_'"'"'data'.csv
You can also use join():
import shlex
file_name = "19-00165_my-test - Copy (7)_Basic_sample_'data"
call(shlex.join(["rm", "-f", f"/tmp/{file_name}.csv"]), shell=True)
# rm -f '/tmp/19-00165_my-test - Copy (7)_Basic_sample_'"'"'data.csv'
Note: This answer is only valid if shell=True is required to make the command work. Otherwise the answer of #Gordon Davisson is way easier.

How to pass a file path out of Python into windows command prompt without escaping issues?

I am trying to set up a subprocess call to bcp, but am running into an issue with windows file paths using backslashes.
In python, a backslash is escaped by another backslash and that works fine in most circumstances.
However, since the command prompt expects a file path to use a single \
and python is passing it \\ , the command will fail.
As I am concatenating the command from multiple arguments (I need a path to the source file, format file and log file), it doesn't seem possible to do those joins without Python inserting the extra escape character.
Even os.join will store the string it creates with escape characters.
Is it possible to make windows ignore the extra backslashes or to remove them programatically?
Here's the code in question:
datacommand = ('bcp.exe ' + table_path + ' in ' + file_path
+ " -f " + format_path + " -e " + log_path + " -S "
+ self.server + instance
+ " -T -q -t" + sep + " -F" + str(int(header) + 1))
subprocess.call(datacommand)
The call is constructed in a member function of a PYODBC wrapper I have, though that's more for logical organization than being contingent on class methods or variables.
The various paths aren't hardcoded but generated further up the function.
If I print the string in the shell and copy-paste it into a command prompt it works perfectly, as the print removes the extra escape characters.

Thanks #Charles Duffy for the detailed breakdown.
As it turns out, the issue was that using subprocess.call from a python script causes hanging since it waits for the command to return something, and in the case of BCP, there is no return.
Solution is to use:
process = subprocess.Popen(command)
process.terminate()

How to change Ghostscript output file (in printer spooler) in Python

I'm trying to print a pdf file with ghostscript in Python3.
I tried to write this code in Python.
os.chdir(documents_dir)
file = "myFile.pdf"
args = '"C:\\\\Program Files\\\\gs\\\\gs9.52\\\\bin\\\\gswin64c" ' \
'-dBATCH ' \
'-dNOPAUSE ' \
'-dFitPage ' \
'-sOutputFile="%printer%{}" ' \
'-c "mark /UserSettings <</DocumentName ({})>> (mswinpr2) finddevice putdeviceprops setdevice" '.format(ptr_name, file)
ghostscript = args + os.path.abspath(file)
subprocess.call(ghostscript, shell=True)
It should print myFile.pdf so that in the printer spooler will be shown myFile.pdf as the job name.
How can I fix that command?
When the ptr_name is Microsoft Print to PDF the program ends up silently with empty PDF file.
When I used another ptr_name it ended with this error message

That's only a partial message, missing all the important text, and ite very hard to read. Rather than posting a picture of the text why not post the actual text (all of the back channel) ?
I don't speak Python, so how about stating (or indeed printing out) the contents of 'ghostscript' from your program (I'm assuming this is a string variable of some type). That would be useful for you too since it would let you see what your are tring to run and can look for errors in the string or syntax.
Clearly, if the command line works from the shell, there must be some difference in the command you are sending from Python.

Escaping a sed expression to execute over ssh from Python

I have a file on a remote server that needs to have a line updated in it. I am trying to do the update via python but appear to be having a character escape issue.
The line in the file I am trying to update is:
BEGRCVDDATE=02/01/2018 00:00 am
The line of code in python script I am using to try and make the update:
os.popen('ssh %s sed -i s/'BEGRCVDDATE=[0-9][0-9]\/[0-9][0-9]\/[0-9][0-9][0-9][0-9] [0-9][0-9]:[0-9][0-9] [a-z][a-z]'/'BEGRCVDDATE=%s'/ %s' % (ip, rcvdate, file_path))
The above code is throwing the following error:
sed: -e expression #1, char 37: unknown option to `s'
I am however able to run the sed command outside of the python script which leads me to believe this is a character escaping issue. Below works outside of the python script.
ssh <ip> "sed -i s/'BEGRCVDDATE=[0-9][0-9]\/[0-9][0-9]\/[0-9][0-9][0-9][0-9] [0-9][0-9]:[0-9][0-9] [a-z][a-z]'/'BEGRCVDDATE=BEGRCVDDATE=03\\/08\\/2018 00:00 pm'/ /tmp/test.txt"
I have tried various combinations of quotes and back-slashes to try and get around the issue I am seeing without success.
If anyone can help me resolve the issue I am seeing it would be much appreciated.
Note: due to python version in my environment os.popen is being using instead of subprocess.

import subprocess, pipes
# Generate your list of arguments *as a list of Python strings*
rcvdate = '02/01/2018 00:00 am'
cmd=['sed', '-i',
's#BEGRCVDDATE=[0-9][0-9]/[0-9][0-9]/[0-9][0-9][0-9][0-9] [0-9][0-9]:[0-9][0-9] [a-z][a-z]#BEGRCVDDATE=%s#' % (rcvdate,),
file_path,
]
# Ask Python itself to correctly form a shell command from that list
cmd_str = ' '.join([pipes.quote(s) for s in cmd])
# Pass that shell command as an argument to `ssh`.
subprocess.call(['ssh', ip, cmd_str])
...or, if you truly don't have the subprocess module, you'll need a second round of shell escaping:
ssh_cmd = ['ssh', ip, cmd_str]
ssh_cmd_str = ' '.join([pipes.quote(s) for s in ssh_cmd])
os.system(ssh_cmd_str)
Note that I changed your sed expression to use #, not /, as a sigil. This means that literal backslashes are no longer needed.

python subprocess module can't parse filename with special characters "("

I have a Python program that reads files and then tars them into tar balls of a certain size.
One of my files not only has spaces in it but also contains parentheses. I have the following code:
cmd = "/bin/tar -cvf " + tmpname + " '" + filename + "'"
NOTE: Those are single quotes inside double quotes outside of the filename variable. It's a little difficult to see.
Where tmpname and filename are variables in a for-loop that are subject to change each iteration (irrelevant).
As you can see the filename I'm tarballing contains single quotes around the file name so that the shell (bash) interprets it literally as is and doesn't try to do variable substitution which "" will do or program execution which ` will do.
As far as I can see, the cmd variable contains the exact syntax for the shell to interpret the command as I want it to. However when I run the following subprocess command substituting the cmd variable:
cmdobj = call(cmd, shell=True)
I get the following output/error:
/bin/tar: 237-r Property Transport Request (PTR) 012314.pdf: Cannot stat: No such file or directory
/bin/tar: Exiting with failure status due to previous errors
unable to tar: 237-r Property Transport Request (PTR) 012314.pdf
I even print the command out to the console before running the subprocess command to see what it will look when running in the shell and it's:
cmd: /bin/tar -cvf tempname0.tar '237-r Property Transport Request (PTR) 012314.pdf'
When I run the above command in the shell as is it works just fine. Not really sure what's going on here. Help please!

Pass a list of args without shell=True and the full path to the file if running from a different directory:
from subprocess import check_call
check_call(["tar","-cvf",tmpname ,"Property Transport Request (PTR) 012314.pdf"])
Also use tar not 'bin/tar'. check_call will raise a CalledProcessError if the command returns a non-zero exit status.

The call method that is part of the subprocess module should have an array of strings passed.
On the command line you would call
tar -cvf "file folder with space/"
The following is equivalent in python
call(["tar", "-cvf", "file folder with space/"])
You are making this call in the shell
"tar -cvf 'file folder with space/'"
Which causes the shell to look for a program with the exact name as `tar -cvf 'file folder with space/'
This avoids string concatenation, which makes for cleaner code.