Develop Reference

Remove file in python with absolute path: error "no such file or directory" but file exists

I'm trying to remove a file in Python 3 on Linux (RHEL) the following way:
os.remove(or.getcwd() + '/file.txt')
(sorry not allowed to publish the real paths).
and it gives me the usual error
No such file or directory: '/path/to/file/file.txt'
(I've respected slash or antislash in the path)
What is strange is that when I just ls the file (by copy pasting, so the very same path) the file does exist.
I've read this post but i'm not on Windows and slash direction seems correct.
Any idea ?
EDIT: as suggested by #DominicPrice os.system('ls') is showing the file while os.listdir() does not show it (but shows other files in the same directory)
EDIT 2: So my issue was due a a bad usage of os.popen. I used this method to copy file but did not wait for the subprocess to be terminated. So my understanding is that the file was not copied yet when I tried to delete it.

The problem is that, as you have explained in the comments, you are creating the file using os.popen("cp ..."). This works asynchronously, so it may not have had time to complete by the time you call os.remove(). You can force python to wait for it to finish by calling the close method:
proc = os.popen("cp myfile myotherfile")
proc.close() # wait for process to finish
os.remove("myotherfile") # we're all good
I would highly recommend staying away from using os.popen in favour of the subprocess library, which has a run function which is way safer to use.
For the specific functions of copying a file, an even better (and cross platform) solution is to use the shutil library:
import shutil
shutil.copyfile("myfile", "myotherfile")

you should use os.path.dirname(__file__).
this is an inbuilt function of os module in python.
you can read more here.
https://www.geeksforgeeks.org/find-path-to-the-given-file-using-python/

How to unit test program interacting with block devices

I have a program that interacts with and changes block devices (/dev/sda and such) on linux. I'm using various external commands (mostly commands from the fdisk and GNU fdisk packages) to control the devices. I have made a class that serves as the interface for most of the basic actions with block devices (for information like: What size is it? Where is it mounted? etc.)
Here is one such method querying the size of a partition:
def get_drive_size(device):
"""Returns the maximum size of the drive, in sectors.
:device the device identifier (/dev/sda and such)"""
query_proc = subprocess.Popen(["blockdev", "--getsz", device], stdout=subprocess.PIPE)
#blockdev returns the number of 512B blocks in a drive
output, error = query_proc.communicate()
exit_code = query_proc.returncode
if exit_code != 0:
raise Exception("Non-zero exit code", str(error, "utf-8")) #I have custom exceptions, this is slight pseudo-code
return int(output) #should always be valid
So this method accepts a block device path, and returns an integer. The tests will run as root, since this entire program will end up having to run as root anyway.
Should I try and test code such as these methods? If so, how? I could try and create and mount image files for each test, but this seems like a lot of overhead, and is probably error-prone itself. It expects block devices, so I cannot operate directly on image files in the file system.
I could try mocking, as some answers suggest, but this feels inadequate. It seems like I start to test the implementation of the method, if I mock the Popen object, rather than the output. Is this a correct assessment of proper unit-testing methodology in this case?
I am using python3 for this project, and I have not yet chosen a unit-testing framework. In the absence of other reasons, I will probably just use the default unittest framework included in Python.

You should look into the mock module (I think it's part of the unittest module now in Python 3).
It enables you to run tests without the need to depened in any external resources while giving you control over how the mocks interact with your code.
I would start from the docs in Voidspace
Here's an example:
import unittest2 as unittest
import mock
class GetDriveSizeTestSuite(unittest.TestCase):
#mock.patch('path/to/original/file.subprocess.Popen')
def test_a_scenario_with_mock_subprocess(self, mock_popen):
mock_popen.return_value.communicate.return_value = ('Expected_value', '')
mock_popen.return_value.returncode = '0'
self.assertEqual('expected_value', get_drive_size('some device'))

Deciphering large program flow in Python

I'm in the process of learning how a large (356-file), convoluted Python program is set up. Besides manually reading through and parsing the code, are there any good methods for following program flow?
There are two methods which I think would be useful:
Something similar to Bash's "set -x"
Something that displays which file outputs each line of output
Are there any methods to do the above, or any other ways that you have found useful?

I don't know if this is actually a good idea, but since I actually wrote a hook to display the file and line before each line of output to stdout, I might as well give it to you…
import inspect, sys
class WrapStdout(object):
_stdout = sys.stdout
def write(self, buf):
frame = sys._getframe(1)
try:
f = inspect.getsourcefile(frame)
except TypeError:
f = 'unknown'
l = frame.f_lineno
self._stdout.write('{}:{}:{}'.format(f, l, buf))
def flush(self):
self._stdout.flush()
sys.stdout = WrapStdout()
Just save that as a module, and after you import it, every chunk of stdout will be prefixed with file and line number.
Of course this will get pretty ugly if:
Anyone tries to print partial lines (using stdout.write directly, or print magic comma in 2.x, or end='' in 3.x).
You mix Unicode and non-Unicode in 2.x.
Any of the source files have long pathnames.
etc.
But all the tricky deep-Python-magic bits are there; you can build on top of it pretty easily.

Could be very tedious, but using a debugger to trace the flow of execution, instruction by instruction could probably help you to some extent.
import pdb
pdb.set_trace()

You could look for a cross reference program. There is an old program called pyxr that does this. The aim of cross reference is to let you know how classes refer to each other. Some of the IDE's also do this sort of thing.

I'd recommend running the program inside an IDE like pydev or pycharm. Being able to stop the program and inspect its state can be very helpful.

Python's win32api only printing to default printer

I'm trying to use win32api to output a PDF document to a particular printer.
win32api.ShellExecute(0, "print", filename, '/d:"%s"' % printername, ".", 0)
filename is a full pathname to the file, and printname is the name of the target printer I get by going through the output of win32api.EnumPrinters(6).
The file is sent to the Windows default printer even if printername is the name of a different target (my expectation is that passing a specific printer would send the named file to that printer, rather than the default).
Any hints as to what I'm doing wrong? Is there a different way of generically printing a PDF file to a specific printer? Barring all else, is there a way of temporarily changing the default printer from my program?

MikeHunter's answer was a decent starting point.
The proposed solution is calling out to Acrobat or Acrobat Reader to do the actual printing, rather than going through the win32api. For my purposes, this is sufficient:
from subprocess import call
acrobat = "C:\Program Files\Adobe\Acrobat 7.0\Acrobat.exe" ## Acrobat reader would also work, apparently
file = "C:\path\to\my\file.pdf"
printer = "Printer Name Goes Here"
call([acrobat, "/T", file, printer])
That starts up Acrobat, and prints the given file to the named printer even if it's not the Windows default. The first print job processed this way takes a few seconds (I'm assuming this is the Acrobat service being started and cached in memory), subsequent jobs print instantly. I have not done any kind of load testing on this, but I assume the call is less than trivial, so don't trust it for massive throughput.

I'm trying to print any old file to a specific printer, so these answers did not help me. However, I did find the perfect solution. Windows has a canonical verb called printto that does not show up in the context menu. It is used as a way for users to drag and drop a document onto a printer to enable printing in that manner. We can use that feature; the second argument is the name of the printer. I could never get the /d: parameter to work correctly in conjunction with the print canonical verb, but this solution solved it for me. I put the printername in quotes in case there are spaces in it.
win32api.ShellExecute(0, "printto", filename, f'"{printername}"', ".", 0)

I use SumatraPDF to achieve a similar solution (Python 3) as user Inaimathi posted:
import time
from subprocess import call
start = time.perf_counter()
sumatra = "C:\\Program Files\\SumatraPDF\\SumatraPDF.exe"
file = "C:\\Users\\spiderman\\Desktop\\report.pdf"
call([sumatra, '-print-to-default', '-silent', file])
end = time.perf_counter()
print("PDF printing took %5.9f seconds" % (end - start))
The list of command-line arguments you can pass to SumatraPDF is here.

The best way I found is:
set the default printer to the printer you need
current_printer = win32print.GetDefaultPrinter()
os.system(f"RUNDLL32 PRINTUI.DLL,PrintUIEntry /y /n {name of needed printer}")
Print file
win32api.ShellExecute(0, "print", "{document}", '/d:"{name of printer}"', ".", 0)
Restore old printer as the default
time.sleep(3)
os.system(f"RUNDLL32 PRINTUI.DLL,PrintUIEntry /y /n {current_printer}")

GDB+Python: Determining target type

Is there a way to determine whether the debugged target is a core dump or a 'live' process?

As far as I know, there is no dedicated way to do it in Python, however, you can still use
gdb.execute("<command>", to_string=<boolean>) to execute a "CLI" command in Python, where to_string being True will tell GDB to collect the output and return it as a string (cf. doc)
maint print target-stack which will print the layers used internally to access the inferior. You should see "core (Local core dump file)" if the core-debugging layer is active.
So all-in-all, a bit of code like
out = gdb.execute("maint print target-stack", to_string=True)
print "Local core dump file" in out
should do the trick.