Develop Reference

tell pycharm that a variable will have a value [duplicate]

This question already has answers here:
Dismiss PyCharm code inspection warning for particular line of code
(2 answers)
Closed 5 months ago.
I am reading in some data with try, except blocks (to allow for managed shutdown in case of errors), this often causes Pycharm to read variables as "may be referenced before assignment" as it is apparently incapable of working out that they will either have a value or the function will be exited.
e.g.
import sys
def program_exit(code):
sys.exit(code)
def read_a(input_value):
try:
a = input_value
except:
program_exit(5)
a += 1 # referenced before assignment warning
return a
I know it is possible to block the warning for a specific line but is there any way to tell pycharm that the variable will be defined so that I don't have to block the warning for all subsequent uses of the variable
e.g.
import sys
def program_exit(code):
sys.exit(code)
def read_a(input_value):
try:
a = input_value
except:
program_exit(5)
# Pycharm: has_value(a)
a += 1 # no warning
return a
Thanks

This warning is correct, because in the try block can occur an exception, and the variable can be not defined. I see two ways of solving it:
Defining the variable: You can define the variable as None before the try/catch, so no matter what happens, the variable will always exist.
import sys
def program_exit(code):
sys.exit(code)
def read_a(input_value):
a = None
try:
a = input_value
except:
program_exit(5)
a += 1
return a
Return in the except: PyCharm does not understand that the program_exit function kills the program, but you can add a return after it, the result will be the same, but PyCharm will understand that the except block stops function execution.
import sys
def program_exit(code):
sys.exit(code)
def read_a(input_value):
try:
a = input_value
except:
program_exit(5)
return
a += 1
return a

How to raise an Exception and still continue the main code in Python

Given the following simple example:
while True:
readme = input("Write here something:")
if readme == "":
raise Exception("That was empty!")
(1) How can the main code / loop continue to work after an Exception has been thrown? (2) And if we simultaneously run another thread, how can we catch the Exception in there?
Edit: is it possible to do this without having a try/except block inside the loop?

Without using the try/catch it is not possible in python. When we call a function in python and it raises a exception, the exceptions is propagated to the caller function and it continues.
If you do not handle the the exception anywhere in the above chain, interpreter just throw it out to the user.

Yes it is possible with try/except.
Try making a function like so:
from inspect import currentframe,getframeinfo
def Raise(ex):
cf = currentframe()
#Get the line
line=cf.f_back.f_lineno
print('Traceback:\n'+ex+'\nin line '+str(line))
wait=input('Press Enter to continue excecution.')
And then call:
myexception('That was empty!)
For example (from the example):
from inspect import currentframe,getframeinfo
def Raise(ex):
cf = currentframe()
#Get the line
line=cf.f_back.f_lineno
print('Traceback:\n'+ex+'\nin line '+str(line))
wait=input('Press Enter to continue excecution.')
readme = input("Write here something:")
if readme == "":
Raise("That was empty!")

Failures with Python multiprocessing.Pool when maxtasksperchild is set

I am using Python 2.7.8 on Linux and am seeing a consistent failure in a program that uses multiprocessing.Pool(). When I set maxtasksperchild to None, then all is well, when testing across a variety of values for processes. But if I set maxtasksperchild=n (n>=1), then I invariably end with an uncaught exception. Here is the main block:
if __name__ == "__main__":
options = parse_cmdline()
subproc = Sub_process(options)
lock = multiprocessing.Lock()
[...]
pool = multiprocessing.Pool(processes=options.processes,
maxtasksperchild=options.maxtasksperchild)
imap_it = pool.imap(recluster_block, subproc.input_block_generator())
#import pdb; pdb.set_trace()
for count, result in enumerate(imap_it):
print "Count = {}".format(count)
if result is None or len(result) == 0:
# presumably error was reported
continue
(interval, block_id, num_hpcs, num_final, retlist) = result
for c in retlist:
subproc.output_cluster(c, lock)
print "About to close_outfile."
subproc.close_outfile()
print "About to close pool."
pool.close()
print "About to join pool."
pool.join()
For debugging I have added a print statement showing the number of times through the loop. Here are a couple runs:
$ $prog --processes=2 --maxtasksperchild=2
Count = 0
Count = 1
Count = 2
Traceback (most recent call last):
File "[...]reclustering.py", line 821, in <module>
for count, result in enumerate(imap_it):
File "[...]/lib/python2.7/multiprocessing/pool.py", line 659, in next
raise value
TypeError: 'int' object is not callable
$ $prog --processes=2 --maxtasksperchild=1
Count = 0
Count = 1
Traceback (most recent call last):
[same message as above]
If I do not set maxtasksperchild, the program runs to completion successfully. Also, if I uncomment the "import pdb; pdb.set_trace()" line and enter the debugger, then the problem does not appear (Heisenbug). So, am I doing something wrong in the code here? Are there conditions on the code that generates the input (subproc.input_block_generator) or the code that processes it (recluster_block), that are known to cause issues like this? Thanks!

maxtasksperchild causes multiprocessing to respawn child processes. The idea is to get rid of any cruft that is building up. The problem is, you can get new cruft from the parent. When the child respawns, it gets the current state of the parent process, which is different than the orignal spawn. You are doing your work in the script's global namespace, so you are changing the environment the child will see quite a bit. Specifically, you use a variable called 'count' that masks a previous 'from itertools import count' statement.
To fix this:
use namespaces (itertools.count, like you said in the comment) to reduce name collisions
do your work in a function so that local variables aren't propagated to the child.

Python readline, tab completion cycling with the Cmd interface

I am using the cmd.Cmd class in Python to offer a simple readline interface to my program.
Self contained example:
from cmd import Cmd
class CommandParser(Cmd):
def do_x(self, line):
pass
def do_xy(self, line):
pass
def do_xyz(self, line):
pass
if __name__ == "__main__":
parser = CommandParser()
parser.cmdloop()
Pressing tab twice will show possibilities. Pressing tab again does the same.
My question is, how do I get the options to cycle on the third tab press? In readline terms I think this is called Tab: menu-complete, but I can't see how to apply this to a Cmd instance.
I already tried:
readline.parse_and_bind('Tab: menu-complete')
Both before and after instantiating the parser instance. No luck.
I also tried passing "Tab: menu-complete" to the Cmd constructor. No Luck here either.
Anyone know how it's done?
Cheers!

The easiest trick would be to add a space after menu-complete:
parser = CommandParser(completekey="tab: menu-complete ")
The bind expression that is executed
readline.parse_and_bind(self.completekey+": complete")
will then become
readline.parse_and_bind("tab: menu-complete : complete")
Everything after the second space is acutally ignored, so it's the same as tab: menu-complete.
If you don't want to rely on that behaviour of readline parsing (I haven't seen it documented) you could use a subclass of str that refuses to be extended as completekey:
class stubborn_str(str):
def __add__(self, other):
return self
parser = CommandParser(completekey=stubborn_str("tab: menu-complete"))
self.completekey+": complete" is now the same as self.completekey.

Unfortunately, it seems as though the only way around it is to monkey-patch the method cmdloop from the cmd.Cmd class, or roll your own.
The right approach is to use "Tab: menu-complete", but that's overriden by the class as shown in line 115: readline.parse_and_bind(self.completekey+": complete"), it is never activated. (For line 115, and the entire cmd package, see this: https://hg.python.org/cpython/file/2.7/Lib/cmd.py). I've shown an edited version of that function below, and how to use it:
import cmd
# note: taken from Python's library: https://hg.python.org/cpython/file/2.7/Lib/cmd.py
def cmdloop(self, intro=None):
"""Repeatedly issue a prompt, accept input, parse an initial prefix
off the received input, and dispatch to action methods, passing them
the remainder of the line as argument.
"""
self.preloop()
if self.use_rawinput and self.completekey:
try:
import readline
self.old_completer = readline.get_completer()
readline.set_completer(self.complete)
readline.parse_and_bind(self.completekey+": menu-complete") # <---
except ImportError:
pass
try:
if intro is not None:
self.intro = intro
if self.intro:
self.stdout.write(str(self.intro)+"\n")
stop = None
while not stop:
if self.cmdqueue:
line = self.cmdqueue.pop(0)
else:
if self.use_rawinput:
try:
line = raw_input(self.prompt)
except EOFError:
line = 'EOF'
else:
self.stdout.write(self.prompt)
self.stdout.flush()
line = self.stdin.readline()
if not len(line):
line = 'EOF'
else:
line = line.rstrip('\r\n')
line = self.precmd(line)
stop = self.onecmd(line)
stop = self.postcmd(stop, line)
self.postloop()
finally:
if self.use_rawinput and self.completekey:
try:
import readline
readline.set_completer(self.old_completer)
except ImportError:
pass
# monkey-patch - make sure this is done before any sort of inheritance is used!
cmd.Cmd.cmdloop = cmdloop
# inheritance of the class with the active monkey-patched `cmdloop`
class MyCmd(cmd.Cmd):
pass
Once you've monkey-patched the class method, (or implemented your own class), it provides the correct behavior (albeit without highlighting and reverse-tabbing, but these can be implemented with other keys as necessary).

How to determine if win32api.ShellExecute was successful using hinstance?

I've been looking around for an answer to my original issue.. how do i determine (programmatically) that my win32api.ShellExecute statement executed successfully, and if a successful execution occurs, execute an os.remove() statement.
Researching I found out that the ShellExecute() call returns the HINSTANCE. Further digging I found that ShellExecute() will return an HINSTANCE > 32 if it was successful. My problem/question now is, how do i use it to control the rest of my program's flow? I tried using an if HINSTANCE> 32: statement to control the next part, but I get a NameError: name 'hinstance' is not defined message. Normally this wouldn't confuse me because it means i need to define the variable 'hinstance' before referencing it; however, because i thought ShellExecute is supposed to return HINSTANCE, i thought that makes it available for use?
Here is my full code where i am trying to implement this. Note that in my print_file() def i am assigning hinstance to the full win32api.ShellExecute() command in attempt to capture the hinstance along with explicitly returning it at the end of the function.. this isn't working either.
import win32print
import win32api
from os.path import isfile, join
import glob
import os
import time
source_path = "c:\\temp\\source\\"
def main():
printer_name = win32print.GetDefaultPrinter()
while True:
file_queue = [f for f in glob.glob("%s\\*.txt" % source_path) if isfile(f)]
if len(file_queue) > 0:
for i in file_queue:
print_file(i, printer_name)
if hinstance > 32:
time.sleep(.25)
delete_file(i)
print "Filename: %r has printed" % i
print
time.sleep(.25)
print
else:
print "No files to print. Will retry in 15 seconds"
time.sleep(15)
def print_file(pfile, printer):
hinstance = win32api.ShellExecute(
0,
"print",
'%s' % pfile,
'/d:"%s"' % printer,
".",
0
)
return hinstance
def delete_file(f):
os.remove(f)
print f, "was deleted!"
def alert(email):
pass
main()

With ShellExecute, you will never know when the printing is complete, it depends on the size of the file and whether the printer driver buffers the contents (the printer might be waiting for you to fill the paper tray, for example).
According to this SO answer, it looks like subprocess.call() is a better solution, since it waits for the command to complete, only in this case you would need to read the registry to obtain the exe associated with the file.
ShellExecuteEx is available from pywin32, you can do something like:
import win32com.shell.shell as shell
param = '/d:"%s"' % printer
shell.ShellExecuteEx(fmask = win32com.shell.shellcon.SEE_MASK_NOASYNC, lpVerb='print', lpFile=pfile, lpParameters=param)
EDIT: code for waiting on the handle from ShellExecuteEx()
import win32com.shell.shell as shell
import win32event
#fMask = SEE_MASK_NOASYNC(0x00000100) = 256 + SEE_MASK_NOCLOSEPROCESS(0x00000040) = 64
dict = shell.ShellExecuteEx(fMask = 256 + 64, lpFile='Notepad.exe', lpParameters='Notes.txt')
hh = dict['hProcess']
print hh
ret = win32event.WaitForSingleObject(hh, -1)
print ret

The return value of ShellExecute is what you need to test. You return that from print_file, but you then ignore it. You need to capture it and check that.
hinstance = print_file(i, printer_name)
if hinstance > 32:
....
However, having your print_file function leak implementation detail like an HINSTANCE seems bad. I think you would be better to check the return value of ShellExecute directly at the point of use. So try to move the > 32 check inside print_file.
Note that ShellExecute has very weak error reporting. If you want proper error reporting then you should use ShellExecuteEx instead.
Your delete/sleep loop is very brittle indeed. I'm not quite sure I can recommend anything better since I'm not sure what you are trying to achieve. However, expect to run into trouble with that part of your program.