Develop Reference

python pdb: conditional breakpoint

I want to set breakpoints in different places of my program. However they are fired if I pass a specific argument to my program, I called it STOP_LEVEL
STOP_LEVEL = 0
def mbp(sl=-1):
if sl == STOP_LEVEL:
# I would like to print which function called this
mlog.info("break point at %s, %s", ...)
breakpoint()
Then I can create a breakpoint like
mbp(2)
and if I set stop_level to 2, it's fired.
First, I would like to know are there other standard methods for such functionality? Second, I would like to know where from my mbp function was called, I mean the caller function, filename and line number. How can I include them in my function as logging information?

There isn't built in functionality like that.
The only way to find where your application was called from would be to examine the stack or to pop from your function. After popping the stack (returning from your function) you'll be at the location in your code it was called from.
To examine the stack you can use inspect.stack()
In pdb you can run the command where

I ended up to this function:
import sys
STOP_LEVEL = 0
def mbp(sl=-1):
if sl == STOP_LEVEL:
fname = sys._getframe().f_back.f_code.co_name
line = sys._getframe().f_back.f_lineno
mlog.info("break point at %s line %s",fname, line)
breakpoint()

Python function return value vs print [duplicate]

This question already has answers here:
What is the purpose of the return statement? How is it different from printing?
(15 answers)
Closed 6 months ago.
I am exceedingly new to programming, so go easy on me. I use Visual Studio Code as my editor in which I am working on a few python files. When I run the code, say for instance a pandas.dataframe.head() function, it doesn't return anything in the terminal. But when I print the same return value, I am seeing the data from a csv file as expected.
Can anyone explain what is going on?
Is it the case that when a function is run, that the data is stored but not necessarily displayed?
If so, is print the only means of displaying the value of a function when debugging?
Tried googling answers, but don't have clarity yet.
import pandas as pd
df = pd.read_csv('sample.csv')
df.head()
# print(df.head())
Expect some data to be displayed in the terminal

I believe you have learned using either Jupyter or a python console. VS Code is an IDE; it's basically a glorified text editor with features that help developers. You must be used to using python in the console where each line/command it automatically prints the results, whereas you are now likely creating a script and expect the same thing to happen. I don't believe `return has anything to do with what you're asking as it acts the same either way.
EDIT (as I found the actual documentation)
When in an interactive console python calls sys.displayhook after every execution. here's the actual documentation:
If value is not None, this function prints repr(value) to sys.stdout,
and saves value in builtins._. If repr(value) is not encodable to
sys.stdout.encoding with sys.stdout.errors error handler (which is
probably 'strict'), encode it to sys.stdout.encoding with
'backslashreplace' error handler.
sys.displayhook is called on the result of evaluating an expression
entered in an interactive Python session. The display of these values
can be customized by assigning another one-argument function to
sys.displayhook.
Here's my very basic explanation I hope I explain it well enough
In the python console each line/command's results are printed after execution (ie: when you hit enter). (For context, every function/operation implicitly returns None if nothing else is returned, therefore not printed)
When running a python script, nothing will display in the console unless explicitly printed (other cases are uncaught error tracebacks, logging, or writing to stdout, etc...)
So basically the line
df.head()
In a script performs the head function on df and returns the results but nothing happens to the results, unless you assign it to a variable or print it. It's the same as just writing:
"This will only print in a console"
If that line is executed in an interactive console it will call sys.displayhook with the value and print the results:
'This will only print in a console'
But if ran in a script it is essentially a needless line of code unless assigned to a variable.
Basically, the console assumes you want to see results as you code. (basically calling a special print at every line that doesn't print None and isn't called when print is explicitly run) Whereas when running a script it only prints to the console when explicitly asked or other special cases.

Are the first 5 rows of your 'sample.csv' file blank per chance? If not selected, the df.head() returns (https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.head.html) the first 5 rows. So:
import pandas as pd
df = pd.DataFrame({'animal':['alligator', 'bee', 'falcon', 'lion', 'monkey', 'parrot', 'shark', 'whale', 'zebra']})
print(df.head())
animal
0 alligator
1 bee
2 falcon
3 lion
4 monkey

If you want to get a value from a function and use it elsewhere, you should return that value at the end of the function definition. That way when you call the function at some point in your code and you can assign it to a variable which would store the output of the function
For instance
#test.py
def square(i):
return i*i
def main():
eight_square = square(8)
print(eight_square)
Only if you print the output you can actually see it in the terminal when you run python3 test.py. There are other ways to check, what the value is in a variable, for instance using a debugger. Visual Studio can be configured with a debugger if it isn't set up.
A breakpoint is to be set at the location where the value of the variable has to be found and the debugger has to be started.
Reference: Microsoft visual studio docs

Importing a py file within itself

This is test.py:
import sys
a = 50
b = [1,2]
def change():
print "Here 1"
import test
print "Here 2"
test.a = -1
test.b = [0,1]
return
def main():
print "Here 3"
change()
print "Here 4"
print a, b
if 1:
main()
The above python code when ran on system generates the following output:
Here 3
Here 1
Here 3
Here 1
Here 2
Here 4
-1 [0, 1]
Here 2
Here 4
50 [1, 2]
What I am confused why is not there an infinite loop of "Here 1 \n Here 3" outputs. How can the print a, b outputs can be justified?

When you run the file as a script, it is not considered to be the test module. It is considered to be the __main__ module.
When execution hits import test, a second execution of the file starts, where the module is considered to be test.
When execution hits import test again, Python recognizes that it's already importing test and does not reexecute the module. Instead, it merely loads the half-initialized test module object into the current namespace and continues on. Python's optimistic assumption is that you've written the code so that the contents of test won't be needed until the import finishes.
When execution hits the assignments to test.a and test.b, that affects the test module, but not __main__, despite the fact that they came from the same file. Thus, the print a, b from the imported module reflects the new values, while the print a, b from __main__ reflects the initial values.

A file can only be imported once. The 'import test' line succeeds the first time it is encountered. When it is encountered a second time, the interpreter will check that it has already been loaded.
When a program is initially run, it does not count as being 'imported'.

The general flow of this script is as follows:
Main is run, so it prints 'Here 3'
change is called, so it prints 'Here 1'
When importing test, python runs the main function of test
When calling change the second time, python is smart enough to know that test is already imported, so it effectively skips that line.
The imported main finishes running
The original script finishes running.

While user2367112's excellent answer explains why this happens, none of the answers here offer a workaround.
There are two easy ways to achieve the desired behavior.
Rather than importing test, use import __main__ instead. If you assign an alias with import __main__ as test, you won't even have to change any other code.
You can set sys.modules['test'] = sys.modules['__main__'] to tell python "Hey, this module already exists". After this, import test will not re-import the module, thus making your code work as expected. The relevant docs on sys.modules can be found here.

Python debugging: step INTO built-in functions and/or standard library classes [duplicate]

I want to set a breakpoint on the set.update() function, but when I try, I get an error message.
Example:
ss= set()
ss.update('a')
Breakpoint:
b set.update
b ss.update
Errors:
The specified object 'ss.update' is not a function
or was not found along sys.path.
The specified object 'set.update' is not a function
or was not found along sys.path.
(Note, I also tried with the parentheses at the end, e.g., b set.update(), but still got the error. I didn't print all the permutations of errors.)

Thanks! Using #avasal's answer and Doug Hellmann's pdb webpage, I came up with this:
Since I was trying to catch set.update, I had to edit the sets.py file, but that wasn't enough, since python was using the builtin set class rather than the one I edited. So I overwrote the builtin sets class:
import sets
locals()['__builtins__'].set=sets.Set
Then I could set conditional break points in the debugger:
b set.update, iterable=='a' #successful
b set.update, iterable=='b' #won't stop for ss.update('a')
My entire example file looks like this:
import pdb
import sets
locals()['__builtins__'].set=sets.Set
pdb.set_trace()
ss = set()
ss.update('a')
print "goodbye cruel world"
Then at the debugger prompt, enter this:
b set.update, iterable=='a'
Hope this helps others too.

Debugging code in the Python interpreter

I like testing functions in the Python interpreter. Is it possible to debug a function in the Python interpreter when I want to see more than a return value and a side effect?
If so, could you show basic debugger operations (launching the function with arguments, setting breakpoint, next step, step into, watching variable)? If not, how would you debug a function another way?
The point is, I want to debug only a particular function which will be supplied with arguments. I don't want to debug whole module code.
thank you for advice

If you want to debug specific function you can using this -
>>> import pdb
>>> import yourmodule
>>> pdb.run('yourmodule.foo()')
over the command line. pdb.set_trace() should be added in your function to break there.
More info on pdb can be seen here - http://docs.python.org/library/pdb.html

See pdb module. Insert into code:
import pdb
pdb.set_trace()
... makes a breakpoint.

The code-to-debug does not need to be modified to include pdb.set_trace(). That call can be made directly in the interpreter just before the code-to-debug:
>>> import pdb
>>> pdb.set_trace(); <code-to-debug>
For example, given test_script.py with the following code:
def some_func(text):
print 'Given text is {}'.format(repr(text))
for index,char in enumerate(text):
print ' '*index, char
an interpreter session to debug some_func using the debugger commands step-into (s), next (n) and continue (c) would look like:
>>> import pdb
>>> import test_script
>>> pdb.set_trace(); test_script.some_func('hello')
--Call--
> c:\src\test_script.py(1)some_func()
-> def some_func(text):
(Pdb) s
> c:\src\test_script.py(2)some_func()
-> print 'Given text is {}'.format(repr(text))
(Pdb) n
Given text is 'hello'
> c:\src\test_script.py(3)some_func()
-> for index,char in enumerate(text):
(Pdb) c
h
e
l
l
o
>>>
See the docs for the pdb module for more information on how to use the debugger: http://docs.python.org/library/pdb.html
Additionally, while using the debugger, the help command provides a nice list of commands and help <command> gives help specific to the given command.