Develop Reference

Run .py file until specified line number

In a linux terminal typing
python script.py
Will run run script.py and exit the python console, but what if I just want to run a part of the script and leave the console open? For example, run script.py until line 15 and leave the console open for further scripting. How would I do this?
Let's say it's possible, then with the console still open and script.py run until line 15, can I then from inside the console call line fragments from other py files?
...something like
python script.py 15 #(opens script and runs lines 1-15 and leaves console open)
Then having the console open, I would like to run lines 25-42 from anotherscript.py
>15 lines of python code run from script.py
> run('anotherscript.py', lines = 25-42)
> print "I'm so happy the console is still open so I can script some more")
I'm so happy the console is still open so I can script some more
>

Your best bet might be pdb, the Python debugger. You can start you script under pdb, set a breakpoint on line 15, and then run your script.
python -m pdb script.py
b 15 # <-- Set breakpoint on line 15
c # "continue" -> run your program
# will break on line 15
You can then inspect your variables and call functions. Since Python 3.2, you can also use the interact command inside pdb to get a regular Python shell at the current execution point!
If that fits your bill and you also like IPython, you can check out IPdb, which is a bit nicer than normal pdb, and drops you into an IPython shell with interact.

if you want to run script.py from line a to line b, simply use this bash snippet:
cat script.py|head -{a+b}|tail -{b-a}|python -i
replace {a+b} and {b-a} with their values

You could use the python -i option to leave the console open at the end of the script.
It lets your script run until it exits, and you can then examine variables, call any function and any Python code, including importing and using other modules.
Of course your script needs to exit first, either at the end or, if your goal is to debug that part of the script, you could add a sys.exit() or os._exit() call where you want it to stop (such as your line 15).
For instance:
import os
print "Script starting"
a=1
def f(x):
return x
print "Exiting on line 8"
os._exit(0) # to avoid the standard SystemExit exception
print "Code continuing"
Usage example:
python -i test_exit.py
Scrit starting
Exiting on line 8
>>> print a
1
>>> f(4)
4
>>>

You cannot do that directly but you can do something similar from inside Python console (or IDLE) with exec :
just open you favorite Python console
load wanted lines into a string and exec them :
script = 'script.py'
txt = ''
with open(script) as sc:
for i, line in enumerate(sc):
if i >= begline and i<= endline:
txt = txt + line
exec(txt)
You can even write your own partial script runner based on that code ...
EDIT
I must admit that above answer alone really deserved to be downvoted. It is technically correct and probably the one that most closely meet what you asked for. But I should have warned you that it is bad pratice. Relying on line numbers to load pieces of source files is error prone and you should avoid it unless you really know what you are doing and why you do it that way. Python debugger at least allows you to control what are the lines you are about to execute.
If you really have to use this solution be sure to always print and double check the lines that you are about to execute. IMHO it is always both simpler and safer to copy and past lines in an IDE like IDLE that is packaged into any standard Python installation.

Python: Is line by line execution possible

Is it possible to run code line by line with Python.
Including running any module code, when used, line by line as well.
I would like to go out and run some code line by line and watch as each of the lines goes through the processing phase and see just what code is getting executed when certain actions occur. I'm curious how certain values are getting passed off to the interpreter.

Just use python -m pdb mycode.py, which will run your code in the python debugger (pdb module).
In the debugger you can execute arbitrary code, watch variables, and jump to different places in the code. Specifically, n will execute the next line and h will show you the debugger help.

To add a breakpoint you can use pdb inline:
import pdb; pdb.set_trace()
From this line bleow you can go step by step and you can enter inside every called function (with s) or jump function execution (with n).
Also, a good tip is to use ipdb instead of pdb because it knows autocomplete (it's an ipython prompt)

Breakpoint-induced interactive debugging of Python with IPython

Say I have an IPython session, from which I call some script:
> run my_script.py
Is there a way to induce a breakpoint in my_script.py from which I can inspect my workspace from IPython?
I remember reading that in previous versions of IPython one could do:
from IPython.Debugger import Tracer;
def my_function():
x = 5
Tracer()
print 5;
but the submodule Debugger does not seem to be available anymore.
Assuming that I have an IPython session open already: how can I stop my program a location of my choice and inspect my workspace with IPython?
In general, I would prefer solutions that do not require me to pre-specify line numbers, since I would like to possibly have more than one such call to Tracer() above and not have to keep track of the line numbers where they are.

The Tracer() still exists in ipython in a different module. You can do the following:
from IPython.core.debugger import Tracer
def my_function():
x = 5
Tracer()()
print 5
Note the additional call parentheses around Tracer
edit: For IPython 6 onwards Tracer is deprecated so you should use set_trace() instead:
from IPython.core.debugger import set_trace
def my_function():
x = 5
set_trace()
print 5

You can run it and set a breakpoint at a given line with:
run -d -b12 myscript
Where -b12 sets a breakpoint at line 12. When you enter this line, you'll immediately drop into pdb, and you'll need to enter c to execute up to that breakpoint.

This is the version using the set_trace() method instead of the deprecated Tracer() one.
from IPython.core.debugger import Pdb
def my_function():
x = 5
Pdb().set_trace()
print 5

Inside the IPython shell, you can do
from IPython.core.debugger import Pdb
pdb = Pdb()
pdb.runcall(my_function)
for example, or do the normal pdb.set_trace() inside your function.

With Python 3 (v3.7+), there's the new breakpoint() function. You can modify it's behaviour so it'll call ipython's debugger for you.
Basically you can set an environment variable that points to a debugger function. (If you don't set the variable, breakpoint() defaults to calling pdb.)
To set breakpoint() to call ipython's debugger, set the environment variable (in your shell) like so:
# for bash/zsh users
export PYTHONBREAKPOINT='IPython.core.debugger.set_trace'
# powershell users
$env:PYTHONBREAKPOINT='IPython.core.debugger.set_trace'
(Note, obviously if you want to permanently set the environment variable, you'll need to modify your shell profile or system preferences.)
You can write:
def my_function():
x = 5
breakpoint()
print(5)
And it'll break into ipython's debugger for you. I think it's handier than having to import from IPython.core.debugger import set_trace and call set_trace().

I have always had the same question and the best workaround I have found which is pretty hackey is to add a line that will break my code, like so:
...
a = 1+2
STOP
...
Then when I run that code it will break, and I can do %debug to go there and inspect. You can also turn on %pdb to always go to point where your code breaks but this can be bothersome if you don't want to inspect everywhere and everytime your code breaks. I would love a more elegant solution.

I see a lot of options here, but maybe not the following simple option.
Fire up ipython in the directory where my_script.py is.
Turn the debugger on if you want the code to go into debug mode when it fails. Type %pdb.
In [1]: %pdb
Automatic pdb calling has been turned ON
Next type
In [2]: %run -d ./my_script.py
*** Blank or comment
*** Blank or comment
NOTE: Enter 'c' at the ipdb> prompt to continue execution.
> c:\users\c81196\lgd\mortgages-1\nmb\lgd\run_lgd.py(2)<module>()
1 # system imports
----> 2 from os.path import join
Now you can set a breakpoint where ever you want it.
Type b 100 to have a breakpoint at line 100, or b whatever.py:102 to have a breakpoint at line 102 in whatever.py.
For instance:
ipdb> b 100
Then continue to run, or continue.
ipdb> c
Once the code fails, or reaches the breakpoint you can start using the full power of the python debugger pdb.
Note that pdb also allows the setting of a breakpoint at a function.
b(reak) [([filename:]lineno | function) [, condition]]
So you do not necessarily need to use line numbers.

OS X & Python 3: Behavior while executing bash command in new terminal?

I've seen similar questions (e.g. Running a command in a new Mac OS X Terminal window ) but I need to confirm this command and its expected behavior in a mac (which I don't have). If anyone can run the following in Python 3 Mac:
import subprocess, os
def runcom(bashCommand):
sp = subprocess.Popen(['osascript'], stdin=subprocess.PIPE, stderr=subprocess.PIPE)
sp.communicate('''tell application "Terminal"\nactivate\ndo script with command "{0} $EXIT"\nend tell'''.format(bashCommand))
runcom('''echo \\"This is a test\\n\\nThis should come two lines later; press any key\\";read throwaway''')
runcom('''echo \\"This is a test\\"\n\necho \\"This should come one line later; press any key\\";read throwaway''')
runcom('''echo \\"This is testing whether I can have you enter your sudo pw on separate terminal\\";sudo ls;\necho \\"You should see your current directory; press any key\\";read throwaway''')
Firstly, and most basically, is the "spawn new terminal and execute" command correct? (For reference, this version of the runcom function came from this answer below, and is much cleaner than my original.)
As for the actual tests: the first one tests that internal double escaped \\n characters really work. The second tests that we can put (unescaped) newlines into the "script" and still have it work just like semicolon. Finally, the last one tests whether you can call a sudo process in a separate terminal (my ultimate goal).
In all cases, the new terminal should disappear as soon as you "press any key". Please also confirm this.
If one of these doesn't work, a correction/diagnosis would be most appreciated. Also appreciated: is there a more pythonic way of spawning a terminal on Mac then executing a (sudo, extended) bash commands on it?
Thanks!

[...] its expected behavior [...]
This is hard to answer, since those commands do what I expect them to do, which might not be what you expect them to do.
As for the actual tests: the first one tests that internal double escaped \n characters really work.
The \\n with the doubled backslash does indeed work correctly in that it causes echo to emit a newline character. However, no double quotes are emitted by echo.
The second tests that we can put (unescaped) newlines into the "script" and still have it work just like semicolon.
That works also.
Finally, the last one tests whether you can call a sudo process in a separate terminal (my ultimate goal).
There is no reason why this should not work also, and indeed it does.
In all cases, the new terminal should disappear as soon as you "press any key". Please also confirm this.
That will not work because of several reasons:
read in bash will by default read a whole line, not just one character
after the script you supply is executed, there is no reason for the shell within the terminal to exit
even if the shell would exit, the user can configure Terminal.app not to close a window after the shell exits (this is even the default setting)
Other problems:
the script you supply to osascript will appear in the terminal window before it is executed. in the examples above, the user will see every "This is a test [...]" twice.
I cannot figure out what $EXIT is supposed to do
The ls command will show the user "the current directory" only in the sense that the current working directory in a new terminal window will always be the user's home directory
throwaway will not be available after the script bashCommand exits
Finally, this script will not work at all under Python 3, because it crashes with a TypeError: communicate() takes a byte string as argument, not a string.
Also appreciated: is there a more pythonic way of spawning a terminal on Mac [...]
You should look into PyObjC! It's not necessarily more pythonic, but at least you would eliminate some layers of indirection.

I don't have Python 3, but I edited your runcom function a little and it should work:
def runcom(bashCommand):
sp = subprocess.Popen(['osascript'], stdin=subprocess.PIPE, stderr=subprocess.PIPE)
sp.communicate('''tell application "Terminal"\nactivate\ndo script with command "{0} $EXIT"\nend tell'''.format(bashCommand))

How to drop into REPL (Read, Eval, Print, Loop) from Python code

Is there a way to programmatically force a Python script to drop into a REPL at an arbitrary point in its execution, even if the script was launched from the command line?
I'm writing a quick and dirty plotting program, which I want to read data from stdin or a file, plot it, and then drop into the REPL to allow for the plot to be customized.

I frequently use this:
def interact():
import code
code.InteractiveConsole(locals=globals()).interact()

You could try using the interactive option for python:
python -i program.py
This will execute the code in program.py, then go to the REPL. Anything you define or import in the top level of program.py will be available.

Here's how you should do it (IPython > v0.11):
import IPython
IPython.embed()
For IPython <= v0.11:
from IPython.Shell import IPShellEmbed
ipshell = IPShellEmbed()
ipshell() # this call anywhere in your program will start IPython
You should use IPython, the Cadillac of Python REPLs. See http://ipython.org/ipython-doc/stable/interactive/reference.html#embedding-ipython
From the documentation:
It can also be useful in scientific
computing situations where it is
common to need to do some automatic,
computationally intensive part and
then stop to look at data, plots, etc.
Opening an IPython instance will give
you full access to your data and
functions, and you can resume program
execution once you are done with the
interactive part (perhaps to stop
again later, as many times as needed).

You can launch the debugger:
import pdb;pdb.set_trace()
Not sure what you want the REPL for, but the debugger is very similar.

To get use of iPython and functionality of debugger you should use ipdb,
You can use it in the same way as pdb, with the addition of :
import ipdb
ipdb.set_trace()

I just did this in one of my own scripts (it runs inside an automation framework that is a huge PITA to instrument):
x = 0 # exit loop counter
while x == 0:
user_input = raw_input("Please enter a command, or press q to quit: ")
if user_input[0] == "q":
x = 1
else:
try:
print eval(user_input)
except:
print "I can't do that, Dave."
continue
Just place this wherever you want a breakpoint, and you can check the state using the same syntax as the python interpreter (although it doesn't seem to let you do module imports).
It's not very elegant, but it doesn't require any other setup.

Great answers above, but if you would like this functionality in your IDE. Using Visual Studio Code (v1.5.*) with Python Setup:
Highlight the lines you would like to run and
right click and select Run Selection/Line in Interactive Window from the drop down.
Press shift + enter on your keyboard.
Right click on the Python file you want to execute in the file explorer and select Run Current File in Interactive Window
This will launch an interactive session, with linting, code completion and syntax highlighting:
Enter the code you would like to evaluate, and hit shift + enter on your keyboard to execute.
Enjoy Python!