How to use gdb python debugging extension inside virtualenv - python

I'm running ubuntu, and installed the python-dbg package. When trying to use the installed version directly everything works great:
$ gdb python2.7-dbg
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
---x snipped x---
Reading symbols from /usr/bin/python2.7-dbg...done.
(gdb) r
Starting program: /usr/bin/python2.7-dbg
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Python 2.7.3 (default, Feb 27 2014, 19:39:25)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
Program received signal SIGINT, Interrupt.
0x00007ffff6997743 in __select_nocancel () at ../sysdeps/unix/syscall-template.S:82
82 ../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) py-bt (<--- works, just has nothing to do)
(gdb)
So, I've been building a virtualenv using the package's binary python2.7-dbg (since some libraries need recompiling), using this command line:
~$ virtualenv ved -p /usr/bin/python2.7-dbg
Its all working fine, but when I'm using gdb inside the virtualenv, atleast the python pretty printers stop working:
~$ . ved/bin/activate
(ved)~$ gdb python
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
---x snipped x---
Reading symbols from /home/itai/ved/bin/python...done.
(gdb) r
Starting program: /home/itai/ved/bin/python
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Python 2.7.3 (default, Feb 27 2014, 19:39:25)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
Program received signal SIGINT, Interrupt.
0x00007ffff6997743 in __select_nocancel () at ../sysdeps/unix/syscall-template.S:82
82 ../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) py-bt
Undefined command: "py-bt". Try "help". (<---- PROBLEM)
(gdb)
Am I missing something within my virtualenv?

I've solved the problem by using strace on gdb, grepping the "open" syscalls.
It seems that gdb makes a search for python-gdb.py in several paths it guesses (according to the python binary), and whenever the file is not found it just fails silently.
Eventually the way to solve the problem is to link /usr/lib/debug/usr/bin/python2.7-gdb.py into the env's bin directory. The name of the link should be <python binary name>-gdb.py, being in my case python2.7-dbg-gdb.py (...).
After that, everything seems to work.

#itai's answer only partially worked for me on Ubuntu Trusty (14.04). I found a couple of other things worked better:
sudo apt-get install python2.7-dbg
then, in the virtualenv:
. bin/activate
mkdir bin/.debug
ln -s /usr/lib/debug/usr/bin/python2.7-gdb.py bin/.debug/python-gdb.py
ln -s /usr/lib/debug/usr/bin/python2.7 bin/.debug/
gdb --args bin/python2.7 ...
This helped gdb find the python debugging symbols as well as the py-bt etc commands.

In Debian 11 with Python 3.7, gdb debugging works out of the box inside virtualenv.
Make sure that you created the venv with the same Python3 version than the python3-dbg package version installed.
In case symbols don't load correctly, rebuild the venv from scratch and try again.

On Ubuntu 12.04, #craigds's answer was very helpful but didn't get me quite all the way there: I was still running into:
IOError: invalid Python installation: unable to open /path/to/venv/lib/python2.7/config_d/Makefile (No such file or directory)
Fixed that, then I ran into:
IOError: invalid Python installation: unable to open /path/to/venv/local/include/python2.7_d/pyconfig.h (No such file or directory)
So the full steps for me to fix up my virtualenv were:
source /path/to/venv/bin/activate
mkdir /path/to/venv/bin/.debug
ln -s /usr/lib/debug/usr/bin/python2.7-gdb.py /path/to/venv/bin/.debug/python-gdb.py
ln -s /usr/lib/debug/usr/bin/python2.7 /path/to/venv/bin/.debug/
ln -s /usr/lib/python2.7/config_d/ /path/to/venv/lib/python2.7/config_d
ln -s /usr/include/python2.7_d/ /path/to/venv/local/include/python2.7_d
ln -s /usr/lib/debug/usr/bin/python2.7-gdb.py /path/to/venv/bin/python-gdb.py

Related

GDB error raised: "no executable file specified"

I am trying to use gdb to debug a program that includes a module written in C++. I downloaded the latest version of gdb using homebrew.
I tried to run the following command:
(gdb) run Pyrh_test.py
However, I get the following error:
Starting program: Pyrh_test.py
No executable file specified.
Use the "file" or "exec-file" command.
Then I tried using the "file" and "exec-file" commands like the following (outputs also included):
(gdb) file Pyrh_test.py
"/Users/danielribeiro/opt/Python/Spyder/Pyrh_test.py": not in executable format: file format not recognized
(gdb) exec-file Pyrh_test.py
"/Users/danielribeiro/opt/Python/Spyder/Pyrh_test.py": not in executable format: file format not recognized
How can I use gdb to run Pyrh_test.py?
What the OP has almost works for me - just needs one tweak which is adding python to the beginning gdb command:
1. $ gdb python3
Then the run command works
2. (gdb) run py.py
Starting program: /usr/bin/python3 py.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
main started
[ ... ]
You can also test if gdb is running python by invoking directly instead of attempting to run a .py file:
$ gdb python3
GNU gdb (Ubuntu 12.0.90-0ubuntu1) 12.0.90
Copyright (C) 2022 Free Software Foundation, Inc.
[ ... ]
(gdb) run
>>> print("This is the python prompt.")
This is the python prompt.
System specs
And for reference:
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.1 LTS
Release: 22.04
Codename: jammy
$ gdb --version
GNU gdb (Ubuntu 12.0.90-0ubuntu1) 12.0.90
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ python3 --version
Python 3.10.6
How can I use gdb to run Pyrh_test.py?
You don't run Pyrh_test.py. You run Python, which interprets the .py file.
The way to debug this is to run Python under GDB, like so:
gdb --args python Pyrh_test.py
(gdb) run

How to correctly parse a Git command (`git log ...`) in Python?

Using a Python script, I'd like to get the email of a person who last committed changes to a specific file /path/to/file.py.
Sounds easy, right? I just need to somehow parse the following
git log -n 1 --pretty=format:%ae -- /path/to/file.py
Package sh is my preferred choice. Unfortunately, in Python
import sh
print(str(sh.git.log('-n 1 --pretty=format:%ae -- /path/to/file.py')))
print(str(sh.git.log('-n', '1', '--pretty=format:%ae', '--', /path/to/file.py')))
both print - (press RETURN).
So maybe I'm messing something up with the arguments.
Otherwise, str(sh.git.status()) correctly returns On branch master ..., and some other tested commands work as expected.
How to solve this?
The - (press RETURN) output sounds like it's something printed by a pager.
Remember, every Git command may (depending on options, arguments, configuration settings, and other environmental details such as whether stdin is a tty) run its output through a pager. The pager used depends on your personal configuration. How that pager acts depends on the pager used and on the input data.
One simple workaround is to run git --no-pager <git-command> to tell Git not to use a pager, even if the configuration and environment suggest that Git should use a pager.
This should work:
print(str(sh.git.log("-n 1", "--pretty=format:%ae", "/path/to/file")))
At least this shows how it works on my machine:
$ git log -n 1 --pretty=format:%ae -- README.md
foo#bar.com
$ python3
Python 3.6.4 (default, Jan 25 2018, 15:54:40)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sh
>>> print(str(sh.git.log("-n 1", "--pretty=format:%ae", "README.md")))
foo#bar.com

Python in Emacs shell-mode turns on stty echo and breaks C-d

When I run an interactive Python inside an Emacs shell buffer (M-x shell), it does two surprising things to the TTY. First, it turns on input echo, which persists after Python exits, until I do stty -echo. Secondly, it doesn't accept C-d (or C-q C-d, i.e. ^D) as EOF: I have to type quit() to leave the Python. How can I stop these two behaviours?
I know that I could run python-shell, but I don't want to: I'm noodling about in the shell and I want to do five lines of Python and then C-d out of it. So "run python-shell" is not an answer to my question.
Python running in a Terminal window is fine: ^D keeps working and echo doesn't change.
Python 2.7.5, GNU Emacs 24.3.1, OS X 10.8.5
Edited to add this snippet from a shell buffer:
bash-3.2$ echo foo
foo # no echo.
bash-3.2$ cat
foo # I typed this.
foo # cat returned it; no additional echo.
bash-3.2$ python
Python 2.7.5 (default, May 19 2013, 13:26:46)
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> # C-d has no effect. C-q C-d has no effect.
# not sure where this blank link has come from.
>>> quit() # I have to type this to get out of Python
quit() # note that it is echoed, like anything I now type.
bash-3.2$ echo foo
echo foo # now I am getting my input echoed.
foo
bash-3.2$ cat
cat # echo of the 'cat' command.
foo # my input
foo # echo of my input.
foo # cat's output.
bash-3.2$ stty -echo # turn echo back off.
stty -echo
bash-3.2$ echo foo
foo # and it's off.
bash-3.2$
0. Summary
If you installed Python via Macports, install the py27-gnureadline port (or py37-gnureadline, or whatever your version is) and the problem is fixed.
1. Reproduction
I can reproduce this (GNU Emacs 23.4.1; OS X 10.8.5; Python 3.3.2). Here's a session in a fresh emacs -Q showing the problem:
$ stty -a > stty-before
$ python3.3
Python 3.3.2 (default, May 21 2013, 11:50:47)
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> quit()
quit()
$ stty -a > stty-after
stty -a > stty-after
bash-3.2$ diff stty-before stty-after
diff stty-before stty-after
2c2
< lflags: icanon isig iexten -echo echoe -echok echoke -echonl echoctl
---
> lflags: icanon isig iexten echo echoe -echok echoke -echonl echoctl
7c7
< oflags: opost -onlcr -oxtabs -onocr -onlret
---
> oflags: opost onlcr -oxtabs -onocr -onlret
11,13c11,13
< eol2 = <undef>; erase = <undef>; intr = ^C; kill = <undef>;
< lnext = ^V; min = 1; quit = ^\; reprint = ^R; start = ^Q;
< status = ^T; stop = ^S; susp = ^Z; time = 0; werase = ^W;
---
> eol2 = <undef>; erase = ^?; intr = ^C; kill = ^U; lnext = ^V;
> min = 1; quit = ^\; reprint = ^R; start = ^Q; status = ^T;
> stop = ^S; susp = ^Z; time = 0; werase = ^W;
So you can see that Python turned on the ECHO and ONLCR flags. Why did it do that? And why does it only do that on OS X?
2. What's calling tcsetattr?
I ran Python under GDB and set a breakpoint on tcsetattr to see what's calling it. Here are the relevant parts of the backtraces:
#0 0x00007fff898e7e63 in tcsetattr ()
#1 0x00000001007cbe96 in tty_init ()
#2 0x00000001007c19cf in el_init ()
#3 0x00000001007d1bb7 in rl_initialize ()
#4 0x00000001003f10ea in PyInit_readline ()
#0 0x00007fff898e7e63 in tcsetattr ()
#1 0x00000001007cc812 in tty_rawmode ()
#2 0x00000001007c610f in read_prepare ()
#3 0x00000001007c203d in el_wset ()
#4 0x00000001007d554d in el_set ()
#5 0x00000001003f128a in call_readline ()
PyInit_readline and call_readline are functions in readline.c, but you can see from the backtraces that this is not the real readline library that is being called here, but the mostly-compatible editline library. OS X ships with the BSD-licensed editline rather than the GPL-licensed readline, so this would explain why the behaviour is different on OS X from other Unixes.
3. It's nothing to do with Python
The same thing happens with other interactive interpreters. I find that the Lua, Ruby and Sqlite3 command-line interpreters also turn on terminal echo when run inside Emacs. So it seems to be some kind of "feature" of the editline library. Let's test that theory by running this short program:
#include <readline/readline.h>
int main() {
char *line = readline("> ");
return 0;
}
and sure enough, when compiled with
$ clang rl.c -lreadline
this program also turns on terminal echo when run inside Emacs. But when compiled with
$ clang rl.c -L/opt/local/lib -lreadline
which causes it to link with the real (GNU) readline library, installed by MacPorts, it works as expected (not turning on echo).
4. Bug and workaround
So this looks like a bug in the editline library. Let's check that this really is the system version of the library, and not (say) the MacPorts version, using DYLD_PRINT_LIBRARIES:
$ export DYLD_PRINT_LIBRARIES=1
$ /usr/bin/python
dyld: loaded: /usr/bin/python
dyld: loaded: /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
[... many lines omitted ...]
dyld: loaded: /usr/lib/libstdc++.6.dylib
Python 2.6.7 (r267:88850, Oct 11 2012, 20:15:00)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
dyld: loaded: /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload/readline.so
dyld: loaded: /usr/lib/libedit.3.dylib
dyld: loaded: /usr/lib/libncurses.5.4.dylib
>>>
I have reported this to Apple as bug 15184759. I understand that Apple use the number of people reporting an issue as an indicator of the issue's severity, so please report the issue yourself if you want it to be fixed.
Now, I believe this broke in a fairly recent upgrade to OS X, so it seems likely that a recent change to libedit introduced the error. Here are the versions of libedit installed by MacPorts:
$ port installed libedit
The following ports are currently installed:
libedit #20110802-3.0_0
libedit #20120601-3.0_0
libedit #20121213-3.0_0 (active)
If I go back in time to the June 2012 version:
$ sudo port activate libedit#20120601-3.0_0
---> Computing dependencies for libedit
---> Deactivating libedit #20121213-3.0_0
---> Cleaning libedit
---> Activating libedit #20120601-3.0_0
---> Cleaning libedit
Then this fixes both problems (the terminal ECHO flag and the broken C-d) in the MacPorts versions of all the interactive interpreters I tested (Python, Ruby, Sqlite3).
So if you are looking for a workaround for your problems, this is it: use MacPorts to revert to a version of libedit before it broke, and put /opt/local/bin on your PATH so that when you type python you get the MacPorts installation of Python rather than the system one. (Possibly you do this already since I see that your Python is 2.7.5 whereas the system version is 2.6.7.)
5. Reporting to upstream
I downloaded the latest version of libedit from upstream to see if the problem has been fixed there. But it hasn't. So I contacted Jess Thrysoee and reported the bug.
6. Update
As of December 2018, the problem has not yet been fixed in libedit. However, if you are using Macports then there is a workaround (see issue #48807): you can install the pyXX-gnureadline port (where XX is your Python version, for example py27-gnureadline or py35-gnureadline) which links Python against the GNU readline library instead of libedit. Now the terminal settings are unchanged:
$ sudo port install py37-gnureadline
[...]
$ stty -a > stty-before
$ python3.7
Python 3.7.1 (default, Oct 21 2018, 09:01:26)
[Clang 10.0.0 (clang-1000.11.45.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> quit()
$ stty -a > stty-after
$ diff stty-before stty-after
$
Installing Python from MacPorts with the readline variant will work around this and other bugs in libedit.
$ sudo port install python27 +readline
I was having problems with Python 2.7.11 interleaving prompts incorrectly with output and leaving the tty in an insane state on exit. Configuring Python to use readline solved these problems.
This doesn't seem to work any more (no readline variant for the python27 port). The underlying problem, of Python leaving the stty settings broken, seems to have gone away, although it's still the case that C-d doesn't do the right thing.

Using "konsole" command to run python script

I'm trying to, from a command line, open an instance of konsole and run a python script. I'm trying:
konsole -hold -e 'python -i hello.py'
The behaviour I'm getting is that a persistent konsole opens, and I am dropped into python, but the script does not run.
Python 2.7.2+ (default, Oct 4 2011, 20:03:08)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
What do I need to do to get the python script to run in the konsole window?
jsbueno's solution is the correct one. However, as described here, you can also do something like this ...
konsole --hold -e /bin/sh -c "python -i hello.py"
P.S. you'll need to specify --workdir (before the -e arg), or provide the full path to the python script, if it's not always in the initial working dir of konsole. But, you probably already knew that.
The problem is the way "konsole" uses the parameters after the -e switch - it seems like it simply pass them in a call that does not interpret the space separators as parameter separators.
However, if you don't put your call parameters inside quotes it will work - that is, simply:
konsole --hold -e python -i hello.py
(I just tested it here)

importing cx_Oracle (python) with MacOSX

Importing cx_Oracle in a python script fails.
I have cx_Oracle installed, using "pip install cx_oracle" - that worked fine, reported installed.
Now when i try:
import cx_Oracle
I get the following error
Traceback (most recent call last):
File "reader.py", line 9, in <module>
import cx_Oracle
ImportError: dlopen(/Library/Python/2.7/site-packages/cx_Oracle.so, 2): Symbol not found: _OCIAttrGet
Referenced from: /Library/Python/2.7/site-packages/cx_Oracle.so
Expected in: flat namespace
in /Library/Python/2.7/site-packages/cx_Oracle.so
Other Information:
Python version 2.7 / mac os 10.7.2 (Lion)
$ python
Python 2.7.1 (r271:86832, Jun 16 2011, 16:59:05)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Oracle 10.2
$ sqlplus -version
SQL*Plus: Release 10.2.0.4.0 - Production
Also, I do not have a /bin directory at all in my ORACLE_HOME folder, I have only the instant client and SDK installed.
ox_Oracle
$ pip freeze
PyRSS2Gen==1.0.0
...
cx-Oracle==5.1.1
(found a lot of questions on getting cx_Oracle installed, but none on this - thanks)
I ran into this problem today and was able to solve it by changing path to the libraries referenced in the InstantClient binaries to the actual locations on the filesystem.
This blog http://blog.caseylucas.com/2013/03/03/oracle-sqlplus-and-instant-client-on-mac-osx-without-dyld_library_path/ provides detailed explanation and the script to adjust all binaries. The only problem is that it uses #executable_path , which does not seem to work anymore with Python 2.7 & El Capitan (I'm not really sure what is responsible for the security exception). Replacing #executable_path with the actual path works just fine.
To summarize, steps to make it work:
install InstantClient to /usr/local/instantclient_11_2
make sure that cx_Oracle.so shared object that you use is at /Library/Python/2.7/site-packages/cx_Oracle.so
copy the following script to /usr/local/instantclient_11_2
#!/bin/sh
# script to change the dynamic lib paths and ids for oracle instant client
# exes and libs
(echo /Library/Python/2.7/site-packages/cx_Oracle.so ; find . -maxdepth 1 -type f \( -perm -1 -o \( -perm -10 -o -perm -100 \) \) -print ) | while read exe
do
echo adjusting executable $exe
baseexe=`basename $exe`
otool -L $exe | awk '/oracle/ {print $1}' | while read lib
do
echo adjusting lib $lib
baselib=`basename $lib`
if [ "$baseexe" = "$baselib" ]
then
echo changing id to $baselib for $exe
install_name_tool -id $baselib $exe
else
echo changing path id for $lib in $exe
install_name_tool -change $lib /usr/local/instantclient_11_2/$baselib $exe
fi
done
done
run the script with root permissions.
Uninstall everything.
Then install oracle instant client:
http://digitalsanctum.com/2007/07/26/installing-oracle-instant-client-on-mac-os-x/
Then use pip to install cx_oracle.
http://www.iceycake.com/2012/02/tutorial-how-to-install-cx_oracle-python-on-mac-os-x-lion/
Then set the path to point to the 32 bit version of oracle.
edit .profile file in your home directory and add the path to your oracle bin home, using this line:
export PATH=$PATH:/usr/local/lib/instantclient/
And it works...

Categories

Resources