Develop Reference

How to correctly parse a Git command (`git log ...`) in Python?

Using a Python script, I'd like to get the email of a person who last committed changes to a specific file /path/to/file.py.
Sounds easy, right? I just need to somehow parse the following
git log -n 1 --pretty=format:%ae -- /path/to/file.py
Package sh is my preferred choice. Unfortunately, in Python
import sh
print(str(sh.git.log('-n 1 --pretty=format:%ae -- /path/to/file.py')))
print(str(sh.git.log('-n', '1', '--pretty=format:%ae', '--', /path/to/file.py')))
both print - (press RETURN).
So maybe I'm messing something up with the arguments.
Otherwise, str(sh.git.status()) correctly returns On branch master ..., and some other tested commands work as expected.
How to solve this?

The - (press RETURN) output sounds like it's something printed by a pager.
Remember, every Git command may (depending on options, arguments, configuration settings, and other environmental details such as whether stdin is a tty) run its output through a pager. The pager used depends on your personal configuration. How that pager acts depends on the pager used and on the input data.
One simple workaround is to run git --no-pager <git-command> to tell Git not to use a pager, even if the configuration and environment suggest that Git should use a pager.

This should work:
print(str(sh.git.log("-n 1", "--pretty=format:%ae", "/path/to/file")))
At least this shows how it works on my machine:
$ git log -n 1 --pretty=format:%ae -- README.md
foo#bar.com
$ python3
Python 3.6.4 (default, Jan 25 2018, 15:54:40)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sh
>>> print(str(sh.git.log("-n 1", "--pretty=format:%ae", "README.md")))
foo#bar.com

Django, uwsgi, nginx, virtualenv, ImportError: No module named site

I have a Django app and a virtual env and I try to run it under nginx+uwsgi. I've configured the whole system as is described here. I have the CentOS 7 as well.
However, I get the famous "ImportError: No module named site". No other topic here on stack overflow helped me to solve this.
In the log of uwsgi I have these two lines:
Set PythonHome to /hedgehog/.virtualenv/hedgehog
ImportError: No module named site
/etc/uwsgi/sites/hedgehog.ini:
[uwsgi]
project = hedgehog
username = hedgehog
base = /var/www/%(username)/code
chdir = /var/www/hedgehog/code
home = /%(username)/.virtualenv/%(username)
module = %(username).wsgi:application
master = true
processes = 5
uid = %(username)
socket = /run/uwsgi/%(project).sock
chown-socket = %(username):nginx
chmod-socket = 660
vacuum = true
logto = /var/www/%(username)/log/uwsgi.log
/etc/systemd/system/uwsgi.service:
[Unit]
Description=uWSGI Emperor service
[Service]
ExecStartPre=/usr/bin/bash -c 'mkdir -p /run/uwsgi; chown hedgehog:nginx/run/uwsgi'
ExecStart=/usr/bin/uwsgi --emperor /etc/uwsgi/sites
Restart=always
KillSignal=SIGQUIT
Type=notify
NotifyAccess=all
[Install]
WantedBy=multi-user.target
The app is in /var/www/hedgehog/code.
Seems that uwsgi somehow doesn't use the virtualenv. I've tried adding this to the ini file:
plugins = python
virtualenv = %(home)
It didn't help.
However, if I run "import site" in python interpreter in this virtualenv it gives no error:
[rreimche#rreimche-web sites]$ sudo -u hedgehog -H bash -l
[sudo] password for rreimche:
[hedgehog#rreimche-web sites]$ python
Python 2.7.5 (default, Jun 24 2015, 00:41:19)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import site
>>> exit()

I had the same error and a mistake:
I used a Python3 virtualenv by mistake. Once I used a Python2.7 virtualenv it worked.
Cheers

You don't really need the virtualenv entry in your config since it's the same option as home entry. /%(username)/.virtualenv/%(username) should resolve as /hedgehog/.virtualenv/hedgehog. What does ls -l /hedgehog/.virtualenv/hedgehog say?
You may also want to check that you are loading the python plugin for the very same version of python you are using creating your virtualenv.
Please paste more log, there may be more hints on what is wrong.

This also happened to me when I switch my virtualenv to use Python 3, it had been working well under Python 2.7. Here's how I solve it:
replace
plugins = python
with
plugins = python3

How to use gdb python debugging extension inside virtualenv

I'm running ubuntu, and installed the python-dbg package. When trying to use the installed version directly everything works great:
$ gdb python2.7-dbg
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
---x snipped x---
Reading symbols from /usr/bin/python2.7-dbg...done.
(gdb) r
Starting program: /usr/bin/python2.7-dbg
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Python 2.7.3 (default, Feb 27 2014, 19:39:25)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
Program received signal SIGINT, Interrupt.
0x00007ffff6997743 in __select_nocancel () at ../sysdeps/unix/syscall-template.S:82
82 ../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) py-bt (<--- works, just has nothing to do)
(gdb)
So, I've been building a virtualenv using the package's binary python2.7-dbg (since some libraries need recompiling), using this command line:
~$ virtualenv ved -p /usr/bin/python2.7-dbg
Its all working fine, but when I'm using gdb inside the virtualenv, atleast the python pretty printers stop working:
~$ . ved/bin/activate
(ved)~$ gdb python
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
---x snipped x---
Reading symbols from /home/itai/ved/bin/python...done.
(gdb) r
Starting program: /home/itai/ved/bin/python
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Python 2.7.3 (default, Feb 27 2014, 19:39:25)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
Program received signal SIGINT, Interrupt.
0x00007ffff6997743 in __select_nocancel () at ../sysdeps/unix/syscall-template.S:82
82 ../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) py-bt
Undefined command: "py-bt". Try "help". (<---- PROBLEM)
(gdb)
Am I missing something within my virtualenv?

I've solved the problem by using strace on gdb, grepping the "open" syscalls.
It seems that gdb makes a search for python-gdb.py in several paths it guesses (according to the python binary), and whenever the file is not found it just fails silently.
Eventually the way to solve the problem is to link /usr/lib/debug/usr/bin/python2.7-gdb.py into the env's bin directory. The name of the link should be <python binary name>-gdb.py, being in my case python2.7-dbg-gdb.py (...).
After that, everything seems to work.

#itai's answer only partially worked for me on Ubuntu Trusty (14.04). I found a couple of other things worked better:
sudo apt-get install python2.7-dbg
then, in the virtualenv:
. bin/activate
mkdir bin/.debug
ln -s /usr/lib/debug/usr/bin/python2.7-gdb.py bin/.debug/python-gdb.py
ln -s /usr/lib/debug/usr/bin/python2.7 bin/.debug/
gdb --args bin/python2.7 ...
This helped gdb find the python debugging symbols as well as the py-bt etc commands.

In Debian 11 with Python 3.7, gdb debugging works out of the box inside virtualenv.
Make sure that you created the venv with the same Python3 version than the python3-dbg package version installed.
In case symbols don't load correctly, rebuild the venv from scratch and try again.

On Ubuntu 12.04, #craigds's answer was very helpful but didn't get me quite all the way there: I was still running into:
IOError: invalid Python installation: unable to open /path/to/venv/lib/python2.7/config_d/Makefile (No such file or directory)
Fixed that, then I ran into:
IOError: invalid Python installation: unable to open /path/to/venv/local/include/python2.7_d/pyconfig.h (No such file or directory)
So the full steps for me to fix up my virtualenv were:
source /path/to/venv/bin/activate
mkdir /path/to/venv/bin/.debug
ln -s /usr/lib/debug/usr/bin/python2.7-gdb.py /path/to/venv/bin/.debug/python-gdb.py
ln -s /usr/lib/debug/usr/bin/python2.7 /path/to/venv/bin/.debug/
ln -s /usr/lib/python2.7/config_d/ /path/to/venv/lib/python2.7/config_d
ln -s /usr/include/python2.7_d/ /path/to/venv/local/include/python2.7_d
ln -s /usr/lib/debug/usr/bin/python2.7-gdb.py /path/to/venv/bin/python-gdb.py

Using "konsole" command to run python script

I'm trying to, from a command line, open an instance of konsole and run a python script. I'm trying:
konsole -hold -e 'python -i hello.py'
The behaviour I'm getting is that a persistent konsole opens, and I am dropped into python, but the script does not run.
Python 2.7.2+ (default, Oct 4 2011, 20:03:08)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
What do I need to do to get the python script to run in the konsole window?

jsbueno's solution is the correct one. However, as described here, you can also do something like this ...
konsole --hold -e /bin/sh -c "python -i hello.py"
P.S. you'll need to specify --workdir (before the -e arg), or provide the full path to the python script, if it's not always in the initial working dir of konsole. But, you probably already knew that.

The problem is the way "konsole" uses the parameters after the -e switch - it seems like it simply pass them in a call that does not interpret the space separators as parameter separators.
However, if you don't put your call parameters inside quotes it will work - that is, simply:
konsole --hold -e python -i hello.py
(I just tested it here)

importing cx_Oracle (python) with MacOSX

Importing cx_Oracle in a python script fails.
I have cx_Oracle installed, using "pip install cx_oracle" - that worked fine, reported installed.
Now when i try:
import cx_Oracle
I get the following error
Traceback (most recent call last):
File "reader.py", line 9, in <module>
import cx_Oracle
ImportError: dlopen(/Library/Python/2.7/site-packages/cx_Oracle.so, 2): Symbol not found: _OCIAttrGet
Referenced from: /Library/Python/2.7/site-packages/cx_Oracle.so
Expected in: flat namespace
in /Library/Python/2.7/site-packages/cx_Oracle.so
Other Information:
Python version 2.7 / mac os 10.7.2 (Lion)
$ python
Python 2.7.1 (r271:86832, Jun 16 2011, 16:59:05)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Oracle 10.2
$ sqlplus -version
SQL*Plus: Release 10.2.0.4.0 - Production
Also, I do not have a /bin directory at all in my ORACLE_HOME folder, I have only the instant client and SDK installed.
ox_Oracle
$ pip freeze
PyRSS2Gen==1.0.0
...
cx-Oracle==5.1.1
(found a lot of questions on getting cx_Oracle installed, but none on this - thanks)

I ran into this problem today and was able to solve it by changing path to the libraries referenced in the InstantClient binaries to the actual locations on the filesystem.
This blog http://blog.caseylucas.com/2013/03/03/oracle-sqlplus-and-instant-client-on-mac-osx-without-dyld_library_path/ provides detailed explanation and the script to adjust all binaries. The only problem is that it uses #executable_path , which does not seem to work anymore with Python 2.7 & El Capitan (I'm not really sure what is responsible for the security exception). Replacing #executable_path with the actual path works just fine.
To summarize, steps to make it work:
install InstantClient to /usr/local/instantclient_11_2
make sure that cx_Oracle.so shared object that you use is at /Library/Python/2.7/site-packages/cx_Oracle.so
copy the following script to /usr/local/instantclient_11_2
#!/bin/sh
# script to change the dynamic lib paths and ids for oracle instant client
# exes and libs
(echo /Library/Python/2.7/site-packages/cx_Oracle.so ; find . -maxdepth 1 -type f \( -perm -1 -o \( -perm -10 -o -perm -100 \) \) -print ) | while read exe
do
echo adjusting executable $exe
baseexe=`basename $exe`
otool -L $exe | awk '/oracle/ {print $1}' | while read lib
do
echo adjusting lib $lib
baselib=`basename $lib`
if [ "$baseexe" = "$baselib" ]
then
echo changing id to $baselib for $exe
install_name_tool -id $baselib $exe
else
echo changing path id for $lib in $exe
install_name_tool -change $lib /usr/local/instantclient_11_2/$baselib $exe
fi
done
done
run the script with root permissions.

Uninstall everything.
Then install oracle instant client:
http://digitalsanctum.com/2007/07/26/installing-oracle-instant-client-on-mac-os-x/
Then use pip to install cx_oracle.
http://www.iceycake.com/2012/02/tutorial-how-to-install-cx_oracle-python-on-mac-os-x-lion/
Then set the path to point to the 32 bit version of oracle.
edit .profile file in your home directory and add the path to your oracle bin home, using this line:
export PATH=$PATH:/usr/local/lib/instantclient/
And it works...