Develop Reference

How to source annotate python when using qcachegrind to process profilestats output

Kcachegrind serves as a wonderful utility to visually represent the hotspot to the source line level when profiling code. I found it pretty useful when micro optimizing my C++ code base.
For my latest python project I started using Kcachegrind to process the output from profilestats. Kcachegrind is a linux only utility but various unofficial ports are available and one I am using is qcachegrind.
Generally it works to a large extent and suffices for most issues except I am having a hard time getting the source annotation work.
On the source Tab I am being greeted with the familiar source missing message
There is no source available for the following function:
'main C:\Projects\module\src\source.py:397'
This is because no debug information is present
Recompile source and redo the profile run.
The function is located in the ELF Object:
'(unknown)'
Using the option
Settings -> Configure -> Source Annotation
and adding the Source Base Directory was not useful.
I have a feeling that the utility wants an ELF Object which is not relevant for Python. Any help in this regard would be useful.
Relevant Information:
Python 2.7
profilestats (2.0)
QCachegrind 0.7.4
Windows 2012R2

I second #taleniat comment. I am an OSX user. I was having some trouble getting qcachegrind to work so I ended up using pyprof2calltree and it works perfectly, source code tab included. YMMV.
First run your script with cProfile
python -m cProfile -o report.profile report.py
Then you can use pyprof2calltree to launch qcachegrind (no need for intermediate conversion).
pyprof2calltree -k -i report.profile
by the way, Python 2.7.10 and qcachegrind 0.7.4 installed via homebrew on OSX 10.11

Installing jedi-vim fails: vim says wrongly that +python is not compiled

I am trying to install jedi-vim. I am doing this at work - so I have to use gVim in Windows. I have already read several other posts in this forum. This has somehow made me smarter but still does not solve my problem.
I did the following things:
installed Pathogen
cloned jedi vim from https://github.com/davidhalter/jedi-vim.git and copied it to the bundle/ folder in the vim-dir.
Now, when I open a *.py-file vim always tells me
jedi-vim requires vim compiled with +python
but ':version' tells me that it's compiled with +python/dyn and +python3/dyn.
So what is the problem?
Additional infos:
vim version: 7.4
When I use ctrl+space to autocomplete a python key word, it tells me:
Error detected while processing function jedi#completions: line 1: E492: Not an editor command: Python jedi_vim.completions()
Error detected while processing function jedi#completions: line 1: E492: Not an editor command: Python jedi_vim.completions()
Press ENTER or type command to continue

I'm not sure if your are still experiencing the same issue, but after looking around I found this blog post that solved the problem for me. Essentially, it's just about re-installing vim (don't delete any config files) and then reinstalling from the source at the vim mercurial repo with the right flags. There might be a shorter/simpler version of doing this, but after trying back and forth many solutions, this one was the only one that worked:
Compiling Vim with Python and Ruby support on Ubuntu
These are the steps:
Check if Vim is compiled with Python:
$ vim --version | ack '(python)'
Remove Vim version installed:
$ sudo apt-get remove vim-common vim-runtime
Install dependencies needed to compile Vim:
$ sudo apt-get build-dep vim
Clone Vim repository, compile it and install the new version:
$ hg clone https://vim.googlecode.com/hg/ vim
$ cd vim
$ ./configure --enable-pythoninterp
$ make
$ sudo make install

+python/dyn means that Python support isn't statically compiled it, it's loaded dynamically when needed at runtime.
This is described in the python-dynamic help section:
On MS-Windows the Python library can be loaded dynamically. The |:version| output then includes |+python/dyn|.
This means that Vim will search for the Python DLL file only when needed.
When you don't use the Python interface you don't need it, thus you can use
Vim without this DLL file.
To use the Python interface the Python DLL must be in your search path. In a console window type "path" to see what directories are used.
The name of the DLL must match the Python version Vim was compiled with.
Currently the name is "python24.dll". That is for Python 2.4. To know for
sure edit "gvim.exe" and search for "python\d*.dll\c".
(Don't worry about the 2.4 there; I just happened to find docs from a few years ago. Read the help in your own copy of vim—or, better, do what the last paragraph says.)
So, either you don't have Python, you have the wrong version of Python, it's not on your %PATH%, vim is searching for it incorrectly, or it's failing to load.

At that time the problem was another one. I mixed up the architectures for vim and python - I used vim64Bit, but pyhton for 32bit.
I repaired that already some time ago but the other day I had to restart my computer and do some microsoft-Updates.
Unfortunately my vim (7.4) was totally broken. I had to re-set my home-dir. Then at least my vimrc was accessible again, but still my jedi in vim does not work.
Python is available in vim but still jedi does not seem to load correctly.
When I type "len(" in the beginning of a .py-file, at the place where arguments should be explained there are some strange characters, similar to this:
==jedi=0, == (_object_*) ==jedi==
When I type "self.", it says "unknown function: pythoncomplete#Complete"
I think it's only a configuration-issue, because it worked before the reboot.
Jedi is installed systemwide (with pip). If I try to install it "again", pip says that jedi is already installed.
I would appreciate your help very much ;-)

Vim compiled with Python support but can't see sys version

I compiled the development version of Vim with both Python 2 and Python 3 support. The output of vim --version has +python/dyn and +python3/dyn in it. I ran the configure file with
g
./configure --enable-pythoninterp --enable-python3interp --with-python-config-dir=/usr/lib64/python2.7/config --with-python3-config-dir=/usr/lib64/python3.3/config --with-x --with-features=huge
However when I run :python import sys; print(sys.version) I get
E448: Could not load library function _PyArg_Parse_SizeT
E263: Sorry, this command is disabled, the Python library could not be loaded.
Why would this be? I found out because of YouCompleteMe stating that it requires Vim compiled with 2.x support.
Thanks

I had a similar issue on my Debian box. If you're using a Debian-based system, you will not be able to load both Python libraries simultaneously. That's why when you set --enable-python-interp and --enable-python3-interp they always load with the /dyn suffix.
If your vim plugins don't need both versions, you should just pick one of the versions and stick with it. The links below provide more info.
Explained by Debian maintainer
Vim Python Support
P.S. - In case you tried this on Windows as well, loading either Python version will work, so the /dyn isn't an issue there.

Vim Editor->clang_complete: no python support available

Case: Testing clang_complete with gVim 7.3
I installed MinGW, then followed the tutorial here to download and compile clang. It compiled for around an hour, then make install. Clang worked. I especially love the error annotation. It's amazing.
Now comes the case of clang_complete. I Installed it. I am using pathogen, so clang_complete fron github comes in bundle folder inside vimfiles.
I opened the vim editor and gave command :scriptnames. It shows clang_complete plugin. :version shows it has python entry, so vim was built with python support.
Also my test system has python installed.
but whenever I issue :save foo1.cpp , vim give this error,
Error detected while processing function 14_ClangCompleteInit..14_initClangCompletePython:
line2
clang_complete:No python support available
line 3
cannot use clang library.
simply puzzled.
*I have python installed on my system.
I Also tried using libclang library path for clang_complete as mentioned in another question on clang_complete here, but to no avail.*
Thank you.

Vim needs to be compiled with Python support, i.e. +python when doing :version. -python means it's not installed. Taken from clang_complete at Github:
You need Vim 7.3 or higher, compiled with python support and ideally,
with the conceal feature.
You said you had this enabled, but it doesn't look like it. Taken from clang_complete.vim.
This is the only place that error message is defined and triggers on !has('python'). In other words this doesn't seem to be a clang_complete issue, but rather that your install is missing or is having problems with Python support.
function! s:initClangCompletePython()
if !has('python')
echoe 'clang_complete: No python support available.'
echoe 'Cannot use clang library'
echoe 'Compile vim with python support to use libclang'
return 0
endif
[..]

Just came across this issue, invoking :version returns for features:
-python
+python3
So it seems this is caused by incompatible plugin that requires python (e.g. 2.x) but only python 3.x is available.

How do I find out what Python libraries are installed on my Mac?

I'm just starting out with Python, and have found out that I can import various libraries. How do I find out what libraries exist on my Mac that I can import? How do I find out what functions they include?
I seem to remember using some web server type thing to browse through local help files, but I may have imagined that!

From the Python REPL (the command-line interpreter / Read-Eval-Print-Loop), type help("modules") to see a list of all your available libs.
Then to see functions within a module, do help("posix"), for example. If you haven't imported the library yet, you have to put quotes around the library's name.

For the web server, you can run the pydoc module that is included in the python distribution as a script:
python /path/to/pydoc.py -p 1234
where 1234 is the port you want the server to run at. You can then visit http://localhost:1234/ and browse the documentation.

Every standard python distribution has these libraries, which cover most of what you will need in a project.
In case you need to find out if a library exists at runtime, you do it like this
try:
import ObscureModule
except ImportError:
print "you need to install ObscureModule"
sys.exit(1) # or something like that

You can install another library: yolk.
yolk is a python package manager and will show you everything you have added via pypi. But it will also show you site-packages added through whatever local package manager you run.

just run the Python interpeter and type the command
import "lib_name"
if it gives an error, you don't have the lib installed...else you are good to go

On Leopard, depending on the python package you're using and the version number, the modules can be found in /Library/Python:
/Library/Python/2.5/site-packages
or in /Library/Frameworks
/Library/Frameworks/Python.framework/Versions/Current/lib/python2.6/site-packages
(it could also be 3.0 or whatever version)...
I guess it is quite the same with Tiger

Considering that in every operating system most of python's packages are installed using 'pip' (see pip documentation) you can also use the command 'pip freeze' on a terminal to print a list of all the packages you have installed through it.
Other tools like 'homebrew' for macOS (used when for some reason you can't install a package using pip) have similar commands, in this specific case 'brew list'.