How to source annotate python when using qcachegrind to process profilestats output - python

Kcachegrind serves as a wonderful utility to visually represent the hotspot to the source line level when profiling code. I found it pretty useful when micro optimizing my C++ code base.
For my latest python project I started using Kcachegrind to process the output from profilestats. Kcachegrind is a linux only utility but various unofficial ports are available and one I am using is qcachegrind.
Generally it works to a large extent and suffices for most issues except I am having a hard time getting the source annotation work.
On the source Tab I am being greeted with the familiar source missing message
There is no source available for the following function:
'main C:\Projects\module\src\source.py:397'
This is because no debug information is present
Recompile source and redo the profile run.
The function is located in the ELF Object:
'(unknown)'
Using the option
Settings -> Configure -> Source Annotation
and adding the Source Base Directory was not useful.
I have a feeling that the utility wants an ELF Object which is not relevant for Python. Any help in this regard would be useful.
Relevant Information:
Python 2.7
profilestats (2.0)
QCachegrind 0.7.4
Windows 2012R2

I second #taleniat comment. I am an OSX user. I was having some trouble getting qcachegrind to work so I ended up using pyprof2calltree and it works perfectly, source code tab included. YMMV.
First run your script with cProfile
python -m cProfile -o report.profile report.py
Then you can use pyprof2calltree to launch qcachegrind (no need for intermediate conversion).
pyprof2calltree -k -i report.profile
by the way, Python 2.7.10 and qcachegrind 0.7.4 installed via homebrew on OSX 10.11

Related

Python with --enable-shared on cloudfoundry

I would like to deploy an application on a cloudfoundry instance, consisting of an Rmarkdown file which should contain both R and python chunks. I'd like to render the document on cloudfoundry, rather than doing so locally and just pushing up the HTML file.
For knitr to work with python, it uses the reticulate package which states in the docs:
Note that for reticulate to bind to a version of Python it must be compiled with shared library support (i.e. with the --enable-shared flag).
I tried to use the cloudfoundry python buildpack to provide python for reticulate, but got the error Python shared library not found, Python bindings not loaded, and reticulate::py_discover_config() on the cloudfoundry app gives:
> reticulate::py_discover_config()
python: /home/vcap/deps/2/bin/python/ # or something very like this
libpython: [NOT FOUND]
which I think suggests that python hasn't been compiled with --enable-shared.
Tantalisingly, the changelog has this line * Add --enable-shared to Python runtime options from 6 years ago, but it looks to be for a specific version that is no longer relevant.
Is there a way I can get a version of python on cloudfoundry which will work with reticulate? Ideally without having to build it from scratch myself...
I have found that I can install a version of python that reticulate is happy to work with, using reticulate itself. Ahead of rendering the Rmarkdown document, I can use:
reticulate::install_miniconda()
reticulate::py_install(readLines("requirements.txt"))
This creates a conda environment with all my requirements and allows my code to run.
The disadvantage is that it shifts the python installation from the app staging (which has 15 minutes to complete by default on cloudfoundry), to the app start command (which has 60s by default). I had to increase the default timeout to 120s to give enough time to install the requirements and render the document.
Following #DanielMikusa's comment, I have also raised a feature request on the python buildpack, to hopefully enable shared library support in future, but this works for now for me.

vienna RNA fold python installation issue

I'm having some issues installing the python wrapper for the vienna RNA package (found here). I previously had been able to install this program's python implementation by moving the compiled python scripts into my pythonpath (the program doesn't seem to want to do it no matter what I do). However, this time I can't seem to get the darn thing to make the python implementation.
previously, I had to configure using the following command:
./configure --disable-openmp \
--disable-dependency-tracking \
CFLAGS="-arch i386 -arch x86_64 -O2" \
CXXFLAGS="-arch i386 -arch x86_64 -O2"\
--prefix=/usr/local\
--with-python
but even with this, the post-configure message says:
##############################################
# Files will be installed in the following ##
# directories: ##
##############################################
Executables: /usr/local/bin
Libraries: /usr/local/lib
Header files: /usr/local/include
Extra Data: /usr/local/share
Man pages: /usr/local/share/man
Documentation: /usr/local/share/doc/ViennaRNA
(HTML): /usr/local/share/doc/ViennaRNA/html
(PDF): /usr/local/share/doc/ViennaRNA
Perl5 Interface:
(binaries): /usr/local/Library/Perl/5.18/darwin-thread-multi-2level
(scripts): /usr/local/Library/Perl/5.18
Python2 Interface: Not to be installed
(binaries):
(scripts):
Python3 Interface: Not to be installed
(binaries):
(scripts):
I went ahead and did make and make install. The program is installed (e.g. I can start it up when I type in RNAfold into the terminal window), however I can't seem to find the python implementation anywhere. After making, there are a few files in the "interfaces" folder named RNA.py and RNA_wrap.cpp (which was produced with SWIG). However, I can't seem to get these to run even if I start a python shell in that folder and type import RNA
Sorry I can't be of more use. Please let me know if you need more information about this particular situation. I'm running a late 2011 macbook pro and OSX el-capitan.
THANKS!
I don't know if you solved this problem yet or not, but I figured it out.
After opening the .gz folder, opening the terminal, and then setting the working directory to the root of the Vienna-2.3.1 package, you should run:
./configure --enable-macosx-installer
make
sudo make install
and then run the command "make check" to see if any compilations that you ran failed.
Then, what I did in order to get the RNA module to function was I first went to my Python IDE, changed my working directory to the Python folder in the interfaces folder, and then opened one of the example Python scripts (from the Examples folder). Then, I ran the example script, and my RNA module just started working.
Let me know if that's how you got it to work or not.
Edit: http://www.tbi.univie.ac.at/RNA/#self_packages
If you scroll down to the link provided, it has an already compiled binary package for Mac OS X. So that might also help.
I just saw this post. You probably figure it out, but you what you need to do is to specify what version of python you want to be installed when you run /.configure. For example in my case, /.configure--with-python3

vim loading python on linux

I have Python 2.7 and Python 3.3 in my Debian. I rebuilt both using the arg --enable-shared.
After than I built VIM 7.4 using the args " --enable-pythoninterp --enable-python3interp --with-features=huge --with-python-config-dir=PathToConfig --with-python3-config-dir=PathToConfig"
when i give the comment vim --version, I could see the flags '+python/dyn and +python3/dyn'
However when I type the command ':python import sys', I get the error message:
E448: Could not load library function _PyArg_Parse_SizeT
E263: Sorry, this command is disabled, the Python library could not be loaded.
When I type the command ':python3 import sys', I get the error message:
E448: Could not load library function PySys_SetArgv
E263: Sorry, this command is disabled, the Python library could not be loaded.
all the vim plugins that needs python is not working due to this.
Please let me know how to fix this.
"Dynamic" Python loading is only available for use on Windows, unfortunately. I have looked into this as well, and it is not available on any other operating system.
The Vim docs: http://vimdoc.sourceforge.net/htmldoc/if_pyth.html#python-dynamic specify: "On MS-Windows the Python library can be loaded dynamically."
Basically the answer is: Nope, "dynamic" will not work on any Mac/*nix systems.
What I've done on my own system is to compile two versions of vim, one with Python2 and the other with Python3 links. Then in my .vimrc I include a version-check to use the right python exec for plugins, etc.

Python 2.7.5 on Cygwin64: requests installation fails

Our project is a mostly J2EE based development with the automatic functional and integration tests written in Python. The test environment is Linux nonetheless developers use Windows 7 (64-bit). We would like to be able to execute the functional tests on the developer machines as well (before comitting). Unfortunately the pexpect-windows-portability issue would leave us no choice but:
To do some serious refactoring on our test libraries to be able to use both winpexpect (or wexpect) and pexpect depending on the os settings.
Or to use cygwin. Guess what, with this second option we seem to have an issue :-) Using Python 2.7.5 on Cygwin64 installing the requests package results in error:
pip says it can not find a file after downloading and extracting the library
easy install doesn't throw an explicit error, but leaves everything in the temporary dir
after copying the files under the site-packages directory a simple import requests in python causes the interpreter to exit
Has anybody encountered this problem? With Cygwin-32 requests install smoothly. (however we have some other issues - see my next post ;-))
Thank you in advance: Joe, the public
Also ran into the same issues when trying to install requests, all the options on http://docs.python-requests.org/en/latest/user/install/#install did not work. I went to https://github.com/kennethreitz/requests and then clicked on "Download Zip" and I got requests-master.zip.
Update: This should be fixed in Cygwin.
This was a bug in CPython that has been fixed in their master branch. I've pushed a candidate package to the Python maintainer for Cygwin, but you might try this hotfix.
I downloaded from "https://github.com/requests/requests" and then i just ran the setup.py from the requests-master folder ( this was placed in cygwin folder). After that I went to cygwin terminal and then I ran python --> import requests. Voila it worked.

Vim Editor->clang_complete: no python support available

Case: Testing clang_complete with gVim 7.3
I installed MinGW, then followed the tutorial here to download and compile clang. It compiled for around an hour, then make install. Clang worked. I especially love the error annotation. It's amazing.
Now comes the case of clang_complete. I Installed it. I am using pathogen, so clang_complete fron github comes in bundle folder inside vimfiles.
I opened the vim editor and gave command :scriptnames. It shows clang_complete plugin. :version shows it has python entry, so vim was built with python support.
Also my test system has python installed.
but whenever I issue :save foo1.cpp , vim give this error,
Error detected while processing function 14_ClangCompleteInit..14_initClangCompletePython:
line2
clang_complete:No python support available
line 3
cannot use clang library.
simply puzzled.
*I have python installed on my system.
I Also tried using libclang library path for clang_complete as mentioned in another question on clang_complete here, but to no avail.*
Thank you.
Vim needs to be compiled with Python support, i.e. +python when doing :version. -python means it's not installed. Taken from clang_complete at Github:
You need Vim 7.3 or higher, compiled with python support and ideally,
with the conceal feature.
You said you had this enabled, but it doesn't look like it. Taken from clang_complete.vim.
This is the only place that error message is defined and triggers on !has('python'). In other words this doesn't seem to be a clang_complete issue, but rather that your install is missing or is having problems with Python support.
function! s:initClangCompletePython()
if !has('python')
echoe 'clang_complete: No python support available.'
echoe 'Cannot use clang library'
echoe 'Compile vim with python support to use libclang'
return 0
endif
[..]
Just came across this issue, invoking :version returns for features:
-python
+python3
So it seems this is caused by incompatible plugin that requires python (e.g. 2.x) but only python 3.x is available.

Categories

Resources