I was wondering if any of you could give me guidance on whether it'd be possible to use some of these TA-LIB functions found here in a python script. I cant find the functions in any other language that I know...
I read this, so there seems to be some level of possibility, however I have little understanding of whats going on in the article since I dont know C at all. Oh and incase you are wondering TA-Lib is ported on python BUT its doesnt really build on mac and most people say they have issues with it.
So essentially, I can't get the whole app to work in swig, I was wondering if I could instead compile the function (not even sure if that makes sense) and use it in a python app (and hopefully some guidance on how to do so).
I believe there are three simple approaches you could take:
SWIG
TA-Lib comes with a Python wrapper that is generated by SWIG. It hasn't been updated in a long time, so is hard-coded to build with Python 2.3. Andy Hawkins wrote some directions to get it to work with newer versions of Python.
Cython
I wrote a TA-Lib python wrapper that uses Cython to wrap all the functions in TA-Lib, and released it on Github. It works really well for me, uses Numpy arrays, is 2-4 times faster, more "pythonic", and easier to install (works on Mac OS X) than the SWIG interface.
Ctypes
If you only need a small number of functions from the library, you can use ctypes to make calls into the TA-Lib library.
If all you need is a single function in a library, yoyu be better of using ctypes - ctypes is a Python module in the standard library that allows to you to perform calls to libraries in native code.
You just have to check on the Python console how to obtain you TA-LIB as a Python object using ctypes, and how to call the function you need. Ctypes converts ints and strings automatically to C for you - you will need to perform some function anotation for other parameter types, though.
Related
I'm using jsonpickle in my program, but found it to be a performance bottleneck. So I'm trying to see if I can compile it to C using Nuitka, then use the C version in my program (through some wrappers perhaps).
But to be honest, I'm new to Nuitka, so I don't even know if this is a legit use case. Can someone give me some hints?
Note: this question is not about how to make a program faster. I'm building a library, not an application, so certain approaches like Pypy won't work. I'm also aware of Cython and is investigating it too, but this question is not about Cython either.
Figured it out myself. Just git clone the repo, go into the directory, and run
python -m nuitka --module jsonpickle --include-package=jsonpickle
Nuitka will generate a .so file, and you can import and use it just like the original package.
This method should be universal and could apply to any package and not just jsonpickle.
Do note that Nuitka is meant to be used in an application, but not good for building a library. The main pain point is that it doesn't support cross-compilation.
I am writing a Python package that, in normal operation, needs to run a helper program that's written in C. The helper program ships as part of the package, and it doesn't make sense to try to run it independently.
How do I persuade Distutils to compile, and install to an appropriate location, an independent C program rather than a C extension module?
How should the Python part of the code locate and start the helper program?
N.B. Porting the actual code (especially the C helper) to Windows would necessitate a >90% rewrite, so I only care about making installation work on Unix.
This is pretty interesting. I've never done this, but I think you can use the distutils compiler directly.
I checked out github for some possible examples that might give you inspiration. Check out this one
The filter I used was distutils ccompiler language:python filename:setup.py in case you want to extend it/trim it down
I've found a library which is in C (?), that I want to try from Python:
https://github.com/centaurean/density
I'd like to see if I can store compressed files on disk and decompress them in memory for faster full-file reading times.
I'm new to using external code from Python. How can I create a Python function that uses this library with a little overhead as possible?
(I will work with Windows or Linux)
One way is to compile the library into a dynamic library (.dll on Windows or .so on Linux) and use ctypes (https://docs.python.org/2/library/ctypes.html) to access it.
ctypes provides an easy path into doing exactly this through FFI, with the alternative of using something like SWIG to interface with the python libraries at a much lower layer.
I would recommend generating a little micro-benchmark program to record what the performance is before and after such an experiment, so that you have concrete values to use in choosing if you want to run with ctypes or python-swig code long term over doing something natively in python, which you could trial in c-python or a variant (such as pypy)
Fortunately for you, Python comes with an FFI built-in called ctypes. It works with both Linux and Windows, and there are plenty of tutorials out there to learn how to use it.
Note that you will need to compile the library beforehand into a shared lib (DLL in Windows, SO in Linux).
I'm trying to create a python script that will disassemble a binary (a Windows exe to be precise) and analyze its code.
I need the ability to take a certain buffer, and extract some sort of struct containing information about the instructions in it.
I've worked with libdisasm in C before, and I found it's interface quite intuitive and comfortable.
The problem is, its Python interface is available only through SWIG, and I can't get it to compile properly under Windows.
At the availability aspect, diStorm provides a nice out-of-the-box interface, but it provides only the Mnemonic of each instruction, and not a binary struct with enumerations defining instruction type and what not.
This is quite uncomfortable for my purpose, and will require a lot of what I see as spent time wrapping the interface to make it fit my needs.
I've also looked at BeaEngine, which does in fact provide the output I need, a struct with binary info concerning each instruction, but its interface is really odd and counter-intuitive, and it crashes pretty much instantly when provided with wrong arguments.
The CTypes sort of ultimate-death-to-your-python crashes.
So, I'd be happy to hear about other solutions, which are a little less time consuming than messing around with djgcc or mingw to make SWIGed libdisasm, or writing an OOP wrapper for diStorm.
If anyone has some guidance as to how to compile SWIGed libdisasm, or better yet, a compiled binary (pyd or dll+py), I'd love to hear/have it. :)
Thanks ahead.
Well, after much meddling around, I managed to compile SWIGed libdisasm!
Unfortunately, it seems to crash python on incorrect (and sometimes correct) usage.
How I did it:
I compiled libdisasm.lib using Visual Studio 6, the only thing you need for this is the source code in whichever libdisasm release you use, and stdint.h and inttypes.h (The Visual C++ compatible version, google it).
I SWIGed the given libdisasm_oop.i file with the following command line
swig -python -shadow -o x86disasm_wrap.c -outdir . libdisasm_oop.i
Used Cygwin to run ./configure in the libdisasm root dir. The only real thing you get from this is config.h
I then created a new DLL project, added x86disasm_wrap.c to it, added the c:\PythonXX\libs and c:\PythonXX\Include folders to the corresponding variables, set to Release configuration (important, either this or do #undef _DEBUG before including python.h).
Also, there is a chance you'll need to fix the path to config.h.
Compiled the DLL project, and named the output _x86disasm.dll.
Place that in the same folder as the SWIG generated x86disasm.py and you're done.
Any suggestions for other, less crashy disasm libs for python?
You might try using ctypes to interface directly with libdisasm instead of going through a SWIG layer. It may be take more development time but AFAIK you should be able to access the underlying functionality using ctypes.
I recommend you look at Pym's disassembly library which is also the backend for Pym's online disassembler.
You can use the distorm library: https://code.google.com/p/distorm/
Here's another build: http://breakingcode.wordpress.com/2009/08/31/using-distorm-with-python-2-6-and-python-3-x-revisited/
There's also BeaEngine: http://www.beaengine.org/
Here's a Windows installer for BeaEngine: http://breakingcode.wordpress.com/2012/04/08/quickpost-installer-for-beaenginepython/
I've been able to use the standard Python modules from IronPython, but I haven't gotten SciPy to work yet. Has anyone been able to use SciPy from IronPython? What did you have to do to make it work?
Update: See Numerical computing in IronPython with Ironclad
Update: Microsoft is partnering with Enthought to make SciPy for .NET.
Some of my workmates are working on Ironclad, a project that will make extension modules for CPython work in IronPython. It's still in development, but parts of numpy, scipy and some other modules already work. You should try it out to see whether the parts of scipy you need are supported.
It's an open-source project, so if you're interested you could even help. In any case, some feedback about what you're trying to do and what parts we should look at next is helpful too.
Anything with components written in C (for example NumPy, which is a component of SciPy) will not work on IronPython as the external language interface works differently. Any C language component will probably not work unless it has been explicitly ported to work with IronPython.
You might have to dig into the individual modules and check to see which ones work or are pure python and find out which if any of the C-based ones have been ported yet.