I have legacy test scripts written in C for use with Loadrunner. I would like to use Grinder instead of Loadrunner. Grinder provides support for anything in Java which can easily be imported into Jython, Grinder's default programming language.
In order to reuse the test scripts written in C could I wrap them in Cython? (http://en.wikipedia.org/wiki/Cython) Cython is a compiled language which will produce extension modules that can be imported into regular Python or CPython code. Once I have this CPython code, will it work in Grinder or does Grinder specifically need Jython?
The question is whether Grinder can support default Python?
Look in the \data subdirectory under your LoadRunner script and you will find the original header and page source that the script was built from.
You are likely better off writing a preprocessor that pulls the original source for the calls and builds a Mock Grinder Script for modification. if you're really cagey you could pull the dynamic parameterization (correlation) tags from the .c files as well as use the script .prm file and parameter tags to modify the grinder prototype as you build it. If you attempt to use the LoadRunner scripts directly you are going to run into all sorts of issues with missing libraries for the support of the scripts.
Related
I'm trying to understand how it works. I know that Python interpreter translates python source code to byte code representation for a virtual machine (Python interpreter is a virtual machine) and executes those instructions. Where exactly does a C/C++ code comes in here? Does this virtual machine (Python interpreter) can also compile and execute C/C++ code?
I don't even know exactly what are right questions to ask here, just want a good explanation of how.
My background: I programmed in Python for a long time (mostly analytics/ML) and I have some basic understanding of computer systems, C compilation process, memory and processor. But I am not even close to being an expert on it.
I just want good understanding, not so much practical tips on how to create a Python module in C.
Thank you, I really appreciate your help!
It's all about a predictable entrypoint. The CPython reference interpreter (and other interpreters like PyPy that support C extensions of this sort), told to look for a module of a given name and finding a file matching the naming conventions for extension modules in one of the sys.path directories (e.g. for the spam module built for CPython 3.10 on x86-64 Linux, it would look for spam.cpython-310-x86_64-linux-gnu.so):
Uses the OS standard method for loading a dynamic library (aka shared object), e.g. LoadLibrary on Windows, dlopen on POSIX systems
Loads the entrypoint (using GetProcAddress on Windows, dlsym on POSIX) in it matching the specified naming convention, e.g. for the module named spam, it looks for a function named PyInit_spam following C name-mangling rules
Invokes that function, which is then wholly responsible for all other setup (calling PyModule_Create, performing any modifications to said module object, and returning it). The various APIs it invokes are what publish information for use by the user.
CPython, the "standard" Python interpreter, is written in C. It provides an extension API in C, so extensions written in C or C++ can register themselves to be called like normal Python modules. The Python interpreter cannot compile C or C++; it is the extension writer's responsibility to compile the module. But Python can run arbitrary C and C++ code through the help of that API.
I have a C++ project that invokes the same function in each python script. But each script does very different things that need access to the C++ project's internal classes.
So I need a python wrapper so I pass a C++ object to the python script and I need a way to run the python script's function from the C++ project too.
From what I understand with Cython and Shed Skin they are utilities to make C++ classes into a python class but not necessarily share run time objects back and forth between the languages.
What can I do?
Thank you Maverick for your question.
Cython is not a way of using C++ classes into python.Cython is simply python with c data types.Means that you can use c data types in your python program which in turn turns your python code to execute faster.Also you can build extensions in python using cython.
Now comming to the problem in the question,you can use boost library for your task.In boost you just need to write a wrapper to use your c++ object in python or vice versa.You just need to compile it to a shared library which can be done by tools like jam,cython etc.I am not going into details of how you are going to do it as you can find many tutorials for the same.
I have working Python script using scipy and numpy functions and I need to run it on the computer with installed Python but without modules scipy and numpy. How should I do that? Is .pyc the answer or should I do something more complex?
Notes:
I don't want to use py2exe. I am aware of it but it doesn't fit to the problem.
I have read, these questions (What is the difference between .py and .pyc files?, Python pyc files (main file not compiled?)) with obvious connection to this problem but since I am a physicist, not a programmer, I got totally lost.
It is not possible.
A pyc-file is nothing more than a python file compiled into byte-code. It does not contain any modules that this file imports!
Additionally, the numpy module is an extension written in C (and some Python). A substantial piece of it are shared libraries that are loaded into Python at runtime. You need those for numpy to work!
Python first "compiles" a program into bytecode, and then throws this bytecode through an interpreter.
So if your code is all Python code, you would be able to one-time generate the bytecode and then have the Python runtime use this. In fact I've seen projects such as this, where the developer has just looked through the bytecode spec, and implemented a bytecode parsing engine. It's very lightweight, so it's useful for e.g. "Python on a chip" etc.
Problem comes with external libraries not entirely written in Python, (e.g. numpy, scipy).
Python provides a C-API, allowing you to create (using C/C++ code) objects that appear to it as Python objects. This is useful for speeding things up, interacting with hardware, making use of C/C++ libs.
Take a look at Nuitka. If you'll be able to compile your code (not necessarily a possible or easy task), you'll get what you want.
I'm trying to write a software plug-in that embeds Python. On Windows the plug-in is technically a DLL (this may be relevant). The Python Windows FAQ says:
1.Do not build Python into your .exe file directly. On Windows, Python must be a DLL to handle importing modules that are themselves DLL’s. (This is the first key undocumented fact.) Instead, link to pythonNN.dll; it is typically installed in C:\Windows\System. NN is the Python version, a number such as “23” for Python 2.3.
My question is why exactly Python must be a DLL? If, as in my case, the host application is not an .exe, but also a DLL, could I build Python into it? Or, perhaps, this note means that third-party C extensions rely on pythonN.N.dll to be present and other DLL won't do? Assuming that I'd really want to have a single DLL, what should I do?
I see there's the dynload_win.c file, which appears to be the module to import C extensions on Windows and, as far as I can see, it scans the extension file to find which pythonX.X.dll it imports; but I'm not experienced with Windows and I don't quite understand all the code there.
You need to link to pythonXY.dll as a DLL, instead of linking the relevant code directly into your executable, because otherwise the Python runtime can't load other DLLs (the extension modules it relies on.) If you make your own DLL you could theoretically link all the Python code in that DLL directly, since it doesn't end up in the executable but still in a DLL. You'll have to take care to do the linking correctly, however, as pretty much none of the standard tools (like distutils) will do this for you.
However, regardless of how you embed Python, you can't make do with just the DLL, nor can you make do with just any DLL. The ABI changes between Python versions, so if you compiled your code against Python 2.6, you need python26.dll; you can't use python25.dll or python27.dll. And Python isn't just a DLL; it also needs its standard library, which includes extension modules (which are DLLs themselves, although they have the .pyd extension.) The code in dynload_win.c you ran into is for loading those DLLs, and are not related to loading of pythonXY.dll.
In short, in order to embed Python in your plugin, you need to either ship Python with the plugin, or require that the right Python version is already installed.
(Sorry, I did a stupid thing, I first wrote the question, and then registered, and now I cannot alter it or comment on the replies, because StackOverflow's engine doesn't think I'm the author. I cannot even properly thank those who replied :( So this is actually an update to the question and comments.)
Thanks for all the advice, it's very valuable. As far as I understand with some effort I can link Python statically into a custom DLL, provided that I compile other dynamically loaded extensions myself and link them against the same DLL. (I know I need to ship the standard library too; my plan was to append a zipped archive to the DLL file. As far as I understand, I will even be able to import pure Python modules from it.)
I also found an interesting place in dynload_win.c. (I understand it loads dynamic extensions that use Python C API, e.g. _ctypes.) As far as I can see it not only looks for init_ctypes symbol or whatever the extension name is, but also scans the .pyd file's import table looking for (regex) python\d+\. and then compares the found symbol with known pythonNN. string to make sure the extension was compiled for this version of Python. If the import table doesn't have such a symbol or it refers to another version, it raises an error.
For me it means that:
If I link an extension against pythonNN.dll and try to load it from my custom DLL that includes a statically linked Python, it will pass the check, but — well, here I'm not sure: will it fail because there's no pythonNN.dll (i.e. even before getting to the check) or it will happily load the symbols?
And if I link it against my custom DLL, it will find symbols, but won't pass the check :)
I guess I could rewrite this piece to suit my needs... Are there any other such places, I wonder.
Python needs to be a dll (with a standard name) such that your application, and the plugin, can use the same instance of python.
Plugin dlls are already going to expect to be loading (and using python from) a python26.dll (or whichever version) - if your python is statically embedded in your exe, then two different instances of the python library would be managing the same data structures.
If the python libraries use no static variables at all, and the compile settings are exactly the same this should not be a problem. However, generally its far safer to simply ensure that only one instance of the python interpreter is being used.
On *nix, all shared objects in a process, including the executable, contribute their exported names into a common pool; any of the shared objects can then pull any of the names from the pool and use them as they like. This allows e.g. cStringIO.so to pull the relevant Python library functions from the main executable when the Python library is statically-linked.
On Windows, each shared object has its own independent pool of names it can use. This means that it must read the relevant different shared objects it needs functions from. Since it is a lot of work to get all the names from the main executable, the Python functions are separated out into their own DLL.
I wonder if it is possible to create an executable module from a Python script. I need to have the most performance and the flexibility of Python script, without needing to run in the Python environment. I would use this code to load on demand user modules to customize my application.
There's pyrex that compiles python like source to python extension modules
rpython which allows you to compile python with some restrictions to various backends like C, LLVM, .Net etc.
There's also shed-skin which translates python to C++, but I can't say if it's any good.
PyPy implements a JIT compiler which attempts to optimize runtime by translating pieces of what's running at runtime to machine code, if you write for the PyPy interpreter that might be a feasible path.
The same author that is working on JIT in PyPy wrote psyco previously which optimizes python in the CPython interpreter.
You can use something like py2exe to compile your python script into an exe, or Freeze for a linux binary.
see: How can I create a directly-executable cross-platform GUI app using Python?
I've had a lot of success using Cython, which is based on and extends pyrex:
Cython is a language that makes
writing C extensions for the Python
language as easy as Python itself.
Cython is based on the well-known
Pyrex, but supports more cutting edge
functionality and optimizations.
The Cython language is very close to
the Python language, but Cython
additionally supports calling C
functions and declaring C types on
variables and class attributes. This
allows the compiler to generate very
efficient C code from Cython code.
This makes Cython the ideal language
for wrapping for external C libraries,
and for fast C modules that speed up
the execution of Python code.
I think you can use jython to compile python to Java bytecode, and then compile that with GCJ.