I have a function call that sometimes produces a segfault. Right now I isolate this code by putting it in a separate Python file and doing Popen('python snippet.py') inside of my main script. Is there a more elegant way of doing this on a Unix system?
Update 7/24
I found the following to be convenient
if not os.fork():
# do stuff that could segfault here
This has the advantage of not having to put all initialization code in a separate file. Apparently there's also multiprocessing module in Python 2.6 that supports inter-process communication
Why not find it and fix it?
strace python yourapp.py yourargs and options >trace.out 2>&1
Then, after it segfaults try tail -50 trace.out and you will see how it leads up to the segfault. Typically it is a pointer that is garbled and points to memory addresses that are not part of the process. It is quite possible for a Python error to create such pointer errors if you are using something like ctypes to access shared libraries.
I
Related
I've been searching how to read a process memory adresses in Python3 on OSX without finding a good and simple way to implement this (I'm new to Python3 and OSX).
I'm aiming at real time analysis (no pause on the process to analyse), and an efficient way to get those variables without having to dump the whole memory.
It seems like ptrace is not the right tool for this issue, and Mach library of OSX (giving access to vm_read functions) doesn't seems to be available on Python3.
I get that I need to spawn a child process with something like the subprocess import in order to get the rights to actually read the memory.
Does anyone have any clue on this issue ?
Thanks !
I am currently getting an issue with an external executable crashing when it is launched from a Python script. So far I have tried using various subprocess calls. As well as the more redundant methods such as os.system and os.startfile.
Now the exe doesn't have this issue when I call it normally from the command line or by double-clicking on it from the explorer window. I've looked around to see if other people have had a similar problem too. As far as I can tell the closest possible cause of this issue is that the child process unnecessarily hangs due to the I/O exceeding 65K. So I've tried using Popen without PIPES and I have also changed the stdout and stdin to write to temporary files to try and alleviate my problem. But unfortunately none of this has worked.
What I eventually want to do is be able to autorun this executable several times with various outputs provided by xmls. Everything else is pretty much in place, including the xml modifications which the executable requires. I have also tested the xml modification portion of the code as a standalone script to make sure that this isn't the issue.
Due to the nature of script I am a bit reluctant to put up any actual code up on the net as the company I work for is a bit strict when it comes to showing code. I would ask my colleagues if I could but unfortunately I'm the only one here who actually has used python.
Any help would be much appreciated.
Thanks.
As I've not had any response I've kind of gone down a different route with this. Rather than relying on the subprocess module to call the exe I have moved that logic out into a batch file. The xmls are still modified by the python script and most of the logic is still handled in script. It's not what ideally would have liked from the program but it will have to do.
Thanks to anybody who gave this some thought and tried to at least look for an alternative. Even if nobody answered.
In case both options available: to call a command line tool with subprocess (say, hg) or to make use of native python API (say, mercurial API), is there a case where it's more favorable to use the former?
If you want to execute some third party native code which you know is not stable and may crash with a segvault then it is better to execute it as a subprocess - you will be able to safely handle the possible crashes from your Python process.
Also, if you want to call several times some code which is known to leak memory, leave open files or other resources, from a long running Python process, then again it may be wise to run it as a subprocess. In this case the leaking memory or other resources will be reclaimed by the operating system for you each time the subprocess exits, and not accumulate.
The only way that i see myself using subprocess instead of a native python api is if some option of the program is not provided in the api.
complete Python noob here (and rusty in C).
I am using a Mac with Lion OS. Trying to use NFCpy, which uses USBpy, which uses libUSB. libUSB is crashing due to a null pointer but I have no idea how to debug that since there are so many parts involved.
Right now I am using xcode to view the code highlighted but I run everything from bash. I can switch to Windows or Linux if this is going to be somehow easier with a different environment.
Any suggestions on how to debug this would be much appreciated ;-)
PS: It would be just fine if I could see the prints I put in C in the bash where I run the Python script
You should see your printf() you put in C in your terminal, something is already wrong here. Are you sure that you're using the latest compiled library? To be sure, instead of print, you can use use assert(0) (you need to include assert.h).
Anyway, you can debug your software using gdb:
gdb --args python yourfile.py
# type "run" to start the program
# if you put assert() in your code, gdb will stop at the assert, or you can put
# manual breakpoint by using "b filename:lineno" before "run"
Enable core dumps (ulimit -Sc unlimited) and crash the program to produce a core file. Examine the core file with gdb to learn more about the conditions leading up to the crash. Inspect the functions and local variables on the call stack for clues.
Or run the program under gdb to begin with and inspect the live process after it crashes and gdb intercepts the signal (SIGSEGV, SIGBUS, whatever).
Both of these approaches will be easier if you make sure all relevant native code (Python, libUSB, etc) have debugging symbols available.
Isolating the problem in a program which is as small as you can manage to make it, as Tio suggested, will also make this process easier.
PS: It would be just fine if I could see the prints I put in C in the bash where I run the Python script
You didn't mention anything about adding prints "in C" elsewhere in your question. Did you modify libUSB to add debugging prints? If so, did you rebuild it? What steps did you take to ensure that your new build would be used instead of the previously available libUSB? You may need to adjust your dylib-related environment variables to get the dynamic linker to prefer your version over the system version. If you did something else, explain what. :)
I've seen a few sites talking about injecting DLL's (such as http://www.codeproject.com/KB/DLL/DLL_Injection_tutorial.aspx), but I'm struggling with how to get an EXE to work. any help/tips would be appreciated.
The best way I know how to explain it is "RunPE" where you execute an exe in the memory. Does that help at all?
If you're asking how to inject code into a running Python process, what you want is https://github.com/lmacken/pyrasite .
You can use the Reflective DLL Injector as described here. Metasploit project uses it to load its meterpreter plug-ins. AFAIK this is the only way to inject a DLL, as MS officially does not support "injecting" from memory, only loading from file system.
On a low level, nothing forbids you from allocating a memory region, loading code there, marking it executable.
Note, that none of these techniques are Python specific or even Python related - it is a win32 problem.
What you're talking about is re-implementing UPX in python with more stuff.
Things you would need to do in order to do this:
Change all VirtualAlloc calls to be VirtualAllocEx calls.
Change all Loadlibrary calls to be loadlibraryEX calls.
Implement the relocation fix-ups.
A better approach would probably be tweaking UPX to output a DLL instead of an executable. Then using some python DLL injection code to throw that into another process. You're going to be working with CTypes a lot if you want to do this. Fair warning...
I would recommend this book http://www.amazon.com/Gray-Hat-Python-Programming-Engineers/dp/1593271921 - especially the chapters on writing your own debugger, but it covers the metasploit and other tools as described above.
To inject a shared object (.so, .dll) into any process you can use injector with C, or pyinjector with python/shell.
To inject python code into a running python process, you can use hypno.