Someone created a C module for python with Nuitka. (The original Python code is not available, the module is already compiled - so it is a machine binary file.) I would like to use the code within another tool, which only excepts Python files. So I would like to include the C code into Python.
To get more specific: So far I have the files thatmodule.pyi and a thatmodule.so. I can include them into my current Python code simply by running import thatmodule inside mymodule.py. Now I only want one single Python file mymodule.py.
My current idea is to copy the code from thatmodule.pyi to the beginning of mymodule.py and to convert thatmodule.so to a binary string with
with open('thatmodule.so', mode='rb') as file:
fileContent = file.read()
... missing ... how to convert fileContent to b'string'...
and put this binary string into mymodule.py. And then I have to execute this binary string from within my python module mymodule.py. How can I do this?
You'll have to write it out to a file (and presumably the .pyi too) and then use python's importlib to dynamically import it.
If you have a documentation, which describes functions for thatmodule.so , you can use the following:
import ctypes
mylib = ctypes.CDLL("thatmodule.so")
Documentation here
Related
I have to use a config-file cfg.yml:
---
paths:
reldir : ../my/dir
In Python, I run:
with open('cfg.yml', 'r') as config_file:
cfg = yaml.load(config_file)
and my goal is to do something with some files in the directory reldir, via the Python file. The above works well.
However, this Python program must be able to run on Windows and Linux. If I am not mistaken, they use different path delimiters, / and \\. Thus, I want to make the reldir in the config file more robust:
---
paths:
reldir : os.path.join('..','my','dir').
If my understanding is correct, this will combine these folder names with the correct delimiter, depending on where the Python program is executed.
However, this doesn't work, and print(reldir) outputs os.path.join('..','my','dir') instead of ../my/dir. I.e., it is taking the string literally without evaluating the os.path.join function.
I experimented with exec() and eval(), but first, I could not get it to run anyway; and second, I read here that I shouldn't use these.
How should I best proceed?
Usually "/" works for Windows as well. You can just try "../my/dir"
I'm planning on using a Python script to change different BAM (Binary Alignment Map) file headers. Right now I am just testing the output of one bam file but every time I want to check my output, the stdout is not human readable. How can I see the output of my script? Should I use samtools view bam.file on my script? This is my code.
#!/usr/bin/env python
import os
import subprocess
if __name__=='__main__':
for file in os.listdir(os.getcwd()):
if file == "SRR4209928.bam":
with open("SRR4209928.bam", "r") as input:
content = input.readlines()
for line in content:
print(line)
Since BAM is a binary type of SAM, you will need to write something that knows how to deal with the compressed data before you can extract something meaningful from it. Unfortunately, you can't just open() and readlines() from that type of file.
If you are going to write a module by yourself, you will need to read Sequence Alignment/Map Format Specification.
Fortunately someone already did that and created a Python module: You can go ahead and check pysam out. It will surely make your life easier.
I hope it helps.
I am trying to pipe the output of xz to a custom python script:
xz -cd file.pcap.xz | myscripy.py
However, I get an error when the script attempts to run this line:
#!/usr/bin/env python2.7
from __future__ import print_function
import pcap
import io
STDIN_ALIAS = '/proc/self/fd/0'
pcap.pcap(io.open(STDIN_ALIAS, 'r'))
and received an error
pcap.pcap(io.open(STDIN_ALIAS, 'r'))
File "pcap.pyx", line 196, in pcap.pcap.__init__
TypeError: expected string or Unicode object, _io.TextIOWrapper found
I am on Ubuntu 18.04 and running under python 2.7.
You can't use Python to pass in packets from a file to pcap.pcap(). The pypcap library you are using is a thin wrapper around the pcap_open_offline() and pcap_create() C functions, and offers no facilities for passing in a Python file object. This wrapper only accepts a filename or a network interface name, nothing else.
The pcap_open_offline() function does accept - as an alias for stdin, so just pass that in directly:
import pcap
sniffer = pcap.pcap('-')
The error message already tell you what happened. You need a string to the pcap() function, not a file object. To fix this, try
pcap.pcap(io.open(STDIN_ALIAS, 'r').read())
But I am not sure this will work as your file might be binary instead of text. In such case, you may want to open with 'rb' instead of 'r' flag, and do some conversion afterwards (especially if you use Python 3 instead of Python 2.7).
I see another issue: your code is not portable as it depends on this:
STDIN_ALIAS = '/proc/self/fd/0'
A pythonic way to read the stdin is the follows (see Reading binary data from stdin)
import sys
string = sys.stdin.read()
Have you tried upgrading PyPcap to work on Python 3? This could help, since Unicode handling is a lot cleaner and less prone to surprises on Python 3. The appropriate package is available, at least on Debian (and probably derived distros as well). Look for: python3-pypcap.
I am trying to load code as a module and then reload the same module but different code programmatically in Python:
import imp
a = """
def test():
print "Hello from a"
"""
b = """
def test():
print "Hello from b"
"""
for code in [a, b]:
with open('user.py', 'wb') as f:
f.write(code)
mod = imp.load_source('user', 'user.py')
getattr(mod, "test")()
Expected output:
Hello from a
Hello from b
Actual output:
Hello from a
Hello from a
Obviously my understanding of how it works is not correct but I can't seem to figure out my mistake.
The only way I could get it to work was if I deleted the generated .pyc file before I wrote the code in the file f. Is there a better way?
If you're going to be loading code dynamically out of a string or file, best use exec/execfile instead of import. import is meant for files which are static or rarely-changing.
If you still want to use imp.load_source, note the following caveat:
Note that if a properly matching byte-compiled file (with suffix .pyc or .pyo) exists, it will be used instead of parsing the given source file.
"properly matching" means that the compiled file's version matches the interpreter and the timestamp matches. As Tim points out, if you write the file twice in quick succession, the timestamp might not change and the .pyc would still be considered valid.
Probably not because the timestamp of your .pyc file (accurate to a second) will not be older than the timestamp for your newly written .py file; therefore imp will use the "current" .pyc file unless you delete it first.
Alternatively, you could try waiting for two seconds before reloading the module.
Whenever you run or import a python file, module or package, the interpreter will check, if a pyc file exists that matches the version of the py file. If so, it will use the pyc file, otherwise it will compile the py file and overwrite the old pyc file. If the pyc matches the version of the py file is based on file dates. Since you are not saving your file (f), Python does not recognize that it something has changed meantime. So if you want your approach to work, you need to save the file after each loop.
I need to write some input data files for a python program, and I need the full thing:
comments, spacing, variable = value, etc.
Is there any library (line argparser for command line arguments) for python or should I write my own?
Thanks!
Take a look at the ConfigParser module (renamed to configparser in Python 3).
Alternatively, you could simply write the input data using Python syntax, and import the result into your main program.
Does the syntax have to look like that? Could you just use a character delimited file (like csv or tab-delimited) with each predefined field in a separate column? Python has well defined modules to handle csv data.
If you specifically want input files that present blocks of code, then aix's suggestion of importing would also work.