I have some problems while trying to import some module (compiled .pyc) in my program. I know that it compiled in Python 2.6.6 (r266:84297), I have installed the same version, but had an error "bad magic number" while trying to import it :(
Does anybody know what I did wrong?
Or maybe it's possible to change magic number in .pyc module?
As the answer linked by Matthew explains, your problem is almost certainly due to different versions of Python being used for compiling and loading the module. You can determine the magic number like this:
with open('pyuca.pyc', 'rb') as f:
print struct.unpack('<H', f.read(2))
You can determine your Python version by printing sys.version (it is also echoed on interactive startup). If you are using Python 2.6.6, the magic number should be 62161. If it is different, you will need to switch to a different Python to be able to import the module.
The exact same applies to .pyo files.
I solved this by running
find . -name '*.pyc' -exec rm {} +
which deleted all the pyc files in my directory. After that it was OK.
If your running python2 and python3 and removing old *.pyc files is too messy you can add the following to try to patch this python3 bug by adjusting the code loader suffix for bytecode files.
import sys
if(sys.version_info.major>=3): # switch byte files end extension on 3
import importlib.machinery
altsuffix = ['.pyc3'] # or some other ending that doesn't create conflicts
importlib.machinery.BYTECODE_SUFFIXES = altsuffix
Related
I'm trying to use the nuitka tool to turn my python program into executable on ubuntu. It works fine if the program doesn't have any import statements but breaks when I use it on a program that imports something e.g.
test.py
import numpy
print "hello, world."
type this on commandline
nuitka --recurse-all --python-version=2.7 test.py
and gives me these errors
Nuitka:WARNING:/usr/lib/python2.7/dist-packages/numpy/numarray/functions.py:45: Cannot find 'copyreg' in package 'numpy.numarray' as absolute import.
Nuitka:WARNING:/usr/lib/python2.7/dist-packages/numpy/distutils/npy_pkg_config.py:11: Cannot find 'configparser' in package 'numpy.distutils' as absolute import.
Nuitka:WARNING:/usr/lib/python2.7/dist-packages/numpy/distutils/system_info.py:1765: Cannot find 'Numeric' in package 'numpy.distutils' as absolute import.
Nuitka:WARNING:/usr/lib/python2.7/dist-packages/numpy/distutils/system_info.py:1770: Cannot find 'numarray' in package 'numpy.distutils' as absolute import.
Nuitka:WARNING:/usr/lib/python2.7/dist-packages/numpy/f2py/diagnose.py:48: Cannot find 'numpy_distutils' in package 'numpy.f2py' as absolute import.
Nuitka:WARNING:/usr/lib/python2.7/dist-packages/numpy/f2py/diagnose.py:87: Cannot find 'numpy_distutils.command.build_flib' in package 'numpy.f2py' as absolute import.
I don't know about your particular use case but I also faced similar Cannot find '' in Package errors when using nuitka.
I was using sqlalchemy and had a similar issue with configparser.
After about a day of debugging I found out that Nuitka trips up with SWIG (Dynamically Loaded shared objects).
What it means basically is, some programs/modules try to increase compatibility by using conditional imports.
For eg:
If python_version==3.5:
import thislibrary
else:
import thatlibrary
specifically the configparser library is named configparser in python3 and ConfigParser in python2.
So what basically is happening is that nuitka is trying to import python 3 stuff when you clearly are using python 2.
For me the fix was to modify the source code of sqlalchemy and change the if else construct to:
import thatlibrary
You can find more information in this Guide
written by Tom Sheffler
Official answer from Nuitaka.
What problem you facing same like another problem one user facing package 'matplotlib' as absolute import. this problem, then there nuitaka given comments below check if it useful for below comment.
I think you might be using 32 bits Python on Windows and hit the 2GB RAM boundary. Use 64 bits for better luck. Incidentally I am working on scalability improvements for the next releases which ought to make this not happen. For now Nuitka loads all say 1000 modules into RAM and compiles them all globally. Needs a lot of RAM.
I see there are some built in packages that I can import from any script like:
from datetime import date
today = date.today()
print today
How can I create a simple package and add it to the system library so that I can import it like datetime in the above example?
You're trying to make a module.
Start by installing the setuptools package; on either Windows or Linux you should be able to type pip install setuptools at a terminal to get that installed. You should now be able to write import setuptools at a python prompt without getting an error.
Once that's working, set up a directory structure containing a setup.py and a folder for your project's code to go in. The directory must contain a file called __init__.py, which allows you to import the directory as though it's a file.
some_folder/
| setup.py
| my_project/__init__.py
In setup.py, drop the following content:
# setup.py
from setuptools import setup
setup(name="My Awesome Project",
version="0.0",
packages=["my_project"])
In my_project/__init__.py, drop some stuff that you'd like to be able to import. Let's say...
# my_project/__init__.py
greeting = "Hello world!"
Now, in order to install the project at a system-wide level, run python setup.py install. Note that you'll need to run this as root if you're on Linux, since you're making changes to the system-wide python libraries.
After this, you should be able to run python from any directory you like and type:
>>> from my_project import greeting
>>> print greeting
Hello world!
>>>
Note that this is enough to tell you how to make a module, but there's one hell of a lot of stuff that setuptools can do for you. Take a look at https://pythonhosted.org/setuptools/setuptools.html for more info on building stuff, and https://docs.python.org/2/tutorial/modules.html for more info on how modules actually work. If you'd like to look at a package that (I hope) is reasonably simple, then I made my LazyLog module a couple of weeks ago on a train, and you're welcome to use it for reference.
The quick way, if you're just making something for your own use and not worrying about packaging, is to put the module (which could be as simple as a single file) in your system's site-packages directory. (On Debian-based systems, you probably want to use dist-packages instead).
To find out where your site-packages/dist-packages directory is, start Python and:
>>> from sys import path
>>> path
['', '/usr/lib/python3.4/site-packages/pip-7.1.2-py3.4.egg', '/usr/lib/python34.zip', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-cygwin', '/usr/lib/python3.4/lib-dynload', '/usr/lib/python3.4/site-packages']
Note the last item in that example: /usr/lib/python3.4/site-packages. That's the sort of thing you're looking for. So in this example, if I save the following to /usr/lib/python3.4/site-packages/foo.py:
def bar():
print('Hello world!')
Then from anywhere on my system:
>>> from foo import bar
>>> bar()
Hello world!
If you really need a package, you can do it with boost, which allows interacts with C++. You may implement algorithms using C++ and compile it as a Python lib. However it's poorly documented. And as doc described, C API should be a basic option. Boost is built on C API any way.
Sample: I made it several years ago on a class. You can do: import tfidf.
If you simply want the module for personal use, just drop it in a folder and add that folder to the PYTHONPATH environment variable.
For example, create a folder in your home directory called ~/python-packages, then add the following line to your .bashrc (assuming you are using bash):
export PYTHONPATH=$HOME/python-packages`
Then, simply drop any modules/packages you want to make available in ~/python-packages.
Add you python script or package's path to sys.path or just move them to one of the location in sys.path.
BUT i do not suggest to do this...
I was believing that when importing a package, it would search from sys.path and use the first hit for import. However, it seems not true:
import mpl_toolkits
print(mpl_toolkits.__path__)
And it outputs:
['/Library/Python/2.7/site-packages/matplotlib-1.5.0-py2.7-macosx-10.11-intel.egg/mpl_toolkits', '/usr/local/lib/python2.7/site-packages/mpl_toolkits']
Can someone please explain to me how exactly python looks for packages if it is installed multiple times in the machine (in different location searchable by sys.path)? Or a pointer to relevant reference would be good.
when you import a module, python uses PYTHON PATH (system variable that contains a list of folders) and loops to search for importable module.
Python will test if it is a package (folder containing init.py) or a module (*.py). it will stop on first module found if no module is found python raises an import error
I am fairly new to Python (3.3.2) and I have minimal experience using
from .... import ....
beyond the use of time, turtle, and math. I am working on several Project Euler problems that require a function to determine the primality of a number using my simple isPrime(n) function. The isPrime(n) function is the only function in a file named isPrime.py.
I have a separate folder for each Project Euler problem I attempt to keep my code straight but I find it inconvenient having to copy my isPrime(n) function directly from isPrime.py and paste it into every new file within a folder devoted to each problem. I have some ideas about improving my isPrime(n) function and I don't want to have to open up several of the Problem folders just to change the lines in each file when I make some revisions.
So I thought that writing the following line at the top of each Problem's file would work:
from isPrime.py import isPrime
or even from isPrime import isPrime
However, I encounter the following error both ways:
ImportError: No module named 'isPrime'
Here is a basic description of my folder hierarchy:
ProjectEuler Folder
Primality Functions Folder
isPrime.py [contains the function isPrime(n)]
anotherFile.py
yetAnotherFile.py
Problem X Folder
problemX.py
something.py
somethingElse.py
Problem Y Folder
problemY.py
something2.py
somethingElse2.py
Problem Z Folder
problemZ.py
something3.py
somethingElse3.py
My question is:
What do I need to type in the opening lines of the files problemX.py,problemY.py, and problemZ.py to use the function isPrime(n) from the file isPrime.py in the folder Primality Functions?
I have searched at length here at stackoverflow and I see many questions are related to Python and difficulty with importing files/modules. However, I don't understand what a system path is, or what a relative import is, and I have no idea what the Python documentation found here is telling me. Please do not mark this question as duplicate of the many Python/Import questions already here. All I'm looking for is a simple 1-3 lines of code to place at the top of my files so I can import my functions from other folders and won't have to copy-and-paste my functions every time I attempt a new problem. I will be most appreciative of the most simply-worded answer. Thank you!
I had the same problem as you i.e., I had some utility modules that may be useful to solve multiple project euler problems and each project euler problem had its own directory.
Initially I put the utility folder into sys.path as #user3114046 suggested:
import sys; sys.path.insert(0, '../Primality Functions Folder')
from isPrime import isPrime
It felt dirty but it worked (note: I even used a relative path here!).
As an alternative, you could create a simple setup.py file in the same directory as isPrime.py:
from distutils.core import setup
NAME = 'isPrime'
setup(name=NAME, version='0.0.1', py_modules=[NAME])
and install your utility module:
$ python setup.py install
after that you could use it in any script:
from isPrime import isPrime
If you want to use several utility modules; you could put them into project_euler_utils directory and install it as Python package. Put setup.py along side project_euler_utils directory:
from distutils.core import setup
NAME = 'project_euler_utils'
setup(name=NAME, version='0.0.1', packages=[NAME])
Run pip install project_euler_utils, to install it. After that; you could use it from any script/module:
from project_euler_utils.isprime import isprime
Note: I've used lowercase names as pep-8 suggests (you need to rename your module and the function in it).
I tried using the fullest path I can think of with "C:\Users\Owner\Documents\Primality Functions Folder", it gave me a Syntax Error: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape.
Use raw string literals for Windows paths r'C:\Users\Owner...' (note: r'' prefix) Otherwise backslash is special inside literal strings, in particular u'\U0001F385' is a not 10 characters; it is a single Unicode codepoint U+1F385.
Since you want to keep the modules you are importing in different directories, you can tell python where to find them by adding those directories to sys.path:
from sys import path
path.append('fully specified path name of directory with module you want to import')
from mymodule import myfunction
This doesn't need to be done if isPrime.py is in the same folder as the program which imports it, but sometimes this is not practical.
It's not optimal for the module to have the same name as one of its functions, though, so I would consider changing it.
I'm trying to determine which files in the Python library are strictly necessary for my script to run. Right now I'm trying to determine where _io.py is located. In io.py (no underscore), the _io.py module (with underscore) is imported on line 60.
Some modules are compiled directly into the interpreter -- there are no files corresponding to them. You can retrieve a list of these modules from sys.builtin_module_names. In my Pyton 3.1 installation, _io is included in this list.
You might want to have a look at snakefood to determine the dependencies of your script.
Not all Python modules are written in Python. Try looking for _io.so or _io.pyd.
Try the DLLs folder under your base python install directory if you are on windows. It contains .pyd modules Ignacio mentions. I had a similar problem with a portable install. Including the DLLs folder contents to my install fixed it. I am using python 2.5.
From python-list email archive: is "_io.py" missing from 2.7.4 ?, the situation for Python 2 and 3 is different:
In Python 2.7:
To find where the _io module lives, at the interactive interpreter run
this:
import _io
_io.__file__
Under Linux, you should get something like this:
'/usr/local/lib/python2.7/lib-dynload/_io.so'
and the equivalent under Windows.
In Python 3:
Note that in Python 3.3, the _io module is now built-in into the
compiler, so _io.__file__ no longer exists.