I am trying to bundle django with cx_freeze. With my setup.py I am able to bundle it together but when I invoke the generated executable I get the below import error. I have tried various ways to fix this but unable to get a way around.
Versions used:
Django==1.6.4
cx-Freeze==4.3.3
./build/exe.linux-x86_64-2.7/script
Traceback (most recent call last):
File "lib/python2.7/site-packages/cx_Freeze/initscripts/Console.py", line 27, in <module>
exec(code, m.__dict__)
File "script.py", line 7, in <module>
from models import Project
File "/remote/vgrnd77/pritam/tryout/package/models.py", line 6, in <module>
from django.db.models import CharField, Model, \
File "lib/python2.7/site-packages/django/db/models/__init__.py", line 5, in <module>
from django.db.models.query import Q
File "lib/python2.7/site-packages/django/db/models/query.py", line 14, in <module>
from django.db.models.fields import AutoField
File "lib/python2.7/site-packages/django/db/models/fields/__init__.py", line 15, in <module>
from django import forms
File "lib/python2.7/site-packages/django/forms/__init__.py", line 8, in <module>
from django.forms.fields import *
File "lib/python2.7/site-packages/django/forms/fields.py", line 17, in <module>
from django.forms.util import ErrorList, from_current_timezone, to_current_timezone
File "lib/python2.7/site-packages/django/forms/util.py", line 4, in <module>
from django.utils.html import format_html, format_html_join
File "lib/python2.7/site-packages/django/utils/html.py", line 12, in <module>
from django.utils.text import normalize_newlines
File "lib/python2.7/site-packages/django/utils/text.py", line 11, in <module>
from django.utils.six.moves import html_entities
ImportError: cannot import name html_entities
Here is my sample script:
#!/usr/bin/env python
import os
import django
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
from models import Project
if __name__ == '__main__':
print "My first exe"
p = Project.objects.all()
for project in p:
print project.name
Here is how the setup.py looks like:
main_python_file = "script.py"
import sys
from cx_Freeze import setup, Executable
# html_entities is missing
base = 'Console'
buildOptions = dict(
create_shared_zip = False,
append_script_to_exe = False,
packages = [],
includes = ['django'],
excludes = ['tkinter']
)
setup(
name = "my_exe",
version = "0.1",
description = "My first exe",
options = dict(build_exe = buildOptions),
executables = [Executable(main_python_file, base = base)])
Here is how the project table looks like:
class Project(Model):
"""
Project table
"""
name = CharField(max_length=255)
Has anyone faced and solved this ?
Freeze Output shows it was not able to figure out django.utils.six.moves. It shows 'six' in missing modules:
Missing modules:
? django.utils.six.moves imported from django.db.backends, django.db.models.base, django.db.models.sql.where, django.dispatch.dispatcher, django.forms.formsets, django.http.cookie, django.http.response, django.utils.crypto, django.utils.functional, django.utils.html_parser, django.utils.ipv6, django.utils.regex_helper, django.utils.text
? django.utils.six.moves.urllib.parse imported from django.core.files.storage, django.core.validators, django.forms.fields, django.forms.widgets, django.http.request, django.http.response, django.utils.encoding, django.utils.html, django.utils.http
If you look in the django.utils.six file you will find the following code:
...
MovedModule("http_cookiejar", "cookielib", "http.cookiejar"),
MovedModule("http_cookies", "Cookie", "http.cookies"),
MovedModule("html_entities", "htmlentitydefs", "html.entities"),
MovedModule("html_parser", "HTMLParser", "html.parser"),
...
What django does dynamically is build up the django.utils.six.moves module out of thin air. The parameters to MovedModule are . So the solution in python33 (the one I used) is to import html.entities in my Executable module. I tried to add it to the includes for the executable, but that didn't seem to work. I assume you could use the import htmlenditydefs for the python2 version.
Related
I have a package with the following structure:
model\
__init__.py (from model.main_trainer import *, etc.)
main_trainer.py
snn.py
splitter.py
The main_trainer.py script takes at least three arguments as inputs:
#main_trainer.py
import numpy as np # Linear algebra
import pandas as pd # Data wrangling
import re # Regular expressions
import matplotlib
# Avoid plotting graphs
matplotlib.use('Agg')
# Custom dependencies
from model.snn import *
from model.splitter import *
def main_trainer(dataset_name, model_dict = None, train_dict = None,
how = 'k-fold cross-validation', save = True):
etc.
if __name__ == '__main__':
dataset_name, model_dict, train_dict, how = sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4]
main_trainer(dataset_name, model_dict, train_dict, how)
However, if I run in the terminal the following:
python main_trainer.py dataset_name model_dict train_dict 'k-fold cross-validation'
I get the following error:
Traceback (most recent call last):
File "main_trainer.py", line 17, in <module>
from model.snn import *
ModuleNotFoundError: No module named 'model'
On the other hand, if I use the relative path as such:
# Custom dependencies
from .snn import *
from .splitter import *
I get this error:
Traceback (most recent call last):
File "main_trainer.py", line 17, in <module>
from .snn import *
ModuleNotFoundError: No module named '__main__.snn'; '__main__' is not a package
I have also tried running it as:
python -m main_trainer ...
and then I get this error:
Traceback (most recent call last):
File "/home/kdqm927/miniconda3/envs/siamese/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/kdqm927/miniconda3/envs/siamese/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/projects/cc/kdqm927/PythonNotebooks/model/main_trainer.py", line 17, in <module>
from .snn import *
ImportError: attempted relative import with no known parent package
I have checked these posts to no avail:
ModuleNotFoundError: What does it mean __main__ is not a package?,
Relative imports in Python 3
Append your script/module path with sys module then import your sub modules.
sys.path.append('/path/to/your/model/modules/')
Hope this will solve your problem.
Edit:
Modified your main_trainer file
#main_trainer.py
import numpy as np # Linear algebra
import pandas as pd # Data wrangling
import re # Regular expressions
import sys
import matplotlib
# Avoid plotting graphs
matplotlib.use('Agg')
# Custom dependencies
sys.path.append('/projects/cc/kdqm927/PythonNotebooks/model/') #folder which contains model, snn etc.,
from snn import *
from splitter import *
I have written a python script in python 2.7 and I am facing a problem after exe conversion of this file using py2exe package. I have got the exe file but when i run, i got an error which i have mentioned below. Please note that I have saved the script file with name test.py. Please tell me the problem #Bryan Oakley
from Tkinter import *
import os
from os.path import expanduser
import time
import datetime
from docx import Document
from docx.shared import Inches
document = Document()
docx_prefix = "Doc_File_"
home_directory = expanduser("~")
folder_directory = home_directory + "\myapp"
doc_file_directory = folder_directory + "/"
if not os.path.exists(folder_directory):
os.makedirs(folder_directory)
def create_file():
now = datetime.datetime.now()
current_time = now.strftime("%Y-%m-%d %H%M%S")
file_name = docx_prefix + current_time
document = Document()
document.add_paragraph("word document")
document.save(doc_file_directory + file_name + '.docx')
main_root = Tk()
main_root.title("Toolbar")
main_root.geometry('105x26+1+1')
toolbar = Frame(main_root, bg="gray")
toolbar.pack(side="top")
create_file_button = Button(toolbar, text="Create File", command =
create_file, bg="gray", width=10)
create_file_button.pack(side="left")
main_root.mainloop()
Setup file for exe conversion using py2exe :
from distutils.core import setup
import py2exe
setup(
console=['test.py'],
options = {'py2exe': {'packages' : ['docx']}}
)
I am getting the following error after exe conversion:
C:\Python27\Lib\site-packages\py2exe\samples\scripting\dist>test.exe
Traceback (most recent call last):
File "test.py", line 6, in <module>
File "docx\__init__.pyc", line 3, in <module>
File "docx\api.pyc", line 14, in <module>
File "docx\package.pyc", line 11, in <module>
File "docx\opc\package.pyc", line 12, in <module>
File "docx\opc\part.pyc", line 12, in <module>
File "docx\opc\oxml.pyc", line 12, in <module>
File "lxml\etree.pyc", line 12, in <module>
File "lxml\etree.pyc", line 10, in __load
File "src/lxml/lxml.etree.pyx", line 92, in init lxml.etree
(src\lxml\lxml.etree.c:225327)
ImportError: cannot import name _elementpath
Module lxml isn't pure Python. It relies on DLLs written in C. You need to add etree.pyd and objectify.pyd in the py2exe package. They are in site-packages\lxml.
I created this simple python program that sends a message to SQS and then retrieves it. It works using python 2.7.11.
import boto3
sqs = boto3.client('sqs')
queue = sqs.get_queue_by_name(QueueName='some-que-name')
queue.send_message(MessageBody='{"phrase": "It\'s the end of the world as we know it" }' )
for message in queue.receive_messages():
print message.body
I then cxFreeze it with this script:
from cx_Freeze import setup, Executable
include_mods = []
excludes = ['tkinter', 'cltk']
buildOptions = dict(packages=[], excludes=excludes, includes=include_mods)
executables = [
Executable('./frozen_boto_3_test.py', 'Console')
]
setup(name='Boto3FrozenTest',
version='1',
description='A test to make sure boto3 is working well when frozen',
options=dict(build_exe=buildOptions),
executables=executables)
I then get this error when I try to run the frozen code
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/cx_Freeze/initscripts/Console.py", line 27, in <module>
exec(code, m.__dict__)
File "./frozen_boto_3_test.py", line 1, in <module>
import boto3
File "/usr/local/lib/python2.7/site-packages/boto3/__init__.py", line 16, in <module>
from boto3.session import Session
File "/usr/local/lib/python2.7/site-packages/boto3/session.py", line 17, in <module>
import botocore.session
File "/usr/local/lib/python2.7/site-packages/botocore/session.py", line 25, in <module>
import botocore.configloader
File "/usr/local/lib/python2.7/site-packages/botocore/configloader.py", line 18, in <module>
from six.moves import configparser
File "/usr/local/lib/python2.7/site-packages/six.py", line 203, in load_module
mod = mod._resolve()
File "/usr/local/lib/python2.7/site-packages/six.py", line 115, in _resolve
return _import_module(self.mod)
File "/usr/local/lib/python2.7/site-packages/six.py", line 82, in _import_module
__import__(name)
ImportError: No module named ConfigParser
In addition to this problem, the library seem to dynamically load services that are not s3, dynamo, or one other service.
Is there a recipe to freezing boto3 ?
The error shows that a hidden (dynamic) import is taking place. If you include the module it is looking for (ConfigParser) in the list of modules you tell cx_Freeze to include, it should work. You may have to do this multiple times.
executables = [cx_Freeze.Executable("MyScript.py")]
includes = ["ConfigParser"]
buildOptions = dict(includes = includes)
cx_Freeze.setup(name, description, options = dict(build_exe = buildOptions),
executables = executables)
Once you have a working program, you can also do this instead of manipulating your specific setup.py. You can add an entry into the cx_Freeze.hooks module that looks something like this:
def load_boto3(finder, module):
finder.IncludeModule("ConfigParser")
Include any others that you discover along the way. Then create a pull request or issue over here:
https://bitbucket.org/anthony_tuininga/cx_freeze
Thanks #Anthony Tuininga
There were a few additional steps that I am posting here:
Added "ConfigParser" and "HTMLParser" as noted by Anthony
from cx_Freeze import setup, Executable
include_mods = ["ConfigParser", "HTMLParser"]
excludes = ['tkinter', 'cltk']
buildOptions = dict(packages=[], excludes=excludes, includes=include_mods)
executables = [
Executable('./frozen_boto_3_test.py', 'Console')
]
setup(name='Boto3FrozenTest',
version='1',
description='A test to make sure boto3 is working well when frozen',
options=dict(build_exe=buildOptions),
executables=executables)
export AWS_DATA_PATH=/usr/local/lib/python2.7/dist-packages/botocore/data
from:
requests.exceptions.SSLError: [Errno 2] No such file or directory
export REQUESTS_CA_BUNDLE=/usr/local/lib/python2.7/dist-packages/botocore/vendored/requests/cacert.pem
Used lower level code to use sqs:
import boto3
sqs = boto3.client('sqs')
queue = sqs.create_queue( QueueName="my-queue" )
sqs.send_message(QueueUrl=queue["QueueUrl"] , MessageBody='{"phrase": "It\'s the end of the world as we know it" }' )
message = sqs.receive_message(QueueUrl=queue["QueueUrl"])
for msg in message['Messages']:
print m
sg['Body']
After this sqs works for the 'hello world' script posted on the OP
Copying the import lines from tap.py...
tap.py:
import site
import user
from site import security, agent
from site.security import models
When I run this i am getting:
File "/home/vikasadmin/user/tap.py", line 31, in init
from site import security, agent
File "/usr/lib/python2.6/site-packages/site/security.py", line 9, in <module>
from site import models
File "/usr/lib/python2.6/site-packages/site/models.py", line 12, in <module>
from site import config
ImportError: cannot import name config
Since I already imported models from site.security this config should have been included. Please explain? How can I solve this?
I have a script which requires multiprocessing. What I found from this script is that there is a problem with the multiprocessing module. To test this theory, I copied and pasted
from multiprocessing import Process
def f(name):
print('hello', name)
if __name__ == '__main__':
p = Process(target=f, args=('bob',))
p.start()
p.join()
into a test script and received the following traceback
Traceback (most recent call last):
File "a.py", line 1, in <module>
from multiprocessing import Process
File "/usr/lib64/python3.3/multiprocessing/__init__.py", line 40, in <module>
from multiprocessing.util import SUBDEBUG, SUBWARNING
File "/usr/lib64/python3.3/multiprocessing/util.py", line 16, in <module>
import threading # we want threading to install it's
File "/usr/lib64/python3.3/threading.py", line 11, in <module>
from traceback import format_exc as _format_exc
File "/usr/lib64/python3.3/traceback.py", line 3, in <module>
import linecache
File "/usr/lib64/python3.3/linecache.py", line 10, in <module>
import tokenize
File "/usr/lib64/python3.3/tokenize.py", line 30, in <module>
from token import *
File "/home/lucas/Server/ClinApp/weblabs/utils/token.py", line 1, in <module>
from django.conf import settings
File "/usr/lib/python3.3/site-packages/django/conf/__init__.py", line 9, in <module>
import logging
File "/usr/lib64/python3.3/logging/__init__.py", line 195, in <module>
_lock = threading.RLock()
AttributeError: 'module' object has no attribute 'RLock'
Also, I am running fedora 18 64-bit on a quad core ivy bridge. Why am I receiving this traceback error?
Suggestion
Here is what happens when I run RLock
$ python3
>>> import threading
>>> threading.RLock()
<_thread.RLock owner=0 count=0>
>>>
File "/usr/lib64/python3.3/tokenize.py", line 30, in <module>
from token import *
File "/home/lucas/Server/ClinApp/weblabs/utils/token.py", line 1, in <module>
from django.conf import settings
Your /home/lucas/Server/ClinApp/weblabs/utils/token.py script is being imported instead of the standard python 'token.py'. Its got a bug in it or it simply shouldn.t be imported as a top level script. You probably have /home/lucas/Server/ClinApp/weblabs/utils/ in your python path in some way.
Generally, its not a good idea to name python scripts after builtin scripts.
After you rename token.py to something else (get_token.py), remember to delete token.pyc in your local working directory. Or else, you will continue to get the Traceback error message you listed above.