I have structure project python
MyProject
+Classes
-stage_competition.py
main.py
Executing Python (main.py) - Show error "ModuleNotFoundError("No module named 'Classes'",)"
I tried import direct, but is not working too, always showing same error
from Classes.stage_competition import Stage_competition
item = Stage_competition("field1", "field2", "field3" , "field4")
Create class Stage_competition
class Stage_competition(object):
"""description of class"""
def __init__(self, type_competition, datalake, competition ,hour):
self.type_competition = type_competition
self.datalake = datalake
self.competition = competition
self.hour = hour
def settype_competition(self, type_competition):
self.type_competition = type_competition
def setdatalake(self, datalake):
self.datalake = datalake
def getcompetition(self):
return self.competition
def gethour(self):
return self.hour
If the Classes is in the same folder as main.py, you should import it this way (notice the dot before Classes):
from .Classes.stage_competition import Stage_competition
Alternatively, you could add directory to Classes to PYTHONPATH, which allows you to import Classes from anywhere.
Related
I have a directory of code filled in like this:
root_dir
├── root.py
├── same_step_0.py
├── same_step_1.py
├── same_step_2.py
└── utils
├── __init__.py
├── file_one.py
└── file_two.py
The root.py file references the functions in the same_step files. For example:
# In root.py
same_step_0.generated_function()
The "step" files have references to functions in the 'utils' directory, with a simple import and call - e.g.
from utils.file_one import function_one
function_one(val="somelongstring")
Ok, now we get to the problem - I'm working against an API which effectively requires each "step" to be a single python file. So, what I need to do is, somehow, append all functions pulled in by each step file into a single file which I can then submit to the service.
IDEALLY, I'd love to do this with as little changes to the underlying code. So, if possible, in same_step_0 I'd like to leave it as:
from utils.file_one import function_one
function_one(val="somelongstring")
and not have to change it to
# Note removing the 'utils' prefix
from file_one import function_one
function_one(val="somelongstring")
So, the question is, is there a way I can append all the dependent files or functions together into a single file such that it mimic'd import behavior? Or at least function call behavior? Ideas I've thought about:
Pickling all subfiles into a serialization format and inserting at the end of the code
Raw appending of all python files - perhaps inside an inline module?
Scraping all files for functions and attaching them manually (however, this would not preserve namespacing)
I haven't dug into these yet - I'm curious if this is even possible
I can do a lot of generation and wrapping of code, but I'd prefer not to touch the inner code (the function_one call, for example).
Goal
To create a minimal set of scripts, and an interchange format that can be stored in a Python executable to recreate a module structure.
Source
Ok here's a simple example that abuses types.ModuleType, sys, and globals() to do this:
import os
import sys
import types
class Module:
def __init__(self, name, source=None, path=None, modules=None):
self.name = name
self.source = source
self.path = path
self.modules = modules
def from_file(name, path):
return Module(name, open(path).read(), path)
def from_dir(name, path):
return Module(name, None, path, [])
def __repr__(self):
return f'Module(name={repr(self.name)}, source={repr(self.source)}, path={repr(self.path)}, modules={self.modules})'
def to_dict(self):
data = { 'name': self.name }
if self.source is not None:
data['source'] = self.source
if self.path is not None:
data['path'] = self.path
if self.modules is not None:
data['modules'] = [i.to_dict() for i in self.modules]
return data
#staticmethod
def from_dict(data):
modules = None
if 'modules' in data:
modules = [Module.from_dict(i) for i in data.pop('modules')]
return Module(**data, modules=modules)
def get_modname(parent, module):
if parent is None:
return module
if module == '__init__':
return parent
return f'{parent}.{module}'
def compile_module(module, parent=None):
modname = get_modname(parent, module.name)
mod = types.ModuleType(modname)
exec(module.source, mod.__dict__)
path = os.path.realpath(module.path)
mod.__path__ = os.path.dirname(path)
mod.__file__ = path
sys.modules[modname] = mod
globals()[modname] = mod
return mod
def compile_module_recursive(package, parent=None):
# Need to do this recursively.
mod = compile_module(package, parent)
if not package.modules:
return mod
for submodule in package.modules:
submod = compile_module_recursive(submodule, parent=get_modname(parent, package.name))
if not hasattr(mod, submodule.name):
setattr(mod, submodule.name, submod)
return mod
def read_module_recursive(directory, parent=None):
# The module order is first the:
# 1. All submodules
# 2. Then definitions inside
# Then, need to define `__name__`, `__path__` and `__file__`.
cwd = os.getcwd()
realpath = os.path.realpath(directory)
parent_dir = os.path.dirname(realpath)
base_dir = os.path.basename(realpath)
if parent is None:
parent = Module.from_dir(base_dir, realpath)
os.chdir(realpath)
for entry in os.listdir(realpath):
path = os.path.join(realpath, entry)
if os.path.isfile(path):
name, ext = os.path.splitext(entry)
if not ext == '.py':
continue
if name == '__init__':
parent.path = path
parent.source = open(path).read()
else:
parent.modules.append(Module.from_file(name, path))
elif os.path.isdir(path):
if entry == '__pycache__':
continue
path = os.path.join(realpath, entry)
# Must have processed __init__.py
if not os.path.isfile(f'{path}/__init__.py'):
continue
module = Module.from_dir(entry, path)
parent.modules.append(module)
read_module_recursive(entry, module)
os.chdir(cwd)
return parent
# SAMPLE USE
# ----------
# module = read_module_recursive('mylib')
# data = module.to_dict() # can store as a Python dict.
# module = Module.from_dict(data)
# compile_module_recursive(module)
How it works
Basically, it reads all .py files from all modules, recursively. It currently does not support extension types, only pure Python files. It then creates an intermediate, tree-like type for the entire module tree. It provides easy serialization to and from dict, so it's much easier to copy-and-paste it into your executable file.
Example use
Say I have the following directory structure:
mylib/
__init__.py
a.py
b.py
c/
__init__.py
d.py
It properly respects the import order, and how it variable assignments override imports or submodules. Please note that it imports everything recursively all at once, which differs than the traditional import structure.
mylib/init.py
a = 'e'
mylib/a.py
def afunc(a):
return str(a)
class AClass:
def x(self):
return 1
mylib/b.py
def bfunc(b):
return repr(b)
class BClass:
def x(self):
return 1
mylib/c/init.py
mylib/c/d.py
def dfunc(d):
return str(d)
class DClass:
def x(self):
return 1
Example
>>> module = read_module_recursive('mylib')
>>> data = module.to_dict() # can store as a Python dict.
>>> module = Module.from_dict(data)
>>> compile_module_recursive(module)
>>> mylib
<module 'mylib' from 'C:\\Users\\user\\OneDrive\\Desktop\\lib\\mylib\\__init__.py'>
>>> mylib.a
'e'
>>> from mylib.a import afunc # still works
>>> afunc(54)
'54'
>>> import mylib # works because `'mylib'` is in `sys.modules`.
Congrats: this is a hack, but it works, and it works nicely, and it uses Python's own packaging system to do everything, so it's much more resilient than copying and pasting. After compiling, it also respects imports (due to adding them to sys.modules). It also respects __doc__, and other attributes.
License
Public Domain or Unlicensed. Use as you see fit. Attribution would be nice, but not required.
I'm using Django and Python 3.7. I have two applications in my project -- common and mainsite . In common, I have a file "model_utils.py" at the root of my application (right next to models.py). It contains this code
class RawCol(Expression):
def __init__(self, model, field_name):
field = model._meta.get_field(field_name)
self.table = model._meta.db_table
self.column = field.column
super().__init__(output_field=CharField())
def as_sql(self, compiler, connection):
sql = f'"{self.table}"."{self.column}"'
return sql, []
How do I reference this class from my other application, "mainsite"? I put this at the top of one of my files in mainsite ..
from common import RawCol
but when I run some tests, I get this error ...
ImportError: cannot import name 'RawCol' from 'common' (/Users/davea/Documents/workspace/mainsite_project/common/__init__.py)
Edit: Project structure at the top level directories looks like ...
+ common
+ mainsite
+ mainsite_project
+ manage.py
+ templates
+ venv
Try from common.model_utils import RawCol
instead of from common import RawCol
You always need to specify the exact .py file (without the .py ending) to import from.
If it still doesn't work, it can be a circular import problem.
If you try to import something from mainsite.model into common.model_utils and the other way around, you are creating an impossible import loop.
You can fix this by creating a seperate file like common/dependent_model.py and putting only the RawCol() class in there without any import from mainsite. Like this, the both files are not importing from each other (which doesn't work).
I have to admit that I am fairly a newbie to python. This question is more related to code organization.
For example, here is app.py file
import backtracker as bt
# Create a class
class TestTracker(bt.Strategy):
def log(self, txt, dt=None):
''' Logging function for this strategy '''
dt = dt or self.datas[0].datetime.date(0)
print('%s, %s' % (dt.isoformat(), txt))
def __init__(self):
self.dataclose = self.datas[0].close
self.datapoint = self.datas[0]
def next(self):
self.log('Close, %2f' %self.dataclose[0])
Now, I would like to separate class file to testStrategy.py and import it into app.py. The folder structure would look something like this
|
|-strategies
| |-testStrategy.py
|-app.py
How do I reference base class bt.strategy in my new class file ?
Thank you for your help.
There are two ways you can go about it.
Add the folder strategies to sys.path and then import your class.
# This gives you the folder in which current file resides
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
if BASE_DIR + '/strategies' not in sys.path:
sys.path.append(BASE_DIR + '/strategies')
from testStrategy import TestTracker
Add __init__.py in every folder and then use your code as a python package. This allows you to import in the following way.
import strategies.testStrategy.TestTracker
Be wary of the trap though.
I have defined 2 classes- Person & Manager. The Manager inherits the Person class.
I get a error while trying to import the Person class..
Code is give below.
Person.py
class Person:
def __init__(self, name, age, pay=0, job=None):
self.name = name
self.age = age
self.pay = pay
self.job = job
def lastname(self):
return self.name.split()[-1]
def giveraise(self,percent):
#return self.pay *= (1.0 + percent)
self.pay *= (1.0 + percent)
return self.pay
Manager.py
from Basics import Person
class Manager(Person):
def giveRaise(self, percent, bonus=0.1):
self.pay *= (1.0 + percent + bonus)
return self.pay
Error statements:
C:\Python27\Basics>Person.py
C:\Python27\Basics>Manager.py
Traceback (most recent call last):
File "C:\Python27\Basics\Manager.py", line 1, in
from Basics import Person
ImportError: No module named Basics
Why do I get the No module found error?
You should look up how import and PYTHONPATH work. In your case, you can solve that using:
from Person import Person
I see you're coming from a Java background (where each file must have a class with the same name of the file), but that's not how Python modules work.
In short, when you run a Python script from the command line, as you did, it looks for modules (among other places) in your current dir. When you import a (simple) name like you did, Python will look for:
A file named Basic.py; or:
A folder named Basic with a file named __init__.py.
Then it will look for a definition inside that module named Person.
Because it's in Person.py, not Basics.py.
from Person import Person
You defined the Person class in a file named Person.py. Therefore, you should import it like this:
from Person import Person
Note that it's the convention in Python to have lowercase module names. For example, rename Person.py to person.py and Manager.py to manager.py. Then you'd import the Person class like this:
from person import Person
If the person module is part of a package, you'd probably want to import like this:
from .person import Person
This will ease the transition to Python 3.
from Basics import Person should be from Person import Person. You don't have a Basics.py module to import from.
There are given two versions of a storage. Based on the version, I need to select the proper interface module to get the result.
The file structure looks like this:
lib/
__init__.py
provider.py
connection.py
device.py
storage/
__init__.py
interface_v1.py # interface for storage of version 1
interface_v2.py # interface for storage of version 2
main.py
The main.py imports provider.py, that should import one of the interfaces listed in the storage subpackage depending on the version of the storage.
main.py:
from lib.provider import Provider
from lib.connection import Connection
from lib.device import Device
connection = Connection.establish(Device)
storage_version = Device.get_storage_version()
massage = Provider.get_data(connection)
provider.py should import an interface to the storage based on storage_version and implement provide some functions:
from storage import interface
class Provider(object):
def __init_(self):
self.storage = interface.Storage
def get_data(self, connection):
return self.storage.get_data()
def clear_storage(self, connection):
self.storage.clear_storage()
This example is not complete, but should be sufficient for the problem explanation.
Additional question:
Is it possible to use storage.__init__ to use import just the
subpackage?
How to proper implement Factory in Python?
Assuming the interface_v1 and interface_v2 bith impleement a class StorageClassI guess something like this:
import storage.interface_v1
import storage.interface_v2
class Provider(object):
def __init__(self , version):
if version == 1:
self.storage = storage.interface_v1.StorageClass
else:
self.storage = storage.interface_v2.StorageClass
Would be the best solution - but https://docs.python.org/2/library/functions.html#import should provide a way to import a module based on name.