From today I have funny problem. Everything was working just fine for months and today I want to start my program as usual and during import it just exits. There is no error or nothing. I tried over console and in pycharm but result is the same. It seems problem is inside init.py but during import from projects.
import of class looks like this:
from engine.database import Database
if I try directly to use Database class from the database.py it works. What has changed, I'm using python 3.6?
EDIT:
in engine directory I have init.py like this:
# Database imports
from engine.database import Database
# Tor imports
from projects.Common.tor_network import TorBuild
from projects.Common.tor_network import Check
from projects.Common.tor_network import kill_all
if I comment imports from project it works.
Because imports from projects doesn't work. Here is other init.py files
init.py in projects looks like this:
from projects.Common import *
and inside projects/Common init.py looks like this:
from os import listdir
from os.path import abspath, dirname, isfile, join
# get location of __init__.py
init_path = abspath(__file__)
# get folder name of __init__.py
init_dir = dirname(init_path)
# get all python files
py_files = [file_name.replace(".py", "") for file_name in listdir(init_dir) \
if isfile(join(init_dir, file_name)) and ".py" in file_name and not ".pyc" in file_name]
# remove this __init__ file from the list
py_files.remove("__init__")
__all__ = py_files
EDIT2:
So source of the problem is located in projects/Common init.py.
If I replace the that code only with __all__ = ['tor_network'] it will work. But that would mean I have to install manualy everytime I add program. That is really funny thing. I worked for moths now it doesnt.
EDIT3:
Ok I found the problem... One of program inside Common was runing function (I forgot to comment later) that exits after run. I forgot on that one.
Thanks for help!
Related
I have the following structure:
LICENSE.md
README.md
requirements.txt
src
routes
route_a.py
__ init __.py
util
__ init __.py
db.py
And in db.py, I have something that looks like this:
import mysql.connector
def get_value():
# Query database using mysql.connector
return value
value = get_value()
def query_that_uses_value(value):
# do stuff with value
return value2
I want to be able to use value inside of route_a.py and also inside of other functions in db.py. What's the best way to do this?
import sys
sys.path.insert(0, "path")
that's how I did it.
the path is the folder u want to use I would choose the main folder so u have the same starting point for your imports.
in your case, the import would look like this
from util.dp.py import get_value
if you have this at the start of your programme that u want the function imported to
sys.path.insert(0,"path_to_src/src")
but path to src must be an absolut path beginning from your root folder
In route_a.py, simply import the variable value (and other functions you need) from ..util.db, which is a relative import that will reference src/util/db.py. Here's what the file src/routes/route_a.py should contain:
from ..util.db import value, function1, function2
Best way to deal with imports is to export PYTHONPATH=$(pwd) in your project root directory where there are src, requirements.txt, etc.
So in your terminal, run export PYTHONPATH=$(pwd) and all your imports should be consistent and start from src.
For example:
from src.util.db import value
from src.routes.route_a import something
Note that everytime you open a new terminal you should run export PYTHONPATH=$(pwd) cause this is not permanent. When your terminal disappears your PYTHONPATH should reset and that's a good thing and is best practice.
Also don't forget to run everything from src. Like: python src/util/db.py
If you follow this structure, I promise you will never have any import problems.
Well, I have pretty hard task and I'm completely stucked, like in any direction.
What program should do:
Import all modules (names are random) from folder
MainScript.py
modules/
mod1.py
mod2.py
mod3.py
...
Execute specific (known name, and everywhere it's same) function.
mod1.main()
mod2.main()
mod3.main()
...
As I understand it, I should list all files in folder , then make list with them and for each [x] in list import module and execute script. I've found that modules[0].main() works only if modules[0] no string, so, it should be modules[0]=main not modules[0]='main'. So and there I need somehow deal with it... but for import I don't know...
I've already googled about it, only found https://stackoverflow.com/a/1057534/10289135
And I guess it will not work for me (I also don't understand how it works and script didn't work for me)
Any ideas?
You can use the following syntax:
from filename(remove the .py) import *
This is a wild card import it imports every thing from a module literally everything .By doing this you dont need to do the work like 'filename.blabla' ,but simply you can do 'blabla'.
import os
import sys
import importlib
modules = []
for i in os.listdir("C:\\Windows\\path\\to\\your\\modules\\"):
mod = i
modules.append(mod)
sys.path.append("C:\\Windows\\path\\to\\your\\modules\\")
for i in modules:
i = i[:i.find(".")]
module = importlib.import_module(f"{i}")
module.main()
I spent some time researching this and I just cannot work this out in my head.
I run a program in its own directory home/program/core/main.py
In main.py I try and import a module called my_module.py thats located in a different directory, say home/program/modules/my_module.py
In main.py this is how I append to sys.path so the program can be run on anyone's machine (hopefully).
import os.path
import sys
# This should give the path to home/program
sys.path.append(os.path.join(os.path.abspath(os.path.dirname(__file__), '..'))
# Which it does when checking with
print os.path.join(os.path.abspath(os.path.dirname(__file__), '..')
# So now sys.path knows the location of where modules directory is, it should work right?
import modules.my_module # <----RAISES ImportError WHY?
However if I simply do:
sys.path.append('home/program/modules')
import my_module
It all works fine. But this is not ideal as it now depends on the fact that the program must exist under home/program.
that's because modules isn't a valid python package, probably because it doesn't contain any __init__.py file (You cannot traverse directories with import without them being marked with __init__.py)
So either add an empty __init__.py file or just add the path up to modules so your first snippet is equivalent to the second one:
sys.path.append(os.path.join(os.path.abspath(os.path.dirname(__file__), '..','modules'))
import my_module
note that you can also import the module by giving the full path to it, using advanced import features: How to import a module given the full path?
Although the answer can be found here, for convenience and completeness here is a quick solution:
import importlib
dirname, basename = os.path.split(pyfilepath) # pyfilepath: /my/path/mymodule.py
sys.path.append(dirname) # only directories should be added to PYTHONPATH
module_name = os.path.splitext(basename)[0] # /my/path/mymodule.py --> mymodule
module = importlib.import_module(module_name) # name space of defined module (otherwise we would literally look for "module_name")
Now you can directly use the namespace of the imported module, like this:
a = module.myvar
b = module.myfunc(a)
Short:
How can I import a module if a different module with the same name has already been imported? I don't need the old module anymore, but I can't import the new one under a different name.
importlib.reload() doesn't have the desired effect and apparently you really shouldn't mess with sys.modules
Context:
I'm automating a testing workflow by writing a script (A). The test script is provided and can not be altered.
The target of the Testing is a class in a script (B) of which there are different versions in subfolders. Unfortunately Script B and the class always have the same name, nothing I can do about that.
main_folder
├──script_a.py
├──test_script.py
│
├──version_1
│ ├──script_b.py
│
├──version_2
│ ├──script_b.py
│
├──version_3
│ ├──script_b.py
The test script imports the object and runs tests on it.
# test_script
from script_b import my_class
# creat instance of my_class
# and test it
script_a iterates over the version folders and runs the test script on script_b.
In each iteration one subfolder is added to the import path so the test script will find the corresponding script_b. The path is removed after the iteration.
If sys.modules already contains a version of test_script,
# script_a
import sys
import os
import importlib
folders = ['version_1', 'version_2', 'version_3']
folders = [os.getcwd() + '/' + folder for folder in folders]
for folder in folders:
sys.path.append(folder)
if 'test_script' in sys.modules:
importlib.reload(sys.modules['test_script'])
else:
importlib.import_module('test_script')
sys.path.remove(folder)
Issue:
It seems reload has no effect on script_b, which is imported by test_script. So although I change the import path to different subfolders test_script always runs on version 1.
How can I make test_script use the different versions of script_b without altering test_script itself?
Follow up:
Although the original question is answered, I was wondering, how is this solution from a design perspective? Is there a better / more elegant way to automate this testing process?
From what I found, it isn't considered good practice to reload modules.
Figured it out while writing the question. Hope it will help someone some day
Even if you can't alter test_script (or script_b) there's a work around.
Because an import statement does nothing if the relevant module is already imported we can reload script_b from the path we want directly in script_a. Since it then is replaced with the new version in sys.modules the import statement in test_script will cause no problem.
Updated code:
# script_a
import sys
import os
import importlib
folders = ['version_1', 'version_2', 'version_3']
folders = [os.getcwd() + '/' + folder for folder in folders]
for folder in folders:
sys.path.append(folder)
if 'test_script' in sys.modules:
importlib.reload(sys.modules['script_b']) # this line added
importlib.reload(sys.modules['test_script'])
else:
importlib.import_module('test_script')
sys.path.remove(folder)
I have a directory structure like
Documents/
Project_1/
module_1.py
Project_2/
module_2.py
If I want to import module_1.py into module_2.py, what is the syntax for it?
I have tried variations on
import ../Project_1/module_1
And
from .Project_1 import module_1
But I can't get it to work and have only done this once before in another project.
You have two alternatives;
from Documents.project_1 import module1.py
or
import Documents.project_1.module1.py
It is a crude solution, but I ended up with something like this:
#This thing modifies the path
from sys import path
#This gets our user directory
from os import environ
#add module_1's location to path
path.append(environ['USERPROFILE'] + "\\Documents\\Project_1")
#import it now
import module_1
This is not the most elegant solution, but it can work on almost any Windows machine, assuming the folders are placed in their Documents.
The code about Environ could be reasonably replaced to match another directory, though.