I have a module that I am testing that depends on another module that won't be available at the time of testing. To get around this, I wrote (essentially):
import mock
import sys
sys.modules['parent_module.unavailable_module'] = mock.MagicMock()
import module_under_test
This works fine as long as module_under_test is doing one of the following import parent_module, import parent_module.unavailable_module. However, the following code generates a traceback:
>>> from parent_module import unavailable_module
ImportError: cannot import name unavailable_module
What's up with this? What can I do in my test code (without changing the import statement) to avoid this error?
Alright, I think I've figured it out. It appears that in the statement:
from parent_module import unavailable_module
Python looks for an attribute of parent_module called unavailable_module. Therefore, the following set up code fully replaces unavailable_module within parent_module:
import mock
import sys
fake_module = mock.MagicMock()
sys.modules['parent_module.unavailable_module'] = fake_module
setattr(parent_module, 'unavailable_module', fake_module)
I tested the four import idioms of which I am aware:
import parent_module
import parent_module.unavailable_module
import parent_module.unavailable_module as unavailabe_module
from parent_module import unavailable_module
and each worked with the above set up code.
Related
e.g.
import os as my_os
import my_os.path
ModuleNotFoundError: No module named 'my_os'
but the following script is ok
import os
import os.path
You can't do that in Python.
import statements are importing from Python file names.
You aren't renaming the file of os to my_os, therefore this wouldn't work.
As mentioned in the documentation:
The import statement combines two operations; it searches for the named module, then it binds the results of that search to a name in the local scope.
Tried the similar way as os.py did:
import json as my_json
from my_json.decoder import * # ModuleNotFoundError: No module named 'my_json'
import sys
import json.decoder as my_decoder
sys.modules['my_json.decoder'] = my_decoder
from my_json.decoder import * # it's ok now
I create a python console app that includes imports of a custom class I'm using. Everytime I run my app I get the error ModuleNotFoundError: "No module named 'DataServices'.
Can you help?
Provided below is my folder structure:
ETL
Baseball
Baseball_DataImport.py
DataServices
DataService.py
ConfigServices.py
PageDataMode.py
SportType.py
Here is the import section from the Baseball_DataImport.py file. This is the file when I run I get the error:
from bs4 import BeautifulSoup
import scrapy
import requests
import BaseballEntity
import mechanize
import re
from time import sleep
import logging
import time
import datetime
from functools import wraps
import json
import DataServices.DataService - Error occurs here
Here is my DataService.py file:
import pymongo
import json
import ConfigServices
import PageDataModel
#from SportType import SportType
class DataServices(object):
AppConfig: object
def __init__(self):
AppConfig = ConfigServices.ConfigService()
#print(AppConfig)
#def GetPagingDataBySport(self,Sport:SportType):
def GetPagingDataBySport(self):
#if Sport == SportType.BASEBALL:
pagingData = []
pagingData.append(PageDataModel.PageDataModel("", 2002, 2))
pagingData.append(PageDataModel.PageDataModel("", 2003, 2))
return pagingData
It might seem that your structure is:
Baseball
Baseball_DataImport.py
Dataservices
Dataservice.py
Maybe you need to do from Dataservices.Dataservice import DataServices
Edit:
I created the folder structure, and the method I showed you works:
Here's the implementation
Dataservice.py only contains:
class DataServices():
pass
Did you try copieing the Dataservice.py into the Projectfolder with the main.py?
How do I import from a higher level directory in python?
For example, I have:
/var/www/PROJECT/subproject/_common.py
/var/www/PROJECT/subproject/stuff/routes.py
I want to import variable A in _common.py to routes.py
# routes.py
import os, sys
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
from _common import A
but I get the error:
ImportError:cannot import name 'A'
Change file directory:
import os, sys
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__),"../../project")))
from _common import A
OLD VERSION
To solve the issue replace ".." with os.pardir:
import os, sys
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir)))
from _common import A
NEW VERSION
The code above does not solve the problem in the question because the true problem lies in the project structure not in the particular line. The problem is circular import. The problem became clear after the full traceback has been provided. Here is the simple way to reproduce the issue - consider 3 files...
main.py:
import a
a.py:
import b
A = 'A'
b.py:
from a import A
... the error is:
ImportError: cannot import name 'A'
OR
b.py:
import a
BB = a.A
... the error is:
AttributeError: module 'a' has no attribute 'A'
The solution to the problem has been discussed many times - search on SO
I have three files under the same directory, namely, main.py, Newtester.py, and fileUtility.py. In Newtester.py there is a class named Function. In main.py, there are the following codes:
from file.py import *
...
def main():
...
funcs = parseFuncSpec(funcInputFile)
parseFuncSpec is defined in fileUtilities.py as:
some code to import Newtester.py
def parseFuncSpec(fName):
curFunc = function(funcName, numTest, [], score)
Regardless of what I put in import Newtester.py, I always get an error saying "Function" (the class defined in the file "Newtester.py") is not defined. Following Python: How to import other Python files, I have attempted
import Newtester
__import__("Newtester")
exec("Newtester.py")
exec("Newtester")
import importlib
importlib.__import__("Newtester")
os.system("Newtester.py")
But none of them seemed to work. Any advice is appreciated. See https://github.com/r2dong/unitTesting if you are interested in seeing the complete files.
It's because you are not using it correctly
well when you use import statement like below only Newstester file is imported
import Newtester
hence instead of using parseFuncSpec() directly you have to use it as Newtester.parseFuncSpec()
or to use parseFuncSpec() directly you need to use below import statement:
from Newtester import parseFuncSpec
I was having a play around with Python 2.7 and everybody knows that at the start of every program, you always have to import modules. For example:
import random
import time
for x in range(1, 300):
print random.randint(1,100)
time.sleep(1)
print "Done!"
Anyway, I was thinking, why do I have to import all my modules manually? Why doesn't Python just import them all like this.
Sure, I can understand why it does not import like this:
from random import randint
from time import *
for x in range(1, 300):
print randint(1,100)
sleep(1)
print "Done!"
As some function names may clash. But, if you have to define where the function is at the start, so for example random. in random.randint(1,100).
Now modern computers are so powerful, it seems logical to import every module automatically instead of wasting lines of code, and time by having to find which module you need then importing it manually when it can easily be automated.
So, why does python not import every module at startup automatically?
EDIT:
I have made a new version of a little program that imports every module that I can find by running:
import sys
sys.builtin_module_names
Here are the results:
x = int(1000000)
def test():
global x
x -= 1
print "Iterations Left: ", x
import __builtin__
import __main__
import _ast
import _bisect
import _codecs
import _codecs_cn
import _codecs_hk
import _codecs_iso2022
import _codecs_jp
import _codecs_kr
import _codecs_tw
import _collections
import _csv
import _functools
import _heapq
import _hotshot
import _io
import _json
import _locale
import _lsprof
import _md5
import _multibytecodec
import _random
import _sha
import _sha256
import _sha512
import _sre
import _struct
import _subprocess
import _symtable
import _warnings
import _weakref
import _winreg
import array
import audioop
import binascii
import cPickle
import cStringIO
import cmath
import datetime
import errno
import exceptions
import future_builtins
import gc
import imageop
import imp
import itertools
import marshal
import math
import mmap
import msvcrt
import nt
import operator
import parser
import signal
import strop
import sys
import thread
import time
import xxsubtype
import zipimport
import zlib
def start():
from timeit import Timer
t = Timer("test()", "from __main__ import test")
print t.timeit()
start()
Because you don't need all of it. There is no point in loading every library if you don't need them.
EDIT:
I copied my libs folder to a test directory and made it into a package by adding an __init__.py file to it. In this file I added:
import os
import glob
__all__ = [ os.path.basename(f)[:-3] for f in glob.glob(os.path.dirname(__file__)+"/*.py")]
I created a test script that contains:
from Lib import *
print('Hello')
When I try to run it in the shell all it does is print 'The Zen of Python' by Tim Peters, opens this webcomic in my browser (2 things I absolutely did not see coming) and throws the following error:
Traceback (most recent call last):
File "C:\Users\Hannah\Documents\dropBox\Python\test\test.py", line 1, in <module>
from Lib import *
AttributeError: 'module' object has no attribute 'crypt'
It takes a noticable amount of time before it does any of this, about 10-15 seconds
Maybe what you would like is a feature that automatically imports the libraries that are used in your script without needing to specify them at the beginning. I found this on the Internet http://www.connellybarnes.com/code/autoimp/
You just need one import at the beginning of your script
from autoimp import *
All other modules are loaded "lazily", i.e. when they are first used.
Example in the interactive shell:
>>> random.random()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'random' is not defined
>>> from autoimp import *
>>> random.random()
0.0679000238267422
From the docs:
For ultimate laziness, place the command "from autoimp import *" in your PYTHONSTARTUP file. Now your interactive session has all modules available by default.
Every module you import takes time to import. Importing every built-in module every time you start Python would kill performance in a lot of important scenarios where new Python interpreters are started frequently.
Python does have a set of modules that are always loaded, its call __builtins__ :).
Python's builtins provide the import statement for you to extend your scope with even more modules! But as other posts have said, deciding your script needs these modules it up to you. -- I have looked into mutating __builtins__ and I promise you, explicitly importing what you need is the better option.
((Big rant about not using from name import * cut from here))
Since most of writing python ultimately becomes packaging and installing that writen python somewhere, this is my goto set of resources for getting a handle on python's infamous import:
Start by sticking to standard tools and libraries (https://packaging.python.org/current/)
Reading and understand The Google Python Standards Guide (https://google.github.io/styleguide/pyguide.html),
Read the Zen of Python (https://www.python.org/dev/peps/pep-0020/)
Be Pythonic (basically adhere to "The Zen of Python"), https://www.youtube.com/watch?v=wf-BqAjZb8M
Supplement your problem-space with tips from The Hitchhikers Guid to Python (http://docs.python-guide.org/en/latest/)
Be preapred to package your code (https://packaging.python.org/distributing/ (Doc), https://github.com/pypa/sampleproject/ (Example))
Being prepared to debug someone else's and, Your own code by getting familiar with tools like:
pdb (import pdb; pdb.set_trace(), > pp variable),
print(help(variable)),
dir(variable),
and pprint.pprint( variable.__dict__ )