Mock environment variables inside __init__ file - python

have a question about mocking environment variables
The module has this structure
sharepoint
|-- __init__.py (Here I initialize some variables I used in get_file and get_token)
|-- get_file.py
|-- get_token
__init__.py
main.py
So, I'm trying to test some get_file methods, the first line you see in get_file.py is this one:
from . import SHAREPOINT_URL, FOLDER_PATH, LIMIT_HOURS, CONTENT_TYPE
As you can see, I get them from environment in __init__.py
FULL_CLIENT, SECRET, SHAREPOINT_URL, FOLDER_PATH, LIMIT_HOURS = [environ[k] for k in ['FULL_CLIENT', 'SECRET', 'SHAREPOINT_URL', 'FOLDER_PATH', 'LIMIT_HOURS']]
When I'm trying to unit test a method, an error appears because I didn't set the environment variables before.
ERROR tests/test_sharepoint.py - KeyError: 'FULL_CLIENT'
I've already tried to mock with
#mock.patch.dict(os.environ, {"FULL_CLIENT": 'full_client', 'SECRET': 'secret', 'SHAREPOINT_URL': 'url', 'FOLDER_PATH': 'path', 'LIMIT_HOURS': 'hours'})
but it seems that it's unreachable for unit test to mock it before enters to the get_file.py.
Can you help me with that?

I guess you should read this:
How to mock an import
However, the good practice is to wrap the retrieval of the environment variables inside dedicated function and mock this function instead. This is clean and universal solution.

Related

Python class with a decorator is being flagged by pylint as unused-import

For example, let's say I have three Python files; one file is the init.py which has routines for creating the namespaces of my Flask API and initializing the Flask app:
from flask import Flask
app = Flask(...)
... initialize the namespace etc. ...
... and another Python file with the definitions of the API Resource subclasses that exist in the enpoint.py file. It contains a few Python classes that make use of decorators from Flask-RESTX to wire up the endpoints:
#namespace.route("/path")
class EndpointApi:
def get():
...
The third file is main.py which simply starts the Flask server running. Unfortunately for me though, it contains an import which is flagged by pylint.
My app is working fine, but when I run pylint, it tells me there are unused imports. If I remove the imports, then the logic in the decorator that adds the route to the Flask API does not execute, and the result is that the endpoint is no longer added to the API.
Is there some way to add a class file (like endpoints.py) without importing it? I want pylint to stop warning me about unused imports, when clearly I'm using the decorator to call some global function that adds the API Resource handlers to Flask.
Sure, I could ignore the pylint error with a comment, but is there a better way? I am truly disgusted with placing a comment on every line of an import statement which I'm sure is not an "unused-import" (I have about 30).
Obviously, I could just refactor the decorator pattern into its constituent parts, and extract the relevant code to be included inside the main.py file. The equivalent code would look like this in main.py:
from endpoint import EndpointApi
EndpointApi = namespace.route("/path")(EndpointApi)
This is exactly the same code that's run in the decorator, so pylint considers my EndpointApi to be unused even though the decorator is using it to append a "/path" route to the namespace. Removing the decorator and adding the equivalent code to main.py decreases maintainability because now the relevant parts of EndpointApi are in two different files instead of all being defined one.
Edit
No, from endpoint import * makes it worse:
main.py:3:0: W0614: Unused import(s) EndpointApi, Resource and ns from wildcard import of endpoint (unused-wildcard-import)
Minimal example
flask-restx-hello $ pylint *py; for x in *py; do echo $x; cat $x; done
************* Module main
main.py:3:0: W0611: Unused EndpointApi imported from endpoint (unused-import)
------------------------------------------------------------------
Your code has been rated at 9.29/10 (previous run: 7.86/10, +1.43)
endpoint.py
"""docstring"""
from flask_restx import Resource
from init import ns
#ns.route('/hello')
class EndpointApi(Resource):
"""docstring"""
def get(self):
"""docstring"""
return {'hello': 'world'}
init.py
"""docstring"""
from flask import Flask
from flask_restx import Api
app = Flask(__name__)
api = Api(app)
ns = api.namespace('sick', description='crazy', path='/root/haha')
main.py
"""docstring"""
from init import app
from endpoint import EndpointApi
if __name__ == '__main__':
app.run(debug=True)
$ cat requirements.txt
aniso8601==9.0.1
attrs==22.1.0
click==8.0.4
dataclasses==0.8
Flask==2.0.3
flask-restx==0.5.1
importlib-metadata==4.8.3
itsdangerous==2.0.1
Jinja2==3.0.3
jsonschema==4.0.0
MarkupSafe==2.0.1
pkg-resources==0.0.0
pyrsistent==0.18.0
pytz==2022.2.1
six==1.16.0
typing-extensions==4.1.1
Werkzeug==2.0.3
zipp==3.6.0
Pylint isn't wrong. You aren't using endpoint anywhere in main.py. The only reason you're importing endpoint is to execute the decorator. Which is fine, but there is no way for pylint to know that.
In this case, it's ok to ignore the warning.
If you don't want to disable each of the import errors on every single link like this:
# pylint: disable=no-name-in-module
you could use:
# pylint: disable=import-error
Commenting this once in the file will disable pylint import errors for the entire file.
Additionally, this post on stack overflow may help:
Is it possible to ignore one single specific line with Pylint?
For more info on W0611 pylint unused imports:
https://pylint.pycqa.org/en/latest/user_guide/messages/warning/unused-import.html

ModuleNotFoundError when using Mock

I'm fairly new to Python. I'm working on a small Python project, structured as so:
artwork_grabber/
|
|-- artwork_grabber/
| |-- __init__.py
| |-- helpers.py
|
|-- tests/
| |-- __init__.py
| |-- test_module.py
|
|-- README
Both __init__.py files are empty of any content.
The helpers.py file contains a few functions, one of which is as follows:
from os import path
from tinytag import TinyTag
def create_search_term(file_path_to_song):
"""
Takes in a file path to a song and returns a phrase that will be used to search for the song's corresponding album artwork.
:param file_path_to_song: a file path to an .mp3 or .m4a file.
:type file_path_to_song: `string`, required.
:return: an object of type string that represents the search term to be used when finding album artwork for song file passed into the function.
:rtype: `string`.
"""
if str(path.isfile(file_path_to_song)):
tag = TinyTag.get(file_path_to_song)
album = tag.get_album()
artist = tag.get_artist()
term = f"{artist} {album} Album Cover"
return term
else:
return False
I would like to write a test for create_search_term() using the Mock Object Library. In the test_module.py function, I have the following:
from unittest import TestCase
from mock import patch
import unittest
from artwork_grabber.helpers import create_search_term
class UnitTests(TestCase):
mock_song_info = {
"album": "A Deeper Understanding",
"artist": "The War On Drugs"
}
# patch where the function is USED, not where it is DEFINED
#mock.patch('artwork_grabber.helpers.create_search_term', return_value=mock_song_info)
def test_create_search_term(self, mock_song):
actual_result = create_search_term(mock_song_info)
expected_result = "A Deeper Understanding The War On Drugs Album Cover"
self.assertEqual(actual_result, expected_result,
"Expected the search terms to match.")
if __name__ == "__main__":
unittest.main()
The problem is that when I run python test_module.py from the terminal (where pwd outputs /path/to/artwork_grabber/tests), I get the following error:
Traceback (most recent call last):
File "test_module.py", line 5, in <module>
from artwork_grabber.artwork_grabber import create_search_term
ModuleNotFoundError: No module named 'artwork_grabber'
Any idea what I might be missing? I've viewed several tutorials on using Mock, but they haven't seemed to help.
As documented in The Module Search Path:
When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:
The directory containing the input script (or the current directory when no file is specified).
PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
...
So in your case, the path that python would look for your modules is at /path/to/artwork_grabber/tests which obviously doesn't contain any artwork_grabber/helpers.py since it only contains an __init__.py and test_module.py. To satisfy either of the 2 ways above as documented, do either of the following:
Go to the parent folder cd /path/to/artwork_grabber and then execute the test python tests/test_module.py. This will satisfy the 1st above which will include the current path thus including artwork_grabber/helpers.py and the subdirectories and files in it.
Define it in the variable export PYTHONPATH=${PYTHONPATH}:/path/to/artwork_grabber to satisfy the 2nd above.
Note that as pointed out by #MatiasCicero it wouldn't make sense to mock function-x to return y and assert that function-x returned y, the point of testing is to test source code, not to test the test if it correctly did the mock. Ideally, you should run the actual create_search_term and pass it a test file file_path_to_song.

how to import a python class's moudle?

I have this main.py, in it:
import uuid
class tools(object):
def generate_uuid(self):
return self.uuid.uuid4()
in my calling program callmain.py, I have
import main
result = main.tool.generate_uuid()
print ("result")
if I run my callmain.py: I get
"TypeError: generate_uuid() missing 1 required positional argument:
'self'
if I add self to the line
result = main.tool.generate_uuid(self): I get
NameError: name 'self' is not defined
How to fix this? thank for help.
because you should make a object from your class first. then call your sub function like this:
import main
result = tools()
result.generate_uuid()
print(result) # "result" is a string! you should just call result without any "".
If you want to use the module of a class you have to create an instance of that class first and call if from that instance, that way the self argument is passed a valid reference to an instance of that class. For example:
import main
tools_instance = main.tools()
result = tools_instance.generate_uuid()
The style of importing shown in your question looks like a package. In packages a folder of python files __init__.py can be arranged in a particular way, documented here Python Packages. An example from the docs
parent/
__init__.py
one/
__init__.py
two/
__init__.py
three/
__init__.py
So an package of the format
main/
__init__.py
tools/
__init__.py # Add function 'generate_uuid' in this file
Could be utilized as follow:
import main
result = main.tools.generate_uuid()
print(result)

Import classes from child directory python

I've been trying to import some python classes which are defined in a child directory. The directory structure is as follows:
workspace/
__init__.py
main.py
checker/
__init__.py
baseChecker.py
gChecker.py
The baseChecker.py looks similar to:
import urllib
class BaseChecker(object):
# SOME METHODS HERE
The gChecker.py file:
import baseChecker # should import baseChecker.py
class GChecker(BaseChecker): # gives a TypeError: Error when calling the metaclass bases
# SOME METHODS WHICH USE URLLIB
And finally the main.py file:
import ?????
gChecker = GChecker()
gChecker.someStuff() # which uses urllib
My intention is to be able to run main.py file and call instantiate the classes under the checker/ directory. But I would like to avoid importing urllib from each file (if it is possible).
Note that both the __init__.py are empty files.
I have already tried calling from checker.gChecker import GChecker in main.py but a ImportError: No module named checker.gChecker shows.
In the posted code, in gChecker.py, you need to do
from baseChecker import BaseChecker
instead of import baseChecker
Otherwise you get
NameError: name 'BaseChecker' is not defined
Also with the mentioned folders structure you don't need checker module to be in the PYTHONPATH in order to be visible by main.py
Then in main.y you can do:
from checker import gChecker.GChecker

importing error in python while importing two classes

I am working with a python script, and i face importing problem when i try to import a class from another python script. Here is how my python project folder looks:
Mysql_Main/
checks.py
Analyzer/
config.py
ip.py
op.py
__init__.py
Now i want to import two classes named: Config() and Sqlite() from config.py into the checks.py script.How do i do it?
This is what i tried, but its resulting in an error!
inside checks.py:
from Analyzer import config
config = config.Config()
sqlite = config.Sqlite()
The problem is that Config class is imported properly, but Sqlite class is not getting imported.It is showing error - Config instance has no attribute 'Sqlite'
When you do:
config = config.Config()
You write over the variable config and it no longer points to the module config. It stores the new Config instance.
Try:
from Analyzer import config
config_instance = config.Config()
sqlite_instance = config.Sqlite()

Categories

Resources