So I have a python 2.7 project with three modules. Two contain classes and one is a script. An example files structure is shown below
project/
__main__.py
__init__.py
- serial_connect/
ser_conn.py
__init__.py
- ui/
parse_file.py
__init__.py
ser_conn.py contains a class which handles all interaction with some RS-232 hardware device. It contains methods such as open(), close(), send_go() just basically everything required for this connection.
parse_file.py contains a class which has methods relating to parsing a file and getting text commands which are associated with serial commands.
e.g. if the text file contains the command "send_go" parse_file.py will parse this command and call Ser_Conn.send_go()
__main.py__ contains the main() function. in main() an instance of the Ser_Conn class is created and then passed to the Parse_File class as there exists only one instance of a serial connection in the program and it is required by both main() and the Parse_File class.
My question is as follows
In methods of Parse_File I call methods of the Ser_Conn instance, such as Ser_Conn.send_go() but parse_file.py does not complain about there being no
from serial_connect.ser_conn import Ser_Conn
There only exists a ser_conn import in __main__.py. why does this work?
In python, an instance carries all the "stuff" that it needs to do it's work along with it. At a high level, (and somewhat simplified) when you write:
qux.whatever
python looks at the object qux and then finds its whatever attribute (if it has one). Note that python doesn't care what type qux is, only that it has a whatever attribute. I could create multiple objects that satisfy this interface:
class Foo(object):
whatever = 'Hey Dawg!'
class Bar(object):
whatever = 'I satisfy the interface too!'
Also note that these objects could be defined anywhere. As long as you manage to get a reference to the object (e.g. it was passed into a function) you can use the object with all of it's attributes and methods.
maybe a more concrete example would help -- Say you have 4 modules, foo, bar, call_method and driver. foo, bar and call_method don't know anything about each other, but driver imports the other 3. Then you can do something like this:
# foo.py
class Foo(object):
def method(self):
return 'foo!'
# bar.py
class Bar(object):
def method(self):
return 'bar!'
# call_method.py
def go(object_with_method):
print(object_with_method.method())
# driver.py
import call_method
import foo
import bar
call_method.go(Foo()) # call the method on a foo instance
call_method.go(Bar()) # call the method on a bar instance
You're passing the connection instance to parse_file which means Python already knows the class and other details of that object. So that's why you don't need to import the class again in the parse_file code.
You only need to import something if you wish to use that something in that file.
When you created the object Python used the class to construct the object and that's sufficient. You can of course add that import line in parse_file but only if you actually need to use the class in that file (otherwise the import line it's very useful).
Related
I am trying to implement python classes and objects in my application code. Currently, I have a file that includes all the frequently used functions. I import them in another file.
funcs.py
class name1():
def func1(x):
return x
def func2(y):
return y
....
file1.py
from funcs import func1
from funcs import func2
I'd like to organize the code in class, method and attributes and then invoke them in different files.
How do I call a method within a class from another file? What changes do I need to make in funcs.py file?
If you want to call a method within a class, first you have to instantiate an object of that class, and then call the method in reference to the object. Below is not an ideal implementation but it's just for example.
example.py
class MyClass:
def my_method(self):
print('something')
object1 = MyClass()
object1.my_method()
Then when you want to call the method in another file you have to first import them.
another.py
from .example import MyClass
object2 = MyClass()
object2.my_method()
If you just want to call the method without having to create an object first you can use #staticmethod.
class MyClass:
#staticmethod
def my_method(self):
print('something')
MyClass.my_method()
Yet as I said this is not the ideal implementation. As #juanpa.arrivillaga said ideally you cannot just throw in any method and bundle them into a single class. The content of a class is all related to the object you want to define as a class.
I discovered an annoying behaviour of reload() (both Python 2 builtin and from importlib) that I am trying to walkarround.
I am analysing data in interactive Python interpreter. My code is organised in modules (both Python 2 and 3 compatible) which I often change.
Restarting the interpreter is not feasible due to long time of loading data, so I prefer to recursively reload modules instead.
The problem is that reload() updates the code but preserves the module global scope (it applies to Python 3 importlib.reload() as well). It seems to be harmful for methods using super() (it took me a while to realise what is going on).
The minimal failing example for a module bar.py:
class Bar(object):
def __init__(self):
super(Bar, self).__init__()
is:
>>> import bar
>>> class Foo(bar.Bar):
... pass
...
>>> reload(bar)
<module 'bar' from '[censored]/bar.py'>
>>> Foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "[censored]/bar.py", line 3, in __init__
super(Bar, self).__init__()
TypeError: super(type, obj): obj must be an instance or subtype of type
I may:
use super() without arguments in Python 3 manner (which is not
compatible with Python 2),
abandon it and call Bar.__init__(self) instead (which is harder to
maintain and discouraged),
monkey-patch the class adding a class attribute containing a circular
reference to the class itself.
None of the ideas I like. Is there any other way of dealing with the issue?
You can't, because it's basically impossible to do anything elegantly and safely where module reloading is concerned. That kind of thing is super tricky to get right.
The specific way that problem is manifesting here is that Foo's superclass is still the old Bar, but the old Bar refers to itself by name, and the Bar name now refers to the new Bar in the namespace where it's looking. The old Bar is trying to pass the new Bar to super.
I can give you some options. All of them are inelegant and unsafe and probably going to give you weird surprises eventually, but all that is also true about module reloading.
Probably the most easily-understood option is to rerun the Foo class definition to get a new Foo class descending from the new Bar:
reload(bar)
class Foo(bar.Bar):
pass
New instances of Foo will then use the new Bar, and will not experience problems with super. Old instances will use the old Foo and the old Bar. They're not going to have problems with __init__, because that already ran, but they're likely to have other problems.
Another option you can take is to update Foo's superclass so Foo now descends from the new Bar:
reload(bar)
Foo.__bases__ = (bar.Bar,)
New and old instances of Foo will then use the new Bar. Depending on what you changed in Bar, the new class may not be compatible with the old instances, especially if you changed what data Bar keeps on its instances or how Bar.__init__ performs initialization.
Sound like what you really want it to do re-import the module. The following allow that (in both Python 2 and 3).
import bar
import sys
class Foo(bar.Bar):
pass
# Manually reload module.
_saved_reference = sys.modules['bar'] # Needed for Python 2.
del sys.modules['bar']
import bar # Re-import instead of reload().
Foo() # No error.
In the constructor of my class under test a socket object is instantiated and assigned to a class member. I mocked the socket class and set a mocked socket object as return value to the socket constructor call. I then want to assert that connect() and sendall() is called on that object. I always get the assert error that the functions are not called when I assert on the original mocked class object or the one that I set to return on constructor call.
I know I can’t mock the class that is under test (and its members) because that would defeat the purpose here.
Pseudo code:
import socket
Class socketHandler():
def __init__(...):
self.mySocket = socket(...)
...
self.mySocket.connect(...)
def write(message):
self.mySocket.sendall(message)
Test:
from unittest import mock
from unittest.mock import MagicMock #not sure if i need this
import pytest
import socketHandler
#mock.patch(socketHandler.socket)
def test_socket_handler(mockSocket):
...
new_sock = mock_socket()
mock_socket.return_value = new_sock
mySocketHandler = SocketHandler(...)
mock_socket.socket.assert_called_with(...)
new_sock.connect.assert_called_with(...) #fails (never called)
mock_socket.connect.assert_called_with(...) #fails (never called)
#likewise for the sendall() method call when mysocketHandler.write(..)
#is called
The purpose of this test is:
ensure the constructor of socket library is called with the right arguments.
ensure that connect() is called with right arguments.
ensure that sendall() is called with exactly what I want it to be called, when I pass message into mySocketHandler.write() method.
The complete answer derived from hints given by #ryanh119 and this post link
I will fix the example given above by ryanh119 and refrain from editing original question which i messed up, so for completeness:
from unittest import mock
import pytest
import socketHandler
#mock.patch("app_directory.socketHandler.socket")
def test_socket_handler(mockSocketClass):
# mockSocketClass is already a mock, so we can call production right away.
mySocketHandler = SocketHandler(...)
# Constructor of mockSocketClass was called, since the class was imported
#like: import socket we need to:
mockSocketClass.socket.assert_called_with(...)
# Production called connect on the class instance variable
# which is a mock so we can check it directly.
# so lets just access the instance variable sock
mySocketHandler.mySocket.connect.assert_called_with(...)
# The same goes for the sendall call:
mySocketHandler.mySocket.sendall.assert_called_with(expectedMessage)
I also did some research and there would have been two more solutions that I want to mention. They are not as pythonicaly correct like the above ones but here it is:
Make use of dependency injection by changing the __init__ of socketHandler to take in a socket object and only instantiate it if not supplied in the args. That way i could have passed in a mock or MagicMock object and used that to do the asserts on.
Make use of a extremely powerful mocking/patching tool called MonkeyPatch which actually can patch/mock instance variables of classes. This approach would have been like trying to kill a fly with a rocket launcher.
You're on the right track, but there are a couple things that need to change for this test to work.
Part of your problem right off the bat is that the mock that patch passes into your test method is called mockSocket, but your test code is referring to something called mock_socket.
Also, patch's first argument, the thing you want to patch, should be a string representation of the path to the module where you want to patch something. If your file structure looks like this:
|-- root_directory
| |
| |-- app_directory
| | |-- socketHandler.py
| | `-- somethingElse.py
| |
| `-- test_directory
| |-- testSocketHandler.py
| `-- testSomethingElse.py
and you run your tests from the root directory, you'd want to call patch like this: #mock.patch("app_directory.socketHandler.socket")
Constructor is called - The most important thing to realize is that mockSocket is a Mock object representing the socket class. So to test that the constructor was called, you need to check mockSocket.assert_called_with(...). That will pass if your production calls socket(...).
You may also want to assert that mySocketHandler.socket is the same object as mockSocket.return_value, to test that mySocketHandler not only calls the constructor, but assigns it to the right attribute.
and 3. connect and sendall are called properly - You should never call your mock in a test, because it can lead to falsely-passing assertions. In other words, you want your production code to be the only thing calling mocks. This means you shouldn't use the line new_sock = mock_socket(), because then your previous assertion about the constructor will pass no matter what your production code does, and I think it's causing your other assertions to fail.
mockSocketis already an instance of Mock, so it's return value will automatically be another, different Mock instance. Therefore, you don't need the first 2 lines of your test code above, and you only need one of the assertions on connect. The same ideas apply to sendall.
That's a lot to take in, here's what your test would look like if I wrote it:
from unittest import mock, TestCase
import pytest
import socketHandler
class TestSocketHandler(TestCase):
#mock.patch("app_directory.socketHandler.socket")
def test_socket_handler(mockSocketClass): # renamed this variable to clarify that it's a mock of a class.
# mockSocketClass is already a mock, so we can call production right away.
mySocketHandler = SocketHandler(...)
# Constructor of mockSocketClass was called
mockSocketClass.assert_called_with(...)
# Instance of mockSocketClass was assigned to correct attribute on SocketHandler
self.assertIs(mockSocketClass.return_value, mySocketHandler.socket)
# Production called connect on the return_value of the mock module, i.e. the instance of socket.
mockSocketClass.return_value.connect.assert_called_with(...)
# If SocketHandler's constructor calls sendall:
mockSocketClass.return_value.sendall.assert_called_with(expectedMessage)
Bonus Round! MagicMocks behave like Mocks, except that they implement some default values for some magic methods. I don't use them unless I absolutely need them. Here's an example:
from mock import Mock, MagicMock
mock = Mock()
magic_mock = MagicMock()
int(mock)
>>>Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: int() argument must be a string or a number, not 'Mock'
len(mock)
>>>Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: object of type 'Mock' has no len()
int(magic_mock)
>>> 1
len(magic_mock)
>>> 0
I must be tired, because surely there is an easy way to do this.
But I've read over the pytest docs and can't figure out this simple use case.
I have a little package I want to test:
class MyClass:
def __init__(self):
pass
def my_method(self, arg):
pass
def the_main_method():
m = MyClass()
m.my_method(123)
I would like to ensure that (1) an instance of MyClass is created, and that (2) my_method is called, with the proper arguments.
So here's my test:
from unittest.mock import patch
#patch('mypkg.MyClass', autospec=True)
def test_all(mocked_class):
# Call the real production code, with the class mocked.
import mypkg
mypkg.the_main_method()
# Ensure an instance of MyClass was created.
mocked_class.assert_called_once_with()
# But how do I ensure that "my_method" was called?
# I want something like mocked_class.get_returned_values() ...
I understand that each time the production code calls MyClass() the unittest framework whips up a new mocked instance.
But how do I get my hands on those instances?
I want to write something like:
the_instance.assert_called_once_with(123)
But where do I get the_instance from?
Well, to my surprise, there is only one mock instance created, no matter how many times you call the constructor (:
What I can write is:
mocked_class.return_value.my_method.assert_called_once_with(123)
The return_value does not represent one return value, though — it accumulates information for all created instances.
It's a rather abstruse approach, in my mind. I assume it was copied from some crazy Java mocking library (:
If you want to capture individual returned objects, you can use .side_effect to return whatever you want, and record it in your own list, etc.
I'm developing a PyQT4 application, and it's getting pretty hard for me to navigate through all of the code at once. I know of the import foo statement, but I can't figure out how to make it import a chunk of code directly into my script, like the BASH source foo statement.
I'm trying to do this:
# File 'functions.py'
class foo(asd.fgh):
def __init__(self):
print 'foo'
Here is the second file.
# File 'main.py'
import functions
class foo(asd.fgh):
def qwerty(self):
print 'qwerty'
I want to include code or merge class decelerations from two separate files. In PHP, there is import_once('foo.php'), and as I mentioned previously, BASH has source 'foo.sh', but can I accomplish this with Python?
Thanks!
For some reason, my first thought was multiple inheritance. But why not try normal inheritance?
class foo(functions.foo):
# All of the methods that you want to add go here.
Is there some reason that this wont work?
Since you just want to merge class definitions, why don't you do:
# main.py
import functions
# All of the old stuff that was in main.foo is now in this class
class fooBase(asd.fgh):
def qwerty(self):
print 'qwerty'
# Now create a class that has methods and attributes of both classes
class foo(FooBase, functions.foo): # Methods from FooBase take precedence
pass
or
class foo(functions.foo, FooBase): # Methods from functions.foo take precedence
pass
This takes advantage of pythons capability for multiple inheritance to create a new class with methods from both sources.
You want execfile(). Although you really don't, since redefining a class, uh... redefines it.
monkey patching in python doesn't work in nearly the same way. This is normally considered poor form, but if you want to do it anyways, you can do this:
# File 'functions.py'
class foo(asd.fgh):
def __init__(self):
print 'foo'
the imported module remains unchanged. In the importing module, we do things quite differently.
# File 'main.py'
import functions
def qwerty(self):
print 'qwerty'
functions.foo.qwerty = qwerty
Note that there is no additional class definition, just a bare function. we then add the function as an attribute of the class.