I have a python package and I would like to use its classes and methods in Matlab. I know that this can be done directly since Matlab 2014b. I mean all you have to do is add py. in the beginning of your statements. So far so good, however, I couldn't figure out how to deal with context managers through MATLAB, which are invoked using the with statement. For instance, assume that we have the following class in a module called app.py,
class App(object):
def __init__(self, input):
self._input = input
self._is_open = False
def __enter__(self):
self._is_open = True
# many other stuff going after this but not relevant to this problem
In Matlab, I can call this as
app = py.app.App(input);
py.getattr(app, '_is_open')
ans =
logical
0
and I see an instance of App in my workspace. However, as expected only __init__ is invoked this way but not __enter__.
So, is there a way to invoke __enter__ from Matlab, as if we are calling it like with App(input) as app: in Python?
Note: I am using Python 3.5.1 and Matlab 2017b
I don't believe there is any way to invoke the __enter__ method of a Python class from MATLAB, but the __exit__ method might be implicitly called (I'll address this further below).
It's important to first consider the purpose of context managers (via the __enter__ and __exit__ methods), which is to provide a way to allocate and release resources in a scope-limited fashion, whether or not that scope is exited normally or via an error. MATLAB has a more limited means of "scoping": each function has its own workspace, and control structures like loops, conditional statements, etc. within that function all share that workspace (unlike many languages in which these control structures have their own sub-scopes).
When a workspace is exited in MATLAB, the variables it contains are cleared, but any resources that were allocated may still need to be released. This can be achieved with onCleanup objects. When they are cleared from memory, they invoke a given function for managing existing resources. An example would be opening and reading from a file:
function openFileSafely(fileName)
fid = fopen(fileName, 'r');
c = onCleanup(#() fclose(fid));
s = fread(fid);
...
end
Here, a file is opened and subsequently read from. An onCleanup object c is created that will close the file when c is cleared from memory upon exit from the function. If the file were simply closed with fclose(fid) at the end of the function, then an error exit from the function (such as during the course of reading data) would cause the file to still remain opened. Using an onCleanup object ensures that the file will be closed regardless of how the function exits. Here's an example of how this could be handled in Python:
with open('some_file', 'w') as opened_file:
opened_file.write('Hola!')
Since MATLAB has a different means of "context management" than Python, this may explain why it's not possible to access the __enter__ method. I tried with a class I knew had one: the io.FileIO class. I first looked for help:
>> py.help('io.FileIO.__enter__')
Help on method_descriptor in io.FileIO:
io.FileIO.__enter__ = __enter__(...)
It finds some help text. It's not particularly helpful, but it's there. However, when I create an object and look at its methods list, neither __enter__ nor __exit__ (nor a clear equivalent) is there:
>> fio = py.io.FileIO('test.txt');
>> methods(fio)
Methods for class py._io.FileIO:
FileIO eq ge le read readinto seek truncate writelines
char fileno gt lt readable readline seekable writable
close flush isatty ne readall readlines tell write
Methods of py._io.FileIO inherited from handle.
Methods for class handle:
addlistener eq findprop gt le ne
delete findobj ge isvalid lt notify
I did notice something interesting when I cleared the fio object, though. While the fio object still existed (with the file open), I couldn't delete or move the file, as expected. However, after issuing the command clear fio, without first closing the file, I was able to interact with the file normally. This implies that the file was automatically closed. This makes me wonder if the __exit__ method might be getting implicitly invoked, but I have yet to determine it for certain.
Related
I am using PyObj-C and am making some methods in a python file to read and write files using NSDocument, which uses the abstract NSFileCoordinater class. Accessing files this way instead of just using python's open let's these classes handle things for me such as preventing files from being edited from more than one program at a time or giving enough time for read/write operations to finish before it could get deadlocked.
These features are very important, and the app I ma building I want to be up to standard as much as I can here.
I have this code that instantiates a NSDocument object that contains the content of whatever file path you put into it, as a function:
#classmethod
def write(cls, file: str):
path = NSURL.fileURLWithPath_(file)
ext = file.split('.')[-1]
doc = NSDocument.alloc().initWithContentsOfURL_ofType_error_(path, ext, None)
When I call this function with a valid file path I get this error:
File "/Users/user123/PycharmProjects/shoutout/src/sutils/cfiles.py", line 27, in write
doc = NSDocument.alloc().initWithContentsOfURL_ofType_error_(path, ext, None)
objc.error: NSInternalInconsistencyException - readFromData:ofType:error: is a subclass responsibility but has not been overridden.
I have tried to find forums both objective-c, swift, or pyobj-c based as it were asking any keywords such as objective-c is a subclass responsibility but has not been overridden on google, and checked stackoverflow, and github for existing posts on this error but I could find none.
As I understand it Objective-C being polymorphic, has my method initWithContentsOfURL:ofType:error: call readFromData:ofType:error, among other ones at the same time. I don't understand exactly however what it means when it's saying that "is a subclass responsibility but has not been overridden." I am not sure also about what it means to override a class or a one being a responsibility so that doesn't help on my part.
A NSInternalInconsistencyException means a "when an internal assertion fails and implies an unexpected condition within the called code." Not sure what a internal "assertion" is either or what this could mean.
Any idea of what I could do to fix this?
NSDocument is an abstract class that requires you to subclass and implement a number of methods to make it usable. This is document in Apple's documentation for the class.
the older Document-Baed App Programming Guide for Mac gives more information on this.
This question already has an answer here:
What is the purpose of a context manager in python [duplicate]
(1 answer)
Closed 1 year ago.
I am studying Python and I found there are two types of file opening operations.
The first one is,
myreadfile = open("bear.txt", "r")
content = myreadfile.read()
second method is
with open("bear.txt") as file:
content = file.read()
I want to know is there any difference between these two methods and which one is most suitable to use.
They are context managers.
Explanation:
The with method is a context manager, if you use it for reading or writing I/O files, it will automatically close the file, don't need to add a line of file.close(), as mentioned in the docs:
Context managers allow you to allocate and release resources precisely when you want to. The most widely used example of context managers is the with statement. Suppose you have two related operations which you’d like to execute as a pair, with a block of code in between. Context managers allow you to do specifically that.
Examples:
There are examples in the docs, a regular with statement:
with open('some_file', 'w') as opened_file:
opened_file.write('Hola!')
Is equivalent to:
file = open('some_file', 'w')
try:
file.write('Hola!')
finally:
file.close()
It says that:
While comparing it to the first example we can see that a lot of boilerplate code is eliminated just by using with. The main advantage of using a with statement is that it makes sure our file is closed without paying attention to how the nested block exits.
A brief introduction of the implementation:
As mentioned in the docs, the context manager could be implemented with a class:
At the very least a context manager has an __enter__ and __exit__ method defined.
As shown there, an example context manager implementation in a class would be something like:
class File(object):
def __init__(self, file_name, method):
self.file_obj = open(file_name, method)
def __enter__(self):
return self.file_obj
def __exit__(self, type, value, traceback):
self.file_obj.close()
with File('demo.txt', 'w') as opened_file:
opened_file.write('Hola!')
The code can behave like a context manager due to the magic methods, __enter__ and __exit__.
The first one does not close the file automatically. The second one does.
As far as I understand, the first one, you need to close the file after you're done with the operations while in the latter the file is automatically closed after execution of that indent block.
The second method is the recommended method. This with syntax is known as a context and will automatically close the file once the context is exited as well as if something goes wrong during the operation.
In the first case, the file is opened and read. It stays open afterwards.
In the second case, you use the file object as a so-called "context manager". Special methods get called on entering and leaving the with block: on leaving, it is closed. This is superior to the other, even superior to
myreadfile = open("bear.txt", "r")
content = myreadfile.read()
myreadfile.close()
because the close() line isn't reached when read() throws an exception.
It is more like
myreadfile = open("bear.txt", "r")
try:
content = myreadfile.read()
finally:
myreadfile.close()
but easier to use.
If I am correct, with statement doesn't introduce a local scope for the with statement.
These are examples from Learning Python:
with open(r'C:\misc\data') as myfile:
for line in myfile:
print(line)
...more code here...
and
lock = threading.Lock() # After: import threading
with lock:
# critical section of code
...access shared resources...
Is the second example equivalent to the following rewritten in a way similar to the first example?
with threading.Lock() as lock:
# critical section of code
...access shared resources...
What are their differences?
Is the first example equivalent to the following rewritten in a way similar to the second example?
myfile = open(r'C:\misc\data')
with myfile:
for line in myfile:
print(line)
...more code here...
What are their differences?
When with enters a context, it calls a hook on the context manager object, called __enter__, and the return value of that hook can optionally be assigned to a name using as <name>. Many context managers return self from their __enter__ hook. If they do, then you can indeed take your pick between creating the context manager on a separate line or capturing the object with as.
Out of your two examples, only the file object returned from open() has an __enter__ hook that returns self. For threading.Lock(), __enter__ returns the same value as Lock.acquire(), so a boolean, not the lock object itself.
You'll need to look for explicit documentation that confirms this; this is not always that clear however. For Lock objects, the relevant section of the documentation states:
All of the objects provided by this module that have acquire() and release() methods can be used as context managers for a with statement. The acquire() method will be called when the block is entered, and release() will be called when the block is exited.
and for file objects, the IOBase documentation is rather on the vague side and you have to infer from the example that the file object is returned.
The main thing to take away is that returning self is not mandatory, nor is it always desired. Context managers are entirely free to return something else. For example, many database connection objects are context managers that let you manage the transaction (roll back or commit automatically, depending on whether or not there was an exception), where entering returns a new cursor object bound to the connection.
To be explicit:
for your open() example, the two examples are for all intents and purposes exactly the same. Both call open(), and if that does not raise an exception, you end up with a reference to that file object named myfile. In both cases the file object will be closed after the with statement is done. The name continues to exist after the with statement is done.
There is a difference, but it is mostly technical. For with open(...) as myfile:, the file object is created, has it's __enter__ method called and then myfile is bound. For the myfile = open(...) case, myfile is bound first, __enter__ called later.
For your with threading.Lock() as lock: example, using as lock will set lock to a True (locking always either succeeds or blocks indefinitely this way). This differs from the lock = threading.Lock() case, where lock is bound to the lock object.
Here's a good explanation. I'll paraphrase the key part:
The with statement could be thought of like this code:
set things up
try:
do something
finally:
tear things down
Here, “set things up” could be opening a file, or acquiring some sort of external resource, and “tear things down” would then be closing the file, or releasing or removing the resource. The try-finally construct guarantees that the “tear things down” part is always executed, even if the code that does the work doesn’t finish.
I'm currently writing code in Python 2.7, which involves creating an object, in which I have two class methods and other regular methods. I need to use this specific combination of methods because of the larger context of the code I am writing- it's not relevant to this question, so I won't go into depth.
Within my __init__ function, I am creating a Pool (a multiprocessing object). In the creation of that, I call a setup function. This setup function is a #classmethod. I define a few variables in this setup function by using the cls.variablename syntax. As I mentioned, I call this setup function within my init function (inside the Pool creation), so these variables should be getting created, based on what I understand.
Later in my code, I call a few other functions, which eventually leads to me calling another #classmethod within the same object I was talking about earlier (same object as the first #classmethod). Within this #classmethod, I try to access the cls.variables I created in the first #classmethod. However, Python is telling me that my object doesn't have an attribute "cls.variable" (using general names here, obviously my actual names are specific to my code).
ANYWAYS...I realize that's probably pretty confusing. Here's some (very) generalized code example to illustrate the same idea:
class General(object):
def __init__(self, A):
# this is correct syntax based on the resources I'm using,
# so the format of argument isn't the issue, in case anyone
# initially thinks that's the issue
self.pool = Pool(processes = 4, initializer=self._setup, initargs= (A, )
#classmethod
def _setup(cls, A):
cls.A = A
#leaving out other functions here that are NOT class methods, just regular methods
#classmethod
def get_results(cls):
print cls.A
The error I'm getting when I get to the equivalent of the print cls.A line is this:
AttributeError: type object 'General' has no attribute 'A'
edit to show usage of this code:
The way I'm calling this in my code is as such:
G = General(5)
G.get_results()
So, I'm creating an instance of the object (in which I create the Pool, which calls the setup function), and then calling get_results.
What am I doing wrong?
The reason General.A does not get defined in the main process is that multiprocessing.Pool only runs General._setup in the subprocesses. This means that it will not be called in the main process (where you call Pool).
You end up with 4 processes where in each of them there is General.A is defined, but not in the main process. You don't actually initialize a Pool like that (see this answer to the question How to use initializer to set up my multiprocess pool?)
You want an Object Pool which is not natively impemented in Python. There's a Python Implementation of the Object Pool Design Pattern question here on StackOverflow, but you can find a bunch by just searching online.
In Python, the with statement is used to make sure that clean-up code always gets called, regardless of exceptions being thrown or function calls returning. For example:
with open("temp.txt", "w") as f:
f.write("hi")
raise ValueError("spitespite")
Here, the file is closed, even though an exception was raised. A better explanation is here.
Is there an equivalent for this construct in Ruby? Or can you code one up, since Ruby has continuations?
Ruby has syntactically lightweight support for literal anonymous procedures (called blocks in Ruby). Therefore, it doesn't need a new language feature for this.
So, what you normally do, is to write a method which takes a block of code, allocates the resource, executes the block of code in the context of that resource and then closes the resource.
Something like this:
def with(klass, *args)
yield r = klass.open(*args)
ensure
r.close
end
You could use it like this:
with File, 'temp.txt', 'w' do |f|
f.write 'hi'
raise 'spitespite'
end
However, this is a very procedural way to do this. Ruby is an object-oriented language, which means that the responsibility of properly executing a block of code in the context of a File should belong to the File class:
File.open 'temp.txt', 'w' do |f|
f.write 'hi'
raise 'spitespite'
end
This could be implemented something like this:
def File.open(*args)
f = new(*args)
return f unless block_given?
yield f
ensure
f.close if block_given?
end
This is a general pattern that is implemented by lots of classes in the Ruby core library, standard libraries and third-party libraries.
A more close correspondence to the generic Python context manager protocol would be:
def with(ctx)
yield ctx.setup
ensure
ctx.teardown
end
class File
def setup; self end
alias_method :teardown, :close
end
with File.open('temp.txt', 'w') do |f|
f.write 'hi'
raise 'spitespite'
end
Note that this is virtually indistinguishable from the Python example, but it didn't require the addition of new syntax to the language.
The equivalent in Ruby would be to pass a block to the File.open method.
File.open(...) do |file|
#do stuff with file
end #file is closed
This is the idiom that Ruby uses and one that you should get comfortable with.
You could use Block Arguments to do this in Ruby:
class Object
def with(obj)
obj.__enter__
yield
obj.__exit__
end
end
Now, you could add __enter__ and __exit__ methods to another class and use it like this:
with GetSomeObject("somefile.text") do |foo|
do_something_with(foo)
end
I'll just add some more explanations for others; credit should go to them.
Indeed, in Ruby, clean-up code is as others said, in ensure clause; but wrapping things in blocks is ubiquitous in Ruby, and this is how it is done most efficiently and most in spirit of Ruby. When translating, don't translate directly word-for-word, you will get some very strange sentences. Similarly, don't expect everything from Python to have one-to-one correspondence to Ruby.
From the link you posted:
class controlled_execution:
def __enter__(self):
set things up
return thing
def __exit__(self, type, value, traceback):
tear things down
with controlled_execution() as thing:
some code
Ruby way, something like this (man, I'm probably doing this all wrong :D ):
def controlled_executor
begin
do_setup
yield
ensure
do_cleanup
end
end
controlled_executor do ...
some_code
end
Obviously, you can add arguments to both controlled executor (to be called in a usual fashion), and to yield (in which case you need to add arguments to the block as well). Thus, to implement what you quoted above,
class File
def my_open(file, mode="r")
handle = open(file, mode)
begin
yield handle
ensure
handle.close
end
end
end
File.my_open("temp.txt", "w") do |f|
f.write("hi")
raise Exception.new("spitesprite")
end
It's possible to write to a file atomically in Ruby, like so:
File.write("temp.txt", "hi")
raise ValueError("spitespite")
Writing code like this means that it is impossible to accidentally leave a file open.
You could always use a try..catch..finally block, where the finally section contains code to clean up.
Edit: sorry, misspoke: you'd want begin..rescue..ensure.
I believe you are looking for ensure.