I'm trying to create a process that dynamically watches jupyter notebooks, compiles them on modification and imports them into my current file, however I can't seem to execute the updated code. It only executes the first version that was loaded.
There's a file called producer.py that calls this function repeatedly:
import fs.fs_util as fs_util
while(True):
fs_util.update_feature_list()
In fs_util.py I do the following:
from fs.feature import Feature
import inspect
from importlib import reload
import os
def is_subclass_of_feature(o):
return inspect.isclass(o) and issubclass(o, Feature) and o is not Feature
def get_instances_of_features(name):
module = __import__(COMPILED_MODULE, fromlist=[name])
module = reload(module)
feature_members = getattr(module, name)
all_features = inspect.getmembers(feature_members, predicate=is_subclass_of_feature)
return [f[1]() for f in all_features]
This function is called by:
def update_feature_list(name):
os.system("jupyter nbconvert --to script {}{} --output {}{}"
.format(PATH + "/" + s3.OUTPUT_PATH, name + JUPYTER_EXTENSION, PATH + "/" + COMPILED_PATH, name))
features = get_instances_of_features(name)
for f in features:
try:
feature = f.create_feature()
except Exception as e:
print(e)
There is other irrelevant code that checks for whether a file has been modified etc.
I can tell the file is being reloaded correctly because when I use inspect.getsource(f.create_feature) on the class it displays the updated source code, however during execution it returns older values. I've verified this by changing print statements as well as comparing the return values.
Also for some more context the file I'm trying to import:
from fs.feature import Feature
class SubFeature(Feature):
def __init__(self):
Feature.__init__(self)
def create_feature(self):
return "hello"
I was wondering what I was doing incorrectly?
So I found out what I was doing wrong.
When called reload I was reloading the module I had newly imported, which was fairly idiotic I suppose. The correct solution (in my case) was to reload the module from sys.modules, so it would be something like reload(sys.modules[COMPILED_MODULE + "." + name])
The title might be misleading or inaccurate, so please correct me if I'm wrong.
I have a package structured like this:
common
__init__.py
foo.py
And here are the codes:
common/__init__.py
name = 'name_common'
def print_current_file_name():
print('current file name: ' + __file__)
def print_name():
print('name is: ' + eval('name'))
foo.py
from common import print_current_file_name, print_name
name = 'name_foo'
if __name__ == '__main__':
print_current_file_name()
print_name()
If I do this:
>>> python foo.py
I'll get this:
current file name: /tmp/common/__init__.py
name is: name_common
But I expect the results to be:
current file name: /tmp/common/foo.py
name is: name_foo
What did I miss? How can I make this right?
I don't even know which keywords should I google...
The use of eval is weird, but these codes are just for demonstration purpose only.
That's not at all how variables work in Python, or in any language I'm aware of.
The global scope of a function is always where it is defined, not where it is executed. There is no way for print_name to access the value of name in foo.py.
Rather, you should pass it in as a parameter. Or, depending on what you actually want to do, you might want to create a class that defines the value of name at class level.
Actually, this is possible.
I Found a similar question:
How to use inspect to get the caller's info from callee in Python?
And I think this is what the built-in library inspect comes for.
common/__init__.py
from os.path import abspath
import inspect
name = 'name_common'
def print_current_file_name():
print('current file name: ' + abspath(inspect.getfile(inspect.currentframe().f_back)))
# or
print('current file name: ' + abspath(inspect.currentframe().f_back.f_globals['__file__']))
def print_name():
print('name is: ' + inspect.currentframe().f_back.f_globals['name'])
Finally,
$ python foo.py
current file name: /tmp/common/foo.py
name is: name_foo
I have a server.py which contains a function and other files like requestor1.py requestor2.py .... requestorN.py
Server.py contains a function :
def callmeforhelp()
return "I am here to help you out!"
and requestor1.py file calls the function callmeforhelp() and it has the imports needed to call the function from server.py
Is there a way my server.py knows which file is calling it?
Something similar like below :
When requestor1.py calls the function, then :
def callmeforhelp()
print "Now I am being called by : "+caller // caller must contain the value as requestor1.py or even full path of requestor1.py
return "I am here to help you out!"
Try it in your server file:
import inspect
def callmeforhelp():
result = inspect.getouterframes(inspect.currentframe(), 2)
print("Caller is: " + str(result[1][1]))
Here is a way to get at the caller's local attributes:
import sys
def callmeforhelp():
print("Called from", sys._getframe(1).f_locals['__file__'])
This is a feature of CPython and is not guaranteed to be present in other language implementations.
In C++, I can print debug output like this:
printf(
"FILE: %s, FUNC: %s, LINE: %d, LOG: %s\n",
__FILE__,
__FUNCTION__,
__LINE__,
logmessage
);
How can I do something similar in Python?
There is a module named inspect which provides these information.
Example usage:
import inspect
def PrintFrame():
callerframerecord = inspect.stack()[1] # 0 represents this line
# 1 represents line at caller
frame = callerframerecord[0]
info = inspect.getframeinfo(frame)
print(info.filename) # __FILE__ -> Test.py
print(info.function) # __FUNCTION__ -> Main
print(info.lineno) # __LINE__ -> 13
def Main():
PrintFrame() # for this line
Main()
However, please remember that there is an easier way to obtain the name of the currently executing file:
print(__file__)
For example
import inspect
frame = inspect.currentframe()
# __FILE__
fileName = frame.f_code.co_filename
# __LINE__
fileNo = frame.f_lineno
There's more here http://docs.python.org/library/inspect.html
Building on geowar's answer:
class __LINE__(object):
import sys
def __repr__(self):
try:
raise Exception
except:
return str(sys.exc_info()[2].tb_frame.f_back.f_lineno)
__LINE__ = __LINE__()
If you normally want to use __LINE__ in e.g. print (or any other time an implicit str() or repr() is taken), the above will allow you to omit the ()s.
(Obvious extension to add a __call__ left as an exercise to the reader.)
You can refer my answer:
https://stackoverflow.com/a/45973480/1591700
import sys
print sys._getframe().f_lineno
You can also make lambda function
I was also interested in a __LINE__ command in python.
My starting point was https://stackoverflow.com/a/6811020 and I extended it with a metaclass object. With this modification it has the same behavior like in C++.
import inspect
class Meta(type):
def __repr__(self):
# Inspiration: https://stackoverflow.com/a/6811020
callerframerecord = inspect.stack()[1] # 0 represents this line
# 1 represents line at caller
frame = callerframerecord[0]
info = inspect.getframeinfo(frame)
# print(info.filename) # __FILE__ -> Test.py
# print(info.function) # __FUNCTION__ -> Main
# print(info.lineno) # __LINE__ -> 13
return str(info.lineno)
class __LINE__(metaclass=Meta):
pass
print(__LINE__) # print for example 18
wow, 7 year old question :)
Anyway, taking Tugrul's answer, and writing it as a debug type method, it can look something like:
def debug(message):
import sys
import inspect
callerframerecord = inspect.stack()[1]
frame = callerframerecord[0]
info = inspect.getframeinfo(frame)
print(info.filename, 'func=%s' % info.function, 'line=%s:' % info.lineno, message)
def somefunc():
debug('inside some func')
debug('this')
debug('is a')
debug('test message')
somefunc()
Output:
/tmp/test2.py func=<module> line=12: this
/tmp/test2.py func=<module> line=13: is a
/tmp/test2.py func=<module> line=14: test message
/tmp/test2.py func=somefunc line=10: inside some func
import inspect
.
.
.
def __LINE__():
try:
raise Exception
except:
return sys.exc_info()[2].tb_frame.f_back.f_lineno
def __FILE__():
return inspect.currentframe().f_code.co_filename
.
.
.
print "file: '%s', line: %d" % (__FILE__(), __LINE__())
Here is a tool to answer this old yet new question!
I recommend using icecream!
Do you ever use print() or log() to debug your code? Of course, you
do. IceCream, or ic for short, makes print debugging a little sweeter.
ic() is like print(), but better:
It prints both expressions/variable names and their values.
It's 40% faster to type.
Data structures are pretty printed.
Output is syntax highlighted.
It optionally includes program context: filename, line number, and parent function.
For example, I created a module icecream_test.py, and put the following code inside it.
from icecream import ic
ic.configureOutput(includeContext=True)
def foo(i):
return i + 333
ic(foo(123))
Prints
ic| icecream_test.py:6 in <module>- foo(123): 456
To get the line number in Python without importing the whole sys module...
First import the _getframe submodule:
from sys import _getframe
Then call the _getframe function and use its' f_lineno property whenever you want to know the line number:
print(_getframe().f_lineno) # prints the line number
From the interpreter:
>>> from sys import _getframe
... _getframe().f_lineno # 2
Word of caution from the official Python Docs:
CPython implementation detail: This function should be used for internal and specialized purposes only. It is not guaranteed to exist in all implementations of Python.
In other words: Only use this code for personal testing / debugging reasons.
See the Official Python Documentation on sys._getframe for more information on the sys module, and the _getframe() function / submodule.
Based on Mohammad Shahid's answer (above).
I have the function changecheck() which is defined in a different module(called check.py). I want to pass changeId as a parameter to this function.
I am calling this function from the file test.py.
I am unable to understand the reason why this parameter is not being passed correctly.
check.py
returnVal = changecheck(changeInfoItem['changeId'])
In a differnt module test.py
def changecheck(changeId):
print changeId //nothing gets printed
This should fix your problem:
In the module test.py
def changecheck(changeId):
print changeId
In the module check.py
import test
returnVal = test.changecheck(changeInfoItem['changeId'])
You have to do this:
import check
returnVal = check.changecheck(changeInfoItem['changeId'])