Omnicompletion for Python seems to fail when there is a "from" import instead of a normal one.
For example, if I have these two files:
Test.py:
class Test:
def method(self):
pass
main.py:
from Test import Test
class Test2:
def __init__(self):
self.x = Test()
If I try to activate omnicompletion for self.x... it says "Pattern not found".
However, if I change the import statement to:
import Test
and the self.x declaration to:
self.x = Test.Test()
then I'm able to use omnicompletion as expected (it suggests "method", for example).
I'm using Vim 7.2.245 and the default plugin for Python code completion (pythoncomplete).
Should I set some variable? Or is this behavior expected?
Update:
Based on Jared's answer, I found out something by accident:
Omnicompletion doesn't work on this:
from StringIO import StringIO
class Test:
def __init__(self):
self.x = StringIO()
self.x.<C-x><C-o>
s = Test()
But works on this:
from StringIO import StringIO
class Test:
def __init__(self):
self.x = StringIO()
self.x.<C-x><C-o>
s = Test()
s.x = StringIO()
The only difference is the redeclaration of x (actually, it also works if I remove the declaration inside __init__).
I tested my example again, and I think the problem is not the "from" import, but the use of the imported class inside another class.
If I change the file main.py to:
from Test import Test
class Test2:
def __init__(self):
self.x = Test()
self.x.<C-x><C-o>
y = Test()
y.<C-x><C-o>
The first attempt to use omnicompletion fails, but the second works fine.
So yep, looks like a bug in the plugin :)
update: ooh, so I checked your example, and I get completion for
x = Test()
x.<C-x><C-o>
but not
o = object()
o.x = Test()
o.x.<C-x><C-o>
...I'm gonna do some digging
update 2: revenge of Dr. Strangelove
and...this is where it get's weird.
from StringIO import StringIO
class M:
pass
s = M()
s.x = StringIO()
s.x.<C-x><C-o>
completes. but this
from StringIO import StringIO
class M: pass
s = M()
s.x = StringIO()
s.x.<C-x><C-o>
Did you catch the difference? nothing syntactically -- just a little whitespace
And yet it breaks completion. So there's definitely a parsing bug in there somewhere (why they don't just use the ast module, I have no idea...)
[end of updates]
On first blush, I can't reproduce your problem; here's my test file:
from os import path
path.<C-x><C-o>
and I get completion. Now, I know it's not exactly your situation, but it shows that pythoncomplete knows about 'from'.
And now the more in-depth example:
from StringIO import StringIO
s = StringIO()
s.<C-x><C-o>
And...completion! Could you try that example to see if it works with builtin modules for you? If that's the case, you should probably check paths...
If it still doesnt work, and you're up for some digging around, check out line #555 of pythoncomplete.vim [at /usr/share/vim/vim72/autoload/pythoncomplete.vim on my ubuntu machine]:
elif token == 'from':
mod, token = self._parsedotname()
if not mod or token != "import":
print "from: syntax error..."
continue
names = self._parseimportlist()
for name, alias in names:
loc = "from %s import %s" % (mod,name)
if len(alias) > 0: loc += " as %s" % alias
self.scope.local(loc)
freshscope = False
as you can see, this is where it handles from statements.
Cheers
Related
When I use HTMLTestRunner for Python 3.5，it shows an error.
I have changed the HTMLTestRunner for support python 3.5.
The code :
import pymysql
import pymysql
import unittest
import time
import unittest.suite
import HTMLTestRunner
import sys
def hell(a):
print(a)
return a
testunit = unittest.TestSuite()
testunit.addTest(hell('ad'))
filename = '/Users/vivi/Downloads/aa.html'
fp = open(filename, 'wb')
runner = HTMLTestRunner.HTMLTestRunner(stream=fp, title=u'print', description=u'简单')
runner.run(testunit)
When I run it, I got this error:
Traceback (most recent call last):
File "/Applications/Python 3.5/……/B.py", line 30, in <module>
testunit.addTest(hell('ad'))
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/suite.py", line 47, in addTest
raise TypeError("{} is not callable".format(repr(test)))
TypeError: 'ad' is not callable
What should I do to make the script works?
You're adding the result of the call to hell('ad') to your tests, not the hell function itself. Since the hell function returns its argument, it returns the string ad, which is not a callable (function or the like).
Use
testunit.addTest(hell)
instead.
What about that argument then, how do you pass that?
Well, there are ways to do that, but generally, try not to let your unit test functions take an argument. Thus, hell() should siply not take an argument.
If you code your unit test correctly, you'll find that you rarely need to pass it an argument.
The line testunit.addTest(hell('ad')) doesn't do what you intend it to do. It doesn't tell the test suite to run hell('ad') later on. Rather, it calls that function immediately and passed the return value (which happens to be the string 'ad' you gave it as an argument) to addTest. That causes the exception later on, since a string is not a valid test case.
I'm not exactly sure how you should fix this. Your hell function doesn't appear to actually test anything, so there's not an obvious way to transform it into a proper test. There is a unittest.FunctionTestCase class that wraps a function up as a TestCase, but it doesn't appear to have any way of passing arguments to the function. Probably you should write a TestCase subclass class, and add various test_-prefixed methods that actually test things.
I had a right answer for my question, give the code:
# -*- coding: utf-8 -*-
import unittest
import HTMLTestRunner
def hell(a):
print(a)
return a
class HellTest(unittest.TestCase):
def setUp(self):
self.hell = hell
def tearDown(self):
pass
def testHell(self):
self.assertEqual(self.hell('aaa'), 'aaa')
if __name__ == '__main__':
testunit = unittest.TestSuite()
testunit.addTest(HellTest('testHell'))
filename = '/Users/vivi/Downloads/aa.html'
fp = open(filename, 'wb')
runner = HTMLTestRunner.HTMLTestRunner(stream=fp, title=u'不要生成error啦！', description=u'简单1112')
runner.run(testunit)
fp.close()
But, I do not know why add the class code 'class HellTest()'.The answer comes from a Chinese people whoes name is '幽谷奇峰'。Source code reference：https://segmentfault.com/q/1010000007427143?_ea=1345414
Long story short, I'm perfectly able to mock class method, when it's just that method that's replaced by mock object, but I'm unable to mock that method when I'm trying to replace the whole class by the mock object
The #mock.patch.object successfully mocks the scan method but #mock.patch fails to do so. I've followed the example at
https://docs.python.org/3/library/unittest.mock.html#unittest.mock.patch
but apparently I'm doing something wrong.
I'm mocking the lexicon module in the same namespace in both cases (it's imported by import lexicon in the sentence_parser) but the mock_lexicon is lexicon.lexicon check fails
#!python
import sys;
sys.path.append('D:\python\lexicon');
import lexicon;
import sentence_parser;
import unittest2 as unittest;
import mock;
class ParserTestCases(unittest.TestCase) :
def setUp(self) :
self.Parser = sentence_parser.Parser();
#mock.patch('lexicon.lexicon')
def test_categorizedWordsAreAssigned_v1(self, mock_lexicon) :
print "mock is lexicon:";
print mock_lexicon is lexicon.lexicon + "\n";
instance = mock_lexicon.return_value;
instance.scan.return_value = "anything";
self.Parser.categorize_words_in_sentence("sentence");
instance.scan.assert_called_once_with("sentence");
#mock.patch.object(lexicon.lexicon, 'scan')
def test_categorizedWordsAreAssigned_v2(self, mock_scan) :
mock_scan.return_value = "anything";
self.Parser.categorize_words_in_sentence("sentence");
mock_scan.assert_called_once_with("sentence");
if (__name__ == '__main__') :
unittest.main()
Output :
mock is lexicon:
False
======================================================================
FAIL: test_categorizedWordsAreAssigned_v1 (__main__.ParserTestCases)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\python\get_img\getImage_env\lib\site-packages\mock\mock.py", line 1305, in patched
return func(*args, **keywargs)
File "./test_sentence_parser.py", line 26, in test_categorizedWordsAreAssigned_v1
instance.scan.assert_called_once_with("sentence");
File "D:\python\get_img\getImage_env\lib\site-packages\mock\mock.py", line 947, in assert_called_once_with
raise AssertionError(msg)
AssertionError: Expected 'scan' to be called once. Called 0 times.
----------------------------------------------------------------------
Ran 2 tests in 0.009s
FAILED (failures=1)
EDIT :
To clarify, the Parser is defined as follows
#!python
import sys;
sys.path.append('D:\python\lexicon');
import lexicon;
class Parser(object) :
my_lexicon = lexicon.lexicon()
def __init__(self) :
self.categorized_words = ['test'];
def categorize_words_in_sentence(self, sentence) :
self.categorized_words = self.my_lexicon.scan(sentence);
if (__name__ == '__main__') :
instance = Parser();
instance.categorize_words_in_sentence("bear");
print instance.categorized_words;
What is real relevant here is how categorize_words_in_sentence Parser's method use lexicon. But first of all we should remove the noise:
print mock_lexicon is lexicon.lexicon + "\n"
Is what can lead us to the wrong direction: try to replace it by
self.assertIs(mock_lexicon, lexicon.lexicon)
and you will understand that you are printing False because mock_lexicon is not lexicon.lexicon + "\n" but just lexicon.lexicon.
Now I cannot tell you why the first test doesn't work because the answer is in categorize_words_in_sentence method or more probably in sentence_parser module where I can guess you can have something like
from lexicon import lexicon
In both case take a look to Where to Patch documentation that can enlighten you on what can be the cause and what you really need to patch in your case.
The second version works just because you are patching the object and not the reference (that should be different).
Finally the more concise and general version can be:
#mock.patch('lexicon.lexicon.scan', return_value="anything")
def test_categorizedWordsAreAssigned_v3(self, mock_scan) :
self.Parser.categorize_words_in_sentence("sentence")
mock_scan.assert_called_once_with("sentence")
One more thing: remove unittest2 at least you're not using python 2.4 and you are interested on backported unittest features.
[EDIT]
Now I can stop to guess and point to you why the first version doesn't work and will never work:
class Parser(object) :
my_lexicon = lexicon.lexicon()
Parser.my_lexicon attribute is evaluated at the load time. That means when you import sentence_parser a lexicon is created and the reference associated to Parser.my_lexicon. When you patch lexicon.lexicon you leave this reference untouched and your parser object still use the original reference created when is imported.
What you can do is to patch the reference in Parser class by
#patch("sentence_parser.Parser.my_lexicon")
You can use create_autospect if you want give to your mock the same lexicon's signature.
#patch("sentence_parser.Parser.my_lexicon", create_autospec("lexicon.lexicon", instance=True))
Is it possible to check if some class exists? I have class names in a json config file.
I know that I can simply try to create an object by class name string but that is actually a bad idea because the class constructor can do some unexpected stuff while at this point in time I just want to check if my config is valid and all mentioned classes are available.
Is there any way to do it?
EDIT: Also I do understand that u can get all the methods from some module, in my case I am not sure and don't actually care from what module comes the method. It can be from any import statement and I probably don't know where exactly from.
Using eval() leaves the door open for arbitrary code execution, for security's sake it should be avoided.
Especially if you ask for a solution for such a problem here.
Then we can assume that you do not know these risks sufficiently.
import sys
def str_to_class(str):
return reduce(getattr, str.split("."), sys.modules[__name__])
try:
cls = str_to_class(<json-fragment-here>)
except AttributeError:
cls = None
if cls:
obj = cls(...)
else:
# fight against this
This avoids using eval and is approved by several SO users.
Solution is similar to Convert string to Python class object?.
You can parse the source to get all the class names:
from ast import ClassDef, parse
import importlib
import inspect
mod = "test"
mod = importlib.import_module(mod)
p = parse(inspect.getsource(mod))
names = [kls.name for kls in p.body if isinstance(kls, ClassDef)]
Input:
class Foo(object):
pass
class Bar(object):
pass
Output:
['Foo', 'Bar']
Just compare the class names from the config to the names returned.
{set of names in config}.difference(names)
If you want to include imported names you can parse the module it was imported from but depending on how it was imported you can still find cases that won't work:
from ast import ClassDef, parse, ImportFrom
import importlib
import inspect
mod = "test"
mod = importlib.import_module(mod)
p = parse(inspect.getsource(mod))
names = []
for node in p.body:
if isinstance(node, ClassDef):
names.append(node.name)
elif isinstance(node, ImportFrom):
names.extend(imp.name for imp in node.names)
print(names)
Input:
from test2 import Foobar, Barbar, foo
class Foo(object):
pass
class Bar(object):
pass
test2:
foo = 123
class Foobar(object):
pass
class Barbar(object):
pass
Output:
['Foobar', 'Barbar', 'Foo', 'Bar']
I tried the built-in type function, which worked for me, but there is maybe a more pythonic way to test for the existence of a class:
import types
def class_exist(className):
result = False
try:
result = (eval("type("+className+")") == types.ClassType)
except NameError:
pass
return result
# this is a test class, it's only purpose is pure existence:
class X:
pass
print class_exist('X')
print class_exist('Y')
The output is
True
False
Of course, this is a basic solution which should be used only with well-known input: the eval function can be a great a back door opener. There is a more reliable (but also compact) solution by wenzul.
In C++, I can print debug output like this:
printf(
"FILE: %s, FUNC: %s, LINE: %d, LOG: %s\n",
__FILE__,
__FUNCTION__,
__LINE__,
logmessage
);
How can I do something similar in Python?
There is a module named inspect which provides these information.
Example usage:
import inspect
def PrintFrame():
callerframerecord = inspect.stack()[1] # 0 represents this line
# 1 represents line at caller
frame = callerframerecord[0]
info = inspect.getframeinfo(frame)
print(info.filename) # __FILE__ -> Test.py
print(info.function) # __FUNCTION__ -> Main
print(info.lineno) # __LINE__ -> 13
def Main():
PrintFrame() # for this line
Main()
However, please remember that there is an easier way to obtain the name of the currently executing file:
print(__file__)
For example
import inspect
frame = inspect.currentframe()
# __FILE__
fileName = frame.f_code.co_filename
# __LINE__
fileNo = frame.f_lineno
There's more here http://docs.python.org/library/inspect.html
Building on geowar's answer:
class __LINE__(object):
import sys
def __repr__(self):
try:
raise Exception
except:
return str(sys.exc_info()[2].tb_frame.f_back.f_lineno)
__LINE__ = __LINE__()
If you normally want to use __LINE__ in e.g. print (or any other time an implicit str() or repr() is taken), the above will allow you to omit the ()s.
(Obvious extension to add a __call__ left as an exercise to the reader.)
You can refer my answer:
https://stackoverflow.com/a/45973480/1591700
import sys
print sys._getframe().f_lineno
You can also make lambda function
I was also interested in a __LINE__ command in python.
My starting point was https://stackoverflow.com/a/6811020 and I extended it with a metaclass object. With this modification it has the same behavior like in C++.
import inspect
class Meta(type):
def __repr__(self):
# Inspiration: https://stackoverflow.com/a/6811020
callerframerecord = inspect.stack()[1] # 0 represents this line
# 1 represents line at caller
frame = callerframerecord[0]
info = inspect.getframeinfo(frame)
# print(info.filename) # __FILE__ -> Test.py
# print(info.function) # __FUNCTION__ -> Main
# print(info.lineno) # __LINE__ -> 13
return str(info.lineno)
class __LINE__(metaclass=Meta):
pass
print(__LINE__) # print for example 18
wow, 7 year old question :)
Anyway, taking Tugrul's answer, and writing it as a debug type method, it can look something like:
def debug(message):
import sys
import inspect
callerframerecord = inspect.stack()[1]
frame = callerframerecord[0]
info = inspect.getframeinfo(frame)
print(info.filename, 'func=%s' % info.function, 'line=%s:' % info.lineno, message)
def somefunc():
debug('inside some func')
debug('this')
debug('is a')
debug('test message')
somefunc()
Output:
/tmp/test2.py func=<module> line=12: this
/tmp/test2.py func=<module> line=13: is a
/tmp/test2.py func=<module> line=14: test message
/tmp/test2.py func=somefunc line=10: inside some func
import inspect
.
.
.
def __LINE__():
try:
raise Exception
except:
return sys.exc_info()[2].tb_frame.f_back.f_lineno
def __FILE__():
return inspect.currentframe().f_code.co_filename
.
.
.
print "file: '%s', line: %d" % (__FILE__(), __LINE__())
Here is a tool to answer this old yet new question!
I recommend using icecream!
Do you ever use print() or log() to debug your code? Of course, you
do. IceCream, or ic for short, makes print debugging a little sweeter.
ic() is like print(), but better:
It prints both expressions/variable names and their values.
It's 40% faster to type.
Data structures are pretty printed.
Output is syntax highlighted.
It optionally includes program context: filename, line number, and parent function.
For example, I created a module icecream_test.py, and put the following code inside it.
from icecream import ic
ic.configureOutput(includeContext=True)
def foo(i):
return i + 333
ic(foo(123))
Prints
ic| icecream_test.py:6 in <module>- foo(123): 456
To get the line number in Python without importing the whole sys module...
First import the _getframe submodule:
from sys import _getframe
Then call the _getframe function and use its' f_lineno property whenever you want to know the line number:
print(_getframe().f_lineno) # prints the line number
From the interpreter:
>>> from sys import _getframe
... _getframe().f_lineno # 2
Word of caution from the official Python Docs:
CPython implementation detail: This function should be used for internal and specialized purposes only. It is not guaranteed to exist in all implementations of Python.
In other words: Only use this code for personal testing / debugging reasons.
See the Official Python Documentation on sys._getframe for more information on the sys module, and the _getframe() function / submodule.
Based on Mohammad Shahid's answer (above).
I decided to try to preprocess function text before it's compilation into byte-code and following execution. This is merely for training. I hardly imagine situations where it'll be a satisfactory solution to be used. I have faced one problem which I wanted to solve in this way, but eventually a better way was found. So this is just for training and to learn something new, not for real usage.
Assume we have a function, which source code we want to be modified quite a bit before compilation:
def f():
1;a()
print('Some statements 1')
1;a()
print('Some statements 2')
Let, for example, mark some lines of it with 1;, for them to be sometimes commented and sometimes not. I just take it for example, modifications of the function may be different.
To comment these lines I made a decorator. The whole code it bellow:
from __future__ import print_function
def a():
print('a()')
def comment_1(s):
lines = s.split('\n')
return '\n'.join(line.replace(';','#;',1) if line.strip().startswith('1;') else line for line in lines)
def remove_1(f):
import inspect
source = inspect.getsource(f)
new_source = comment_1(source)
with open('temp.py','w') as file:
file.write(new_source)
from temp import f as f_new
return f_new
def f():
1;a()
print('Some statements 1')
1;a()
print('Some statements 2')
f = remove_1(f) #If decorator #remove is used above f(), inspect.getsource includes #remove inside the code.
f()
I used inspect.getsourcelines to retrieve function f code. Then I made some text-processing (in this case commenting lines starting with 1;). After that I saved it to temp.py module, which is then imported. And then a function f is decorated in the main module.
The output, when decorator is applied, is this:
Some statements 1
Some statements 2
when NOT applied is this:
a()
Some statements 1
a()
Some statements 2
What I don't like is that I have to use hard drive to load compiled function. Can it be done without writing it to temporary module temp.py and importing from it?
The second question is about placing decorator above f: #replace. When I do this, inspect.getsourcelines returns f text with this decorator. I could manually be deleted from f's text. but that would be quite dangerous, as there may be more than one decorator applied. So I resorted to the old-style decoration syntax f = remove_1(f), which does the job. But still, is it possible to allow normal decoration technique with #replace?
One can avoid creating a temporary file by invoking the exec statement on the source. (You can also explicitly call compile prior to exec if you want additional control over compilation, but exec will do the compilation for you, so it's not necessary.) Correctly calling exec has the additional benefit that the function will work correctly if it accesses global variables from the namespace of its module.
The problem described in the second question can be resolved by temporarily blocking the decorator while it is running. That way the decorator remains, along all the other ones, but is a no-op.
Here is the updated source.
from __future__ import print_function
import sys
def a():
print('a()')
def comment_1(s):
lines = s.split('\n')
return '\n'.join(line.replace(';','#;',1) if line.strip().startswith('1;') else line for line in lines)
_blocked = False
def remove_1(f):
global _blocked
if _blocked:
return f
import inspect
source = inspect.getsource(f)
new_source = comment_1(source)
env = sys.modules[f.__module__].__dict__
_blocked = True
try:
exec new_source in env
finally:
_blocked = False
return env[f.__name__]
#remove_1
def f():
1;a()
print('Some statements 1')
1;a()
print('Some statements 2')
f()
def remove_1(f):
import inspect
source = inspect.getsource(f)
new_source = comment_1(source)
env = sys.modules[f.__module__].__dict__.copy()
exec new_source in env
return env[f.__name__]
I'll leave a modified version of the solution given in the answer by user4815162342. It uses ast module to delete some parts of f, as was suggested in the comment to the question. To make it I majorly relied on the information in this article.
This implementation deletes all occurrences of a as standalone expression.
from __future__ import print_function
import sys
import ast
import inspect
def a():
print('a() is called')
_blocked = False
def remove_1(f):
global _blocked
if _blocked:
return f
import inspect
source = inspect.getsource(f)
a = ast.parse(source) #get ast tree of f
class Transformer(ast.NodeTransformer):
'''Will delete all expressions containing 'a' functions at the top level'''
def visit_Expr(self, node): #visit all expressions
try:
if node.value.func.id == 'a': #if expression consists of function with name a
return None #delete it
except(ValueError):
pass
return node #return node unchanged
transformer = Transformer()
a_new = transformer.visit(a)
f_new_compiled = compile(a_new,'<string>','exec')
env = sys.modules[f.__module__].__dict__
_blocked = True
try:
exec(f_new_compiled,env)
finally:
_blocked = False
return env[f.__name__]
#remove_1
def f():
a();a()
print('Some statements 1')
a()
print('Some statements 2')
f()
The output is:
Some statements 1
Some statements 2