Develop Reference

Refactoring Test Logic in Pytest to Avoid Complex Raises Block

I'm using the flake8-pytest-style plugin and it flags a certain test as violating PT012. This is about having too much logic in the raises() statement.
The code in question is this:
def test_bad_python_version(capsys) -> None:
import platform
from quendor.__main__ import main
with pytest.raises(SystemExit) as pytest_wrapped_e, mock.patch.object(
platform,
"python_version",
) as v_info:
v_info.return_value = "3.5"
main()
terminal_text = capsys.readouterr()
expect(terminal_text.err).to(contain("Quendor requires Python 3.7"))
expect(pytest_wrapped_e.type).to(equal(SystemExit))
expect(pytest_wrapped_e.value.code).to(equal(1))
Basically this is testing the following code:
def main() -> int:
if platform.python_version() < "3.7":
sys.stderr.write("\nQuendor requires Python 3.7 or later.\n")
sys.stderr.write(f"Your current version is {platform.python_version()}\n\n")
sys.exit(1)
What I do is just pass in a version of Python that is less than the required and make sure the error appears as expected. The test itself works perfectly fine. (I realize it can be questionable as to whether this should be a unit test at all since it's really testing more of an aspect of Python than my own code.)
Clearly the lint check is suggesting that my test is a little messy and I can certainly understand that. But it's not clear from the above referenced page what I'm supposed to do about it.
I do realize I could just disable the quality check for this particular test but I'm trying to craft as good of Python code as I can, particularly around tests. And I'm at a loss as to how to refactor this code to meet the criteria.
I know I can create some other test helper function and then have that function called from the raises block. But that strikes me as being less clear overall since now you have to look in two places in order to see what the test is doing.

the lint error is a very good one! in fact in your case because the lint error is not followed you have two lines of unreachable code (!) (the two capsys-related lines) because main() always raises
the lint is suggesting that you only have one line in a raises() block -- the naive refactor from your existing code is:
with mock.patch.object(
platform,
"python_version",
return_value="3.5",
):
with pytest.raises(SystemExit) as pytest_wrapped_e:
main()
terminal_text = capsys.readouterr()
expect(terminal_text.err).to(contain("Quendor requires Python 3.7"))
expect(pytest_wrapped_e.type).to(equal(SystemExit))
expect(pytest_wrapped_e.value.code).to(equal(1))
an aside, you should never use platform.python_version() for version comparisons as it produces incorrect results for python 3.10 -- more on that and a linter for it here

Why is try: except: not working for this particular line of code with python Wikipedia API?

i am trying to execute this very short code that took out from my bigger code because i was having problems with it. I was able to reproduce the problem (it gave the exact same error).
BACKGROUND INFO: i am just trying to get the first sentence out of this wikipedia search. but because the word i am searching for (kip, means chicken in Dutch) has a wide variety of meanings or something i get an error. i want to bypass this error using try: except: but it keeps displaying the error message anyway.
here is the code that just doesnt seem to work:
import wikipedia
wikipedia.set_lang('nl')
try:
summry = wikipedia.summary('kip', sentences=1)
print(summry + "\n")
except:
print("error")
i have tried replacing except: with this
except wikipedia.exceptions.DisambiguationError:
but it still doesnt work :( it always displays the error code regradless and prints "error" afterwards
/opt/virtualenvs/python3/lib/python3.8/site-packages/wikipedia/wikipedia.py:389:
GuessedAtParserWarning: No parser was explicitly specified, so I'm using the best available HTML
parser for this system ("html5lib"). This usually isn't a problem, but if you run this code on
another system, or in a different virtual environment, it may use a different parser and behave
differently.
The code that caused this warning is on line 389 of the file
/opt/virtualenvs/python3/lib/python3.8/site-packages/wikipedia/wikipedia.py. To get rid of this
warning, pass the additional argument 'features="html5lib"' to the BeautifulSoup constructor.
lis = BeautifulSoup(html).find_all('li')
error
i am using repl.it to program this
if anyone has any idea about why it keeps displaying the error anyway please please let me know :D

First of all, thank you all for commenting :D it helped a lot
At the end the solution that worked for me was inspired from Askold Ilvento's comment
although
with warnings.catch_warnings(): warnings.simplefilter("ignore")
didn't work when i adapted it a little bit it did the job!
this is the code that solved the problem and allowed me to ignore what was actually a warning (not an exception) and stop displaying it
import warnings
warnings.catch_warnings()
warnings.simplefilter("ignore")
i just added this at the start of the script and it solved the problem :D
once again all the credit for this code goes to Askold Ilvento i just had to add a little bit to it to make it work

How to tackle SyntaxError in Python

Since there are a bunch of questions here on Stack Overflow that deal with SyntaxError in Python, we might want to know:
How do we tackle a SyntaxError? Are there strategies that can generally be applied?

0. Before the Error appears: Syntax Highlighting and Code Formatting
Even before running into a SyntaxError, there are important measurements to deal with SyntaxErrors, because the best way to deal with SyntaxErrors is to avoid them in the first place. This can be done first and foremost by using an editor or an Integrated Development Environment (IDE) which has syntax highlighting for Python.
Besides that, we can decrease the risk of running into a SyntaxError by good code and formatting style. There is a formal definition of the term "good formatting style", PEP 8 -- Style Guide for Python Code. Proper formatting makes our code much more readable which decreases the risk writing code that leads to a SyntaxError.
A very good way to apply good formatting to our code is to use an automatic code formatting tool. A code formatter has multiple advantages, amongst which are the following: Its code formatting is consistent. It applies best practices you might not even have thought of yet. It is very convenient.
For Python, black is a great code formatting tool.
1. Understand the Error Message
The Syntax Error indicates in which file and in which line the interpreter came across a problem in our code. We should use this information to find the bug.
We should be aware that the Python interpreter sometimes indicates a SyntaxError in the line after the actual problem. This is because the parser expects something in the erroneous line and can recognise that this is missing only when the whole line has been parsed. The prototypic example for that kind of SyntaxError is a missing parenthesis. So for instance, the following code raises a SyntaxError in line 2, even though the bug is in line 1:
bar = foo(
baz()
EOL stands for "End Of Line". This helps understanding the very common SyntaxError: EOL while scanning string literal. This is usually raised when you did not properly close a string definition with closing quotation marks, such as in the following example:
foo = "bar
2. Simplify the Code
Generally, a good strategy of bug fixing is to reduce any code that throws an Error or an Exception (or that does not return the expected output) to a minimal example. (This is a requirement for questions here on Stack Overflow, but much more than this, it is a good technique for pinning down a bug.)
In case of a SyntaxError, producing a minimal example is usually very easy, because a SyntaxError does not depend on any values of a variable, or any state of an object or any other semantics of your code. That's why the source of a SyntaxError is usually one line of code.
So, to identify the bug, we remove all the code besides the line that we think is the source of the Error. If the Error vanishes, it has been in a different line. If the Error persists, we try to simplify this line. For instance, we replace nested parentheses, by defining intermediate variables that hold the values:
Instead of
bar = foo(foo(baz(foo()))
the following (logically equivalent) code:
first = foo()
second = baz(first)
third = foo(second)
bar = foo(third
makes it much easier for us to identify the missing closing parenthesis.

Refer to documentation. Syntax Errors unfortunately cannot be captured in a Try: Except: block, so the only way to deal with them is to read the message returned, and if that doesn't help, following up with the python documentation:
https://docs.python.org/3/

Ignore the rest of the python file

My python scripts often contain "executable code" (functions, classes, &c) in the first part of the file and "test code" (interactive experiments) at the end.
I want python, py_compile, pylint &c to completely ignore the experimental stuff at the end.
I am looking for something like #if 0 for cpp.
How can this be done?
Here are some ideas and the reasons they are bad:
sys.exit(0): works for python but not py_compile and pylint
put all experimental code under def test():: I can no longer copy/paste the code into a python REPL because it has non-trivial indent
put all experimental code between lines with """: emacs no longer indents and fontifies the code properly
comment and uncomment the code all the time: I am too lazy (yes, this is a single key press, but I have to remember to do that!)
put the test code into a separate file: I want to keep the related stuff together
PS. My IDE is Emacs and my python interpreter is pyspark.

Use ipython rather than python for your REPL It has better code completion and introspection and when you paste indented code it can automatically "de-indent" the pasted code.
Thus you can put your experimental code in a test function and then paste in parts without worrying and having to de-indent your code.
If you are pasting large blocks that can be considered individual blocks then you will need to use the %paste or %cpaste magics.
eg.
for i in range(3):
i *= 2
# with the following the blank line this is a complete block
print(i)
With a normal paste:
In [1]: for i in range(3):
...: i *= 2
...:
In [2]: print(i)
4
Using %paste
In [3]: %paste
for i in range(10):
i *= 2
print(i)
## -- End pasted text --
0
2
4
In [4]:
PySpark and IPython
It is also possible to launch PySpark in IPython, the enhanced Python interpreter. PySpark works with IPython 1.0.0 and later. To use IPython, set the IPYTHON variable to 1 when running bin/pyspark:1
$ IPYTHON=1 ./bin/pyspark

Unfortunately, there is no widely (or any) standard describing what you are talking about, so getting a bunch of python specific things to work like this will be difficult.
However, you could wrap these commands in such a way that they only read until a signifier. For example (assuming you are on a unix system):
cat $file | sed '/exit(0)/q' |sed '/exit(0)/d'
The command will read until 'exit(0)' is found. You could pipe this into your checkers, or create a temp file that your checkers read. You could create wrapper executable files on your path that may work with your editors.
Windows may be able to use a similar technique.
I might advise a different approach. Separate files might be best. You might explore iPython notebooks as a possible solution, but I'm not sure exactly what your use case is.

Follow something like option 2.
I usually put experimental code in a main method.
def main ():
*experimental code goes here *
Then if you want to execute the experimental code just call the main.
main()

With python-mode.el mark arbitrary chunks as section - for example via py-sectionize-region.
Than call py-execute-section.
Updated after comment:
python-mode.el is delivered by melpa.
M-x list-packages RET
Look for python-mode - the built-in python.el provides 'python, while python-mode.el provides 'python-mode.
Developement just moved hereto: https://gitlab.com/python-mode-devs/python-mode

I think the standard ('Pythonic') way to deal with this is to do it like so:
class MyClass(object):
...
def my_function():
...
if __name__ == '__main__':
# testing code here
Edit after your comment
I don't think what you want is possible using a plain Python interpreter. You could have a look at the IEP Python editor (website, bitbucket): it supports something like Matlab's cell mode, where a cell can be defined with a double comment character (##):
## main code
class MyClass(object):
...
def my_function():
...
## testing code
do_some_testing_please()
All code from a ##-beginning line until either the next such line or end-of-file constitutes a single cell.
Whenever the cursor is within a particular cell and you strike some hotkey (default Ctrl+Enter), the code within that cell is executed in the currently running interpreter. An additional feature of IEP is that selected code can be executed with F9; a pretty standard feature but the nice thing here is that IEP will smartly deal with whitespace, so just selecting and pasting stuff from inside a method will automatically work.

I suggest you use a proper version control system to keep the "real" and the "experimental" parts separated.
For example, using Git, you could only include the real code without the experimental parts in your commits (using add -p), and then temporarily stash the experimental parts for running your various tools.
You could also keep the experimental parts in their own branch which you then rebase on top of the non-experimental parts when you need them.

Another possibility is to put tests as doctests into the docstrings of your code, which admittedly is only practical for simpler cases.
This way, they are only treated as executable code by the doctest module, but as comments otherwise.

Code changes from Python 2.6 to 3.x

I am trying to get pywbem working in Python 3.2 (it works fine in 2.6) but the build fails on this part of code in mof_compiler.py:
File "pywbem-0.7.0\mof_compiler.py", line 1341
print s
^
SyntaxError: invalid syntax
It's a macro, defined like this:
def _print_logger(s):
print s
I don't understand why this is invalid, please explain how to do the same in Python 3.2.
Note: I have little or no experience with Python.
PS: I have already done some small changes to the code for 3.2 like
changing
except CIMError, ce:
to
except CIMError as ce:
based on Lennart Regebro's answer here are some other changes I found (placing them here since it may be useful for others).
exec "import %s as lextab" % tabfile -> exec ("import %s as lextab" % tabfile)
raise ValueError,"Expected a string" -> raise ValueError("Expected a string")

That's not a macro, it's a function definition, and in Python 3 the print statement is now a function. So do print(s) instead.
The list of changes between Python 2 and Python 3 is here: http://docs.python.org/release/3.0.1/whatsnew/3.0.html
It's not so easy to read, but I don't know if there is a better one online (although books exist).
If you are going to use Python 3, you would probably do good to get a Python 3 book. There are a couple of them out now. Or at least refer to the Python 3 documentation: http://docs.python.org/release/3.2/ It has a decent tutorial.

One of the most visible changes in python 3 is print is no longer a statement, but is a function, so you have to use parenthesis for calling that function. print(s)
Also, if you have your Python2 code, just use 2to3 which can do a source to source translation of your python2 to python3, which can fix most of the syntax level changes for you like the above problems. 2to3 is installed with python3 binary.

Sorry for answering an old question, but I just recently wanted to get PyWBEM running under Python 3, so I forked it, made the required changes, and removed a Python 2.x dependency (M2Crypto) from it for the 3.x series. Here's the source from GitHub:
https://github.com/deejross/python3-pywbem
Quick note, this supports Python 2.6, 2.7, and 3.4+