I found that Python's assert statement is a good way to catch situations that should never happen. And it can be removed by Python optimization when the code is trusted to be correct.
It seems to be a perfect mechanism to run Python applications in debug mode. But looking at several Python projects like django, twisted and zope, the assert is almost never used. So, why does this happen?
Why are asserts statements not frequently used in the Python community?
I guess the main reason for assert not being used more often is that nobody uses Python's "optimized" mode.
Asserts are a great tool to detect programming mistakes, to guard yourself from unexpected situations, but all this error checking comes with a cost. In compiled languages such as C/C++, this does not really matter, since asserts are only enabled in debug builds, and completely removed from release builds.
In Python, on the other hand, there is no strict distinction between debug and release mode. The interpreter features an "optimization flag" (-O), but currently this does not actually optimize the byte code, but only removes asserts.
Therefore, most Python users just ignore the -O flag and run their scripts in "normal mode", which is kind of the debug mode since asserts are enabled and __debug__ is True, but is considered "production ready".
Maybe it would be wiser to switch the logic, i.e., "optimize" by default and only enable asserts in an explicit debug mode(*), but I guess this would confuse a lot of users and I doubt we will ever see such a change.
((*) This is for example how the Java VM does it, by featuring a -ea (enable assertions) switch.)
Several reasons come to mind...
It is not a primary function
Many programmers, lets not get bogged down by the rationale, disrespect anything which is not a direct participant in the program's penultimate functionality. The assert statement is intended for debugging and testing, and so, a luxury they can ill-afford.
Unit Testing
The assert statement predates the rise and rise of unit-testing. Whilst the assert statement still has its uses, unit-testing is now widely used for constructing a hostile environment with which to bash the crap out of a subroutine and its system. Under these conditions assert statements start to feel like knives in a gunfight.
Improved industry respect for testing
The assert statement serves best as the last line of defence. It rose to lofty and untouchable heights under the C language, when that language ruled the world, as a great way to implement the new-fangled "defensive programming"; it recognises and traps catastrophic disasters in the moment they teeter on the brink. This was before the value of Testing became widely recognised and respected and disasters were substantially more common.
Today, it is unheard of, for any serious commercial software to be released without some form of testing. Testing is taken seriously and has evolved into a massive field. There are Testing professionals and Quality Assurance departments with big checklists and formal sign-offs. Under these conditions programmers tend not to bother with asserts because they have confidence that their code will be subjected to so much tiresome testing that the odds of wacky brink-of-disaster conditions are so remote as to be negligible. That's not to say they're right, but if the blame for lazy programming can be shifted to the QA department, hell why not?
I'm not an author of any of those projects, so this is just a guess based on my own experiences. Without directly asking people in those projects you won't get a concrete answer.
Assert is great when you're trying to do debugging, etc in your own application. As stated in the link you provided, however, using a conditional is better when the application might be able to predict and recover from a state. I haven't used zope, but in both Twisted and Django, their applications are able to recover and continue from many errors in your code. In a sense, they have already 'compiled away' the assertions since they actually can handle them.
Another reason, related to that, is that often applications using external libraries such as those you listed might want to do error handling. If the library simply uses assertions, no matter what the error is it will raise an AssertionError. With a conditional, the libraries can actually throw useful errors that can be caught and handled by your application.
As per my experience, asserts are majorly used in development phase of a program-to check the user defined inputs. asserts are not really needed to catch programming errors. Python itself is very well capable of trapping genuine programming errors like ZeroDivisionError, TypeError or so.
Related
I have a static library I created from C++, and would like to test this using a Driver code.
I noticed one of my professors like to do his tests using python, but he simply executes the program (not a library in this case, but an executable) using random test arguments.
I would like to take this approach, but I realized that this is a library and doesn't have a main function; that would mean I should either create a Driver.cpp class, or wrap the library into python using SWIG or boost python.
I’m planning to do the latter because it seems more fun, but logically, I feel that there is going to be more bugs when trying to wrap a library to a different language just to test it, rather than test it in its native language.
Is testing programs in a different language an accepted practice in the real world, or is this bad practice?
I'd say that it's best to test the API that your users will be exposed to. Other tests are good to have as well, but that's the most important aspect.
If your users are going to write C/C++ code linking to your library, then it would be good to have tests making use of your library the same way.
If you are going to ship a Python wrapper (why not?) then you should have Python tests.
Of course, there is a convenience aspect to this, as well. It may be easier to write tests in Python, and you might have time constraints that make it more appealing, etc.
I guess what I'm saying is: There's nothing inherently wrong with tests being in a different language from the code under test (that's totally normal for testing a REST API, for instance), but make sure you have tests for the public-facing API at a minimum.
Aside, on terminology:
I don't think the types of tests you are describing are "unit tests" in the usual sense of the term. Probably "functional test" would be more accurate.
A unit test typically tests a very small component - such as a function call - that might be one piece of larger functionality. Unit tests like these are often "white box" tests, so you can see the inner workings of your code.
Testing something from a user's point-of-view (such as your professor's commandline tests) are "black box" tests, and in these examples are at a more functional level rather than "unit" level.
I'm sure plenty of people may disagree with that, though - it's not a rigidly-defined set of terms.
A few things to keep in mind:
If you are writing tests as you code, then, by all means, use whatever language works best to give you rapid feedback. This enables fast test-code cycles (and is fun as well). BUT.
Always have well-written tests in the language of the consumer. How is your client/consumer going to call your functions? What language will they be using? Using the same language minimizes integration issues later on in the life-cycle.
It really depends on what it is you are trying to test. It almost always makes sense to write unit tests in the same language as the code you are testing so that you can construct the objects under test or invoke the functions under test, both of which can be most easily done in the same language, and verify that they work correctly. There are, however, cases in which it makes sense to use a different language, namely:
Integration tests that run a number of different components or applications together.
Tests that verify compilation or interpretation failures which could not be tested in the language, itself, since you are validating that an error occurs at the language level.
An example of #1 might be a program that starts up multiple different servers connected to each other, issues requests to the server, and verifies those responses. Or, as a simpler example, a program that simply forks an application under test as a subprocess and verifies that it produces the expected outputs for a given input.
An example of #2 might be a program that verifies that a certain piece of C++ code will produce a static assertion failure or that a particular template instantiation which is intentionally disallowed will result in a compilation failure if someone attempts to use it.
To answer your larger question, it is not bad practice per-se to write tests in a different language. Whatever makes the tests more convenient to write, easier to understand, more robust to changes in implementation, more sensitive to regressions, and better on any one of the properties that define good testing would be a good justification to write the tests one way vs another. If that means writing the tests in another language, then go for it. That being said, small unit tests typically need to be able to invoke the item under test directly which, in most cases, means writing the unit tests in the same language as the component under test.
I would say it depends on what you're actually trying to test. For true unit testing, it is, I think, best to test in the same language, or at least a binary-compatible language (i.e. testing Java with Groovy -- I use Spock in this case, which is Groovy based, to unit-test my Java code, since I can intermingle the Java with the Groovy), but if you are testing results, then I think it's fair to switch languages.
For example, I have tested the expected results when given a specific set of a data when running a Perl application via nose in Python. This works because I'm not unit testing the Perl code, per se, but the outcomes of that Perl code.
In that case, to unit test actual Perl functions that are part of the application, I would use a Perl-based test framework such as Test::More.
Why not, it's an awesome idea because you really understand that you are testing the unit like a black box.
Of course there may be technical issues involved, what if you need to mock some parts of the unit under test, that may be difficult in a different language.
This is a common practice for integration tests though, I've seen lots of programs driven from external tools such as a website from selenium, or an application from cucumber. Both those can be considered the same as a custom python script.
If you consider the difference between integration testing and unit testing is the number of things under test at any given time, the only reason why you shouldn't do this is tool support.
While I tried to submit my code which contains eval() function, all of the web compilers like those in pythontutor.com, programmr.com, coursera.org autotester, etc., returned a name error. What is the reason for not implementing this function on web compilers?
An eval function can certainly be abused and is a security risk.
Code execution can be limited on multiple levels. The most powerful tool to limit the functionality of code is AST inspection and AST modification. Python code is parsed, tokenized, transformed into an abstract syntax tree and finally the abstract syntax tree is validated and modified.
Eval and exec can be abused to sneak code around AST inspection. Because the feature is not needed for online examples it's simply omitted.
I'll take a stab at this:
EDIT
I don't like admitting I'm wrong but in this case I think I have to.
I'll point out that it's still true that you're executing use code either way, so you need to actually secure the interpreter. I still believe that "there is no point whatsoever trying to blacklist, or even whitelist, allowed actions if those actions exist within the interpreter" and thus blocking exec and eval aids only minor benefit.
Here's the key point:
The maker of that Python Tutor executed code handed to them from the outside. Despite blocking many things, including exec, because they allowed executing of foreign code there is an exploit I have found. Currently I've only imported a blocked module, but reading the source this also allows me to do unlimited IO.
Further Edit: I've contacted the owner of Python Tutor and he's said he's aware of such, supposedly protections exist at a lower level, too. So it's not an exploit, but that's primarily because there are protections at "a lower level". That's how you're meant to guard against attacks, not blacklists.
Further, Further Edit: I've brought down Python Tutor, although the site seems to be up again (not functional, just up), 'cause I was seeing whether I could read/write anything with this exploit of mine (at the request of Philip Guo, the owner). It seems I can at least break the site :/. Sorry, 'twas an accident. It does prove that running untrusted code is dangerous, though, which is what I've been saying all along.
However, the question asks about intention, not about whether blocking exec actually makes a difference.
Reading http://pgbovine.net/projects/pubs/guo-sigcse-preprint_2012-11-13.pdf, namely
Since the backend is running untrusted Python code from the web, it implements sandboxing to prevent the execution of dangerous constructs such as eval, exec, file I/O and most module imports (except for a customisable whitelist of modules such as math).
I won't admit that I'm wrong about how you should treat exec and eval, but I was wrong about what the site's owner thought you should do (although admittedly he realises that you also have to sandbox).
What I previously wrote:
Firstly, it is almost definitely nothing to do with safety. Safety by obscurity never works, and it's a silly thing to try. You cannot blacklist bad functions by looking for calls to the function in a dynamic language. An easy example of obscuring
import os; while True: os.fork()
would be
import os
while True:
os.__dict__["fork"]()
so there's no protection. I'll say it again: there is no point whatsoever trying to blacklist, or even whitelist, allowed actions if those actions exist within the interpreter. This means that anything eval can reach can be reached without eval, so blocking eval gives you no security.
I reckon the real reason is that these aren't typical Python compiler-interpreters. Much of what the code is doing is pushed onto Javascript or some other awkward platform.
If you ever deeply inspect code, you'll realise that at compile-to-bytecode-time some stuff gets hard coded, such as name bindings. When running things like "eval" you cannot just run the code inline, you have to compile and dynamically extract things from a scope. Supporting such dynamism in a target language is a challenging task, and the writers probably just didn't think that it was worth doing.
This is especially true if the code is compiled in the cloud and executed locally: eval would therefore have to take a round-trip to the server in order to work! This isn't an advised course of action.
That's my guess, although I don't know how this stuff is actually implemented.
So I have Emacs 24.3 and with it comes a quite recent python.el file providing a Python mode for editing.
But I keep reading that there is a python-mode.el on Launchpad, and comparing the two files it jumps out to me that the former is under 4000 lines, while the latter is almost 20000. This suggests that the latter is much more feature-rich.
And I can't find any online feature comparison about them, documentation, or at least a list about the features for each of them. Yep, there is syntax highlighting and embedded interpreter, but what about completion in shell buffer, completion in source file buffer, autoindent, reindent etc.
So what are the important features of these modes? (Or any other Python mode for Emacs which you recommend.) Please provide detailed answers.
I was python-mode.el user once but quit using it a year ago because I felt the way it was developed was not well organized. Here is a list from the note I took at that time. But I need to warn you that almost a year is passed since then so the situation may be changed.
Many copy & paste functions.
Many accidentally working code. For example, not passing around variables but using implicit binding. This produces many compile errors (and will not work if you change it to lexical scope).
Rough granularity of commit. I send a patch, and it was committed with unrelated changes.
One thing I like about python-mode.el is it comes with automated test set (although I've never run it). python.el does not have a test set yet. But I know the author of python.el is writing it now.
While python.el is compact, it does not mean you get poor functionality. It is more like keeping core small and let others to extend it by providing concise API. Same author of python.el wrote python-django.el to extend python.el for django projects. I wrote auto-completion plugin for Python called Jedi.el and advanced IPython plugin called EIN. Both of them have better support for python.el than python-mode.el (well, that's because I don't use python-mode.el, though).
I had a few thing I missed from python-mode.el first, but they are quickly fixed in python.el (Of course, this probably means that I was not using so much functionality in python-mode.el).
what about completion in shell buffer, completion in source file buffer, autoindent, reindent etc.
completion in shell buffer:
It sort of works in both python.el and python-mode.el. But sometimes it does not work if you have bad combination of Emacs version and python(-mode).el version. So probably python.el is safer in this manner.
But if you want better solution, use EIN :)
completion in source file buffer:
Just use Jedi.el :)
autoindent/reindent:
I don't know which one is better in performance-wise. However, keybind for return differs one to the other. In python-mode.el, if you type RET you get autoindent. In python.el, RET does not give you indentation and you should use C-j instead. Actually C-j for newline+indentation is universal behavior in Emacs. So python.el is better if you do programming in other languages.
Being heavily involved in developing python-mode.el last years, my comment probably is biased: Recommend to stay with python.el for Emacs beginners. Also its author deserves credit for some useful approaches.
python-mode.el is designed to boost edits productivity. It makes it easy to run or execute via python2 and python3 or IPython shells in parallel.
It reduces number of needed keystrokes providing tailored commands. It makes edits faster, assists programming by speech, macro-driven input etc.
Supports Python language features not known from python.el currently:
py-up, py-down - travelling nested blocks
Avoid typos fetching forms at point, a clause for example:
py-backward-clause
py-copy-clause
py-down-clause
...
No need to customize when testing different versions:
py-execute-clause-python2
py-execute-clause-python3
py-execute-clause-ipython
...
notion of finer grained parts - py-expression, py-minor-expression
commands running versioned and paralleled (I)Python-executables, no need to re-define the default Python
largely removing the need of an active region marked before, see py-execute-line and a whole bunch more
To get an overview, have a look at the menu. Directory "doc" lists commands.
As the quality of code is raised: a way to compare both modes probably is checking for the bugs listed in http://debbugs.gnu.org/. See for example bug #15510, #16875; or http://lists.gnu.org/archive/html/help-gnu-emacs/2014-04/msg00250.html
Commented at "rough granularity of commit" already: While tkf basically is right looking for smaller pieces, sometimes conditions make me leave the rule. Considerable parts are not written by hand, but by programs residing in directory "devel". They create files used in development branch frist - i.e. components-python-mode. When starting a new feature it's often not obvious if the path chosen is fruitful.
After a hundred would-be-commits or so it still might turn out impossible or not so recommendable. Instead of posting all the meanders, used to keep that experimental branch for several days in these cases and check in when tests passed.
BTW assume tkf refers not to compile-errors --which would be looked for instantly-- but compiler warnings. Unfortunately Emacs mixes warning about backed style preferences with real errors.
I've never been a huge Python fan. I learned it for a course where the teacher was really into it, but his enthusiasm never quite made it to the rest of our class it seems: as soon as we had the chance, we all jumped off to C#/Java.
Anyways. This wasn't a concluding experience, and what annoyed me the most in the language was that to find out if Python code would work, you actually have to execute it, and risk dying halfway through because of something stupid like a typo in a variable name (throwing up a NameError). Stuff that compilers for compiled languages catch at the very first glance, but that Python won't bother to complain about until it's too late. (I know you can always die half through a test with compiled programs too, but at least it won't be from a typo.)
I'm not really giving it a second chance yet, but for the sake of the next students, are there Python statical analysis or validation tools out there that would catch most errors (I understand you can't catch them all) compilers would catch at compile-time?
"but that Python won't bother to complain about until it's too late"
It's not that the message comes too late. It's that you're waiting too long to use Python. Don't type a mountain of code and then complain that one small piece is bad.
Use Unit Testing. Write less code before running a test.
Use python Interactively to experiment. You can do most statistical processing from the >>> prompt.
Don't write long, main-program-like scripts. Write short scripts -- in small pieces -- and test the small pieces.
Take a look at the following programs:
pylint
pyflakes
pychecker
In addition to the ones mentioned by ars.
Try Pydev, it has static code analysis build-in. Or Pida which has a couple of different static analysis tools available.
Or if you are looking for a standalone library, try Rope
I am basically from the world of C language programming, now delving into the world of scripting languages like Ruby and Python.
I am wondering how to do debugging.
At present the steps I follow is,
I complete a large script,
Comment everything but the portion I
want to check
Execute the script
Though it works, I am not able to debug like how I would do in, say, a VC++ environment or something like that.
My question is, is there any better way of debugging?
Note: I guess it may be a repeated question, if so, please point me to the answer.
Your sequence seems entirely backwards to me. Here's how I do it:
I write a test for the functionality I want.
I start writing the script, executing bits and verifying test results.
I review what I'd done to document and publish.
Specifically, I execute before I complete. It's way too late by then.
There are debuggers, of course, but with good tests and good design, I've almost never needed one.
Here's a screencast on ruby debugging with ruby-debug.
Seems like the problem here is that your environment (Visual Studio) doesn't support these languages, not that these languages don't support debuggers in general.
Perl, Python, and Ruby all have fully-featured debuggers; you can find other IDEs that help you, too. For Ruby, there's RubyMine; for Perl, there's Komodo. And that's just off the top of my head.
There is a nice gentle introduction to the Python debugger here
If you're working with Python then you can find a list of debugging tools here to which I just want to add Eclipse with the Pydev extension, which makes working with breakpoints etc. also very simple.
My question is, is there any better way of debugging?"
Yes.
Your approach, "1. I complete a large script, 2. Comment everything but the portion I want to check, 3. Execute the script" is not really the best way to write any software in any language (sorry, but that's the truth.)
Do not write a large anything. Ever.
Do this.
Decompose your problem into classes of objects.
For each class, write the class by
2a. Outline the class, focus on the external interface, not the implementation details.
2b. Write tests to prove that interface works.
2c. Run the tests. They'll fail, since you only outlined the class.
2d. Fix the class until it passes the test.
2e. At some points, you'll realize your class designs aren't optimal. Refactor your design, assuring your tests still pass.
Now, write your final script. It should be short. All the classes have already been tested.
3a. Outline the script. Indeed, you can usually write the script.
3b. Write some test cases that prove the script works.
3c. Runt the tests. They may pass. You're done.
3d. If the tests don't pass, fix things until they do.
Write many small things. It works out much better in the long run that writing a large thing and commenting parts of it out.
Script languages have no differences compared with other languages in the sense that you still have to break your problems into manageable pieces -- that is, functions. So, instead of testing the whole script after finishing the whole script, I prefer to test those small functions before integrating them. TDD always helps.
There's a SO question on Ruby IDEs here - and searching for "ruby IDE" offers more.
I complete a large script
That's what caught my eye: "complete", to me, means "done", "finished", "released". Whether or not you write tests before writing the functions that pass them, or whether or not you write tests at all (and I recommend that you do) you should not be writing code that can't be run (which is a test in itself) until it's become large. Ruby and Python offer a multitude of ways to write small, individually-testable (or executable) pieces of code, so that you don't have to wait for (?) days before you can run the thing.
I'm building a (Ruby) database translation/transformation script at the moment - it's up to about 1000 lines and still not done. I seldom go more than 5 minutes without running it, or at least running the part on which I'm working. When it breaks (I'm not perfect, it breaks a lot ;-p) I know where the problem must be - in the code I wrote in the last 5 minutes. Progress is pretty fast.
I'm not asserting that IDEs/debuggers have no place: some problems don't surface until a large body of code is released: it can be really useful on occasion to drop the whole thing into a debugging environment to find out what is going on. When third-party libraries and frameworks are involved it can be extremely useful to debug into their code to locate problems (which are usually - but not always - related to faulty understanding of the library function).
You can debug your Python scripts using the included pdb module. If you want a visual debugger, you can download winpdb - don't be put off by that "win" prefix, winpdb is cross-platform.
The debugging method you described is perfect for a static language like C++, but given that the language is so different, the coding methods are similarly different. One of the big very important things in a dynamic language such as Python or Ruby is the interactive toplevel (what you get by typing, say python on the command line). This means that running a part of your program is very easy.
Even if you've written a large program before testing (which is a bad idea), it is hopefully separated into many functions. So, open up your interactive toplevel, do an import thing (for whatever thing happens to be) and then you can easily start testing your functions one by one, just calling them on the toplevel.
Of course, for a more mature project, you probably want to write out an actual test suite, and most languages have a method to do that (in Python, this is doctest and nose, don't know about other languages). At first, though, when you're writing something not particularly formal, just remember a few simple rules of debugging dynamic languages:
Start small. Don't write large programs and test them. Test each function as you write it, at least cursorily.
Use the toplevel. Running small pieces of code in a language like Python is extremely lightweight: fire up the toplevel and run it. Compare with writing a complete program and the compile-running it in, say, C++. Use that fact that you can quickly change the correctness of any function.
Debuggers are handy. But often, so are print statements. If you're only running a single function, debugging with print statements isn't that inconvenient, and also frees you from dragging along an IDE.
There's a lot of good advice here, i recommend going through some best practices:
http://github.com/edgecase/ruby_koans
http://blog.rubybestpractices.com/
http://on-ruby.blogspot.com/2009/01/ruby-best-practices-mini-interview-2.html
(and read Greg Brown's book, it's superb)
You talk about large scripts. A lot of my workflow is working out logic in irb or the python shell, then capturing them into a cascade of small, single-task focused methods, with appropriate tests (not 100% coverage, more focus on edge and corner cases).
http://binstock.blogspot.com/2008/04/perfecting-oos-small-classes-and-short.html