I have been reading about testing in django. One thing that was recommended was use of django-webtest for functional testing. I found a decent article here that teaches how to go about functional testing in selenium using python. But people have also recommended Ian Bicking's WebTest's extension djagno-webtest to use for testing forms in django. How is testing with webtest and testing with selenium different in context of django forms?
So from functional testing point of view:
How does django-webtest and selenium go side by side?
Do we need to have both of them or any one would do?
The key difference is that selenium runs an actual browser, while WebTest hooks to the WSGI.
This results in the following differences:
You can't test JS code with WebTest, since there is nothing to run it.
WebTest is much faster since it hooks to the WSGI, this also means a smaller memory footprint
WebTest does not require to actually run the server on a port so it's a bit easier to parallize
WebTest does not check different problems that occur with actual browsers, like specific browser version bugs (cough.. internet explorer.. cough..)
Bottom line:
PREFER to use WebTest, unless you MUST use Selenium for things that can't be tested with WebTest.
The important thing to know about Selenium is that it's primarily built to be a server-agnostic testing framework. It doesn't matter what framework or server-side implementation is used to create the front-end as long as it behaves as expected. Also, while you can (and when possible you probably should) write tests manually in Selenium, many tests are recorded macros of someone going through the motions that are then turned into code automatically.
On the other hand, django-webtest is built to work specifically on Django websites. It's actually a Django-specific extension to WebTest, which is not Django-only, but WSGI-only (and therefore Python-only). Because of that, it can interact with the application with a higher level of awareness of how things work on the server. This can make running tests faster and can also makes it easy to write more granular, detailed tests. Also, unlike Selenium, your tests can't be automatically written as recorded macros.
Otherwise, the two tools have generally the same purpose and are intended to test the same kinds of things. That said, I would suggest picking one rather than using both.
Related
Question - What approach or design pattern would make it easy in Django to use stubs for external integrations when running tests? With 'external integrations' I mean a couple of external REST APIs, and NAS file system. The external integrations are already separate modules/classes.
What I do now -
Currently, I disable external dependencies in tests mainly by sprinkling mock.path() statements across my test code.
But this is getting unpractical (needs to be applied per test; also easy to forget especially in more high-level tests), and links too much to the internals of certain modules.
Some details of what I am looking for
I like the concept of 'nullable infrastructure' described at
https://www.jamesshore.com/v2/blog/2018/testing-without-mocks#nullable-infrastructure.
I am especially looking for an approach that integrates well with Django, i.e. considering the settings.py file approach, and running tests via python manage.py test.
I would like to be able to easily:
state that all tests should use the nullable counterpart of an infrastructure class or function
override that behaviour per test, or test class, when needed (e.g. when testing the actual external infrastructure).
I tried the approach outlined in https://medium.com/analytics-vidhya/mocking-external-apis-in-django-4a2b1c9e3025, which basically says to create an interface implementation, a real implementation and a stub implementation. The switching is done using a django setting parameter and a class decorator on the interface class (which returns the chosen class, rather than the interface). But it isn't working out very well: the class decorator in my setup does not work with #override_settings (the decorator applies the settings upon starting of django, not when running the test), and there is really a lot of extra code (which also feels un-pythonic).
I couldn't immediately determine if this was possible by reading the source of the sdk.
But, is there a way to get a wsgi version of the application that dev_appserver would load from app.yaml?
I was so hoping there would be a function like
def app_from_yaml(path_to_yaml):
...
If this existed I could actually write automated tests for the blobstore logic and not have to do the crap manually anymore. Any Ideas?
I'm not aware of any solution that does what you want. I strongly suspect the reason is that dev_appserver does a lot of things when loading an application, including parsing the various yaml files, set up routing, stub out APIs (both, App Engine and Python), restrict the environment to emulate appserver, and so on. A function app_from_yaml(path_to_yaml) would have to do what dev_appserver.py does. And since dev_appserver.py does it already, I think no-one bothered to add another implementation.
I see 2 ways to solve your problem.
Make the blobstore API more testable
start your app with dev_appserver and run tests against it
The former one is rather difficult because it would require to refactor how things are done currently which risks the introduction of subtle regressions. The latter is something we do a lot for such large tests (which are really integration tests). We use gaedriver for that.
In your specific case where you want to test blobstore (which we do, too), we start an app from within a test, upload a blob using a certain URL and then hit another and check if the blob was processed correctly. It's not as nice as using testbed, but it works, is fairly straight-forward and fairly fast.
I'm not 100% sure of what you're asking, but the answer may lie in google.appengine.ext.webapp.util.run_wsgi_app.
In terms of the blobstore itself, there is already google.appengine.api.blobstore.blobstore_stub which can be used to test against the Blobstore (though I don't really understand what "manually" means in your context, so maybe that's not helpful).
So, I programmed this twisted application a few months ago, which I now would like to extend with a web-based user interface for configuration.
The Twisted website recommends Nevow, but I am not really sure if this is a good choice. Their website is down for a while it seems and their launchpad page hadn't seen any update in half a year. Is this project dead?
Additionally I have seen discussion of moving parts of Nevow into twisted.web on the twisted-web mailinglist. So, is it still recommended for new developments?
Another idea was using Django. I would need user authentication and permissions anyway in the config-interface, and I am quite familiar with it. (I have never worked with Nevow or twisted.web)
But it seems quite difficult to interface both worlds, all I could find were examples of running Django with WSGI in Twisted.
Are there any other possibilities to have a slick looking user interface on top of twisted?
First, let me address the perception that Nevow is dead. The launchpad project containing the code for Nevow (and the rest of the Divmod projects) is divmod.org on launchpad. A hardware failure has badly impacted the project's public presence, but it's still there, and other things (like the wiki and the tickets) are in the process of being recovered. There isn't a lot of active maintenance work going on right now, but that's mostly because it's good enough for most of its users; there are lots of people who depend on Nevow and would be very upset if it stopped working. Those people have the skills and experience necessary to continue maintaining it. So, while it's not being actively promoted right now, I think it's unlikely that it's going to go away.
My long-term hope for Nevow would be as follows. (I'd say "plan", but since I haven't been actively involved with its maintenance lately, this is really up to those who are.) First, I'd like to extract its templating facilities and move them into twisted.web. The clean, non-deprecated API for Nevow is mostly covered by nevow.page.Element and the various loaders. Twisted itself wants to generate HTML in a few places and these facilities could be useful. Then we should throw out the "appserver" and resource-model parts of Nevow. Those are mostly just a random collection of bugfixes or alterations for twisted.web, most of which were present in some form in twisted.web2 and will therefore either be rolled back into twisted.web anyway, or have already been applied there. Finally there's the question of Athena. While two-way communication is one of Twisted's strengths, Athena is itself a gigantic, sprawling JavaScript codebase and should probably remain its own project.
Third, on to the main question, given this information, what should you do now?
Generally speaking, I'd say, "use nevow". The project has some warts, it needs more documentation and its API needs to be trimmed to eliminate some old and broken stuff, but it's still quite useful and very much alive. To make up for the slightly sparse documentation, you can join the #divmod or #twisted.web channels on Freenode to get help with it. If you help out by contributing patches where you can, you will find that you'll get a lot of enthusiastic help there. When you ignore the deprecated parts Nevow has a pretty small, sane, twisted friendly API. The consequence of the plan for Nevow's evolution that I outlined above are actually pretty minimal. If it even happens at all, what it means for you is, in 1-5 years, when you upgrade to a new version of Twisted, you'll get a couple of deprecation warnings, change some import lines in your code from from nevow.page import ...; from nevow.loaders import ... to some hypothetical new thing like from twisted.web.page.element import ...; from twisted.web.page.templates import ..., or somesuch. Most of the API past that point should remain the same, and definitely the high-level concepts shouldn't change much.
The main advantage that you get from using Nevow is that it's async-friendly and can render pages in your main thread without blocking things. Plus, you can then get really easy COMET for free with Athena.
You can also use Django. This is not quite as async-friendly but it obviously does have a broader base of support. However, "not as async friendly" doesn't mean "hard to use". You can run it in twisted.web via WSGIResource, and simply use blockingCallFromThread in your Django application to invoke any Twisted API that returns a Deferred, which should be powerful enough to do just about anything you want. If you have a more specific question about how to instantiate Twisted web resources to combine Twisted Web and Django, you should probably ask it in its own Stack Overflow question.
Nevow is still a good choice if you want support for Deferreds in the templating system you use (it's not dead). It also has a few advantages over plain Twisted Web when it comes to complicated URL dispatch. However, it is basically just a templating system. Twisted Web is the real web server. So either way, you're going to use Twisted Web. In fact, even if you use Django in Twisted Web's WSGI container, you're still going to use Twisted Web. So learning things about Twisted Web isn't going to hurt you.
If you're going to be generating any amount of HTML, then you very much want to use an HTML templating library. By this point no one should be constructing HTML using primitive string operations. So if you want to use one of the other Python HTML templating libraries out there - Cheetah, Quixote, etc - instead of Nevow, that's great! You're just going to use the templating library to get a string to write out in response to an HTTP request. Twisted Web doesn't care where the string came from.
And if you do want to do something with Django (or another WSGI-based system), then you can certainly deploy this in your Twisted process using Twisted Web's WSGI support. And you can still interact between the WSGI applications and the rest of your Twisted code, as long as you exercise a little care - WSGI applications run in a thread pool, and Twisted APIs are not thread-safe, you have to invoke them with reactor.callFromThread or one of the small number of similar APIs (in particular, blockingCallFromThread is sometimes a useful higher-level tool to use).
At this point Nevow is definitively dead. As illustration of how dead it is, there is a bug that prevents installation of Nevow using pip, which was fixed on trunk in 2009, but it isn't in any release because there has been no release since then.
twisted.web and in particular twisted.web.template cover pretty much all of what was useful in Nevow, and should be used for any new project that was considering using Nevow.
I'm writing tests for a Django application that uses an external data source. Obviously, I'm using fake data to test all the inner workings of my class but I'd like to have a couple of tests for the actual fetcher as well. One of these will need to verify that the external source is still sending the data in the format my application expects, which will mean making the request to retrieve that information in the test.
Obviously, I don't want our CI to come down when there is a network problem or the data provider has a spot of downtime. In this case, I would like to throw a warning that skips the rest of that test method and doesn't contribute to an overall failure. This way, if the data arrives successfully I can check it for consistency (failing if there is a problem) but if the data cannot be fetched it logs a warning so I (or another developer) know to quickly check that the data source is ok.
Basically, I would like to test my external source without being dependant on it!
Django's test suite uses Python's unittest module (at least, that's how I'm using it) which looks useful, given that the documentation for it describes Skipping tests and expected failures. This feature is apparently 'new in version 2.7', which explains why I can't get it to work - I've checked the version of unittest I have installed on from the console and it appears to be 1.63!
I can't find a later version of unittest in pypi so I'm wondering where I can get hold of the unittest version described in that document and whether it will work with Django (1.2).
I'm obviously open to recommendations / discussion on whether or not this is the best approach to my problem :)
[EDIT - additional information / clarification]
As I said, I'm obviously mocking the dependancy and doing my tests on that. However, I would also like to be able to check that the external resource (which typically is going to be an API) still matches my expected format, without bringing down CI if there is a network problem or their server is temporarily down. I basically just want to check the consistency of the resource.
Consider the following case...
If you have written a Twitter application, you will have tests for all your application's methods and behaviours - these will use fake Twitter data. This gives you a complete, self-contained set of tests for your application. The problem is that this doesn't actually check that the application works because your application inherently depends on the consistency of Twitter's API. If Twitter were to change an API call (perhaps change the URL, the parameters or the response) the application would stop working even though the unit tests would still pass. (Or perhaps if they were to completely switch off basic authentication!)
My use case is simpler - I have a single xml resource that is used to import information. I have faked the resource and tested my import code but I would like to have a test that checks the format of that xml resource has not changed.
My question is about skipping tests in Django's test runner so I can throw a warning if the resource is unavailable without the tests failing, specifically getting a version of Python's unittest module that supports this behaviour. I've given this much background information to allow anyone with experience in this area to offer alternative suggestions.
Apologies for the lengthy question, I'm aware most people won't read this now.
I've 'bolded' the important bits to make it easier to read.
I created a separate answer since your edit invalidated my last answer.
I assume you're running on Python version 2.6 - I believe the changes that you're looking for in unittest are available in Python version 2.7. Since unittest is in the standard library, updating to Python 2.7 should make those changes available to you. Is that an option that will work for you?
One other thing that I might suggest is to maybe separate the "external source format verification" test(s) into a separate test suite and run that separately from the rest of your unit tests. That way your core unit tests are still fast and you don't have to worry about the external dependencies breaking your main test suites. If you're using Hudson, it should be fairly easy to create a separate job that will handle those tests for you. Just a suggestion.
The new features in unittest in 2.7 have been backported to 2.6 as unittest2. You can just pip install and substitute unittest2 for unittest and your tests will work as thyey did plus you get the new features without upgrading to 2.7.
What are you trying to test? The code in your Django application(s) or the dependency? Can you just Mock whatever that external dependency is? If you're just wanting to test your Django application, then I would say Mock the external dependency, so your tests are not dependent upon that external resource's availability.
If you can post some code of your "actual fetcher" maybe you will get some tips on how you could use mocks.
This is a follow-up to two questions I asked a week or so back. The upshot of those was that I was building a prototype of an AI-based application for the web, and I wondered what language(s) to use. The conclusion seemed to be that I should go for something like python and then convert any critical bits into something faster like Java or C/C++.
That sounds fine to me, but I'm wondering now whether python is really the right language to use for building a web application. Most web applications I've worked on in the past were C/C++ CGI and then php. The php I found much easier to work with as it made linking the user interface to the back-end so much easier, and also it made more logical sense to me.
I've not used python before, but what I'm basically wondering is how easy is CGI programming in python? Will I have to go back to the tedious way of doing it in C/C++ where you have to store HTML code in templates and have the CGI read them in and replace special codes with appropriate values or is it possible to have the templates be the code as with php?
I'm probably asking a deeply ignorant question here, for which I apologise, but hopefully someone will know what I'm getting at! My overall question is: is writing web applications in python a good idea, and is it as easy as it is with php?
Python is a good choice.
I would avoid the CGI model though - you'll pay a large penalty for the interpreter launch on each request. Most Python web frameworks support the WSGI standard and can be hooked up to servers in a myriad of ways, but most live in some sort of long-running process that the web server communicates with (via proxying, FastCGI, SCGI, etc).
Speaking of frameworks, the Python landscape is ripe with them. This is both good and bad. There are many fine options but it can be daunting to a newcomer.
If you are looking for something that comes prepackaged with web/DB/templating integration I'd suggest looking at Django, TurboGears or Pylons. If you want to have more control over the individual components, look at CherryPy, Colubrid or web.py.
As for whether or not it is as "easy as PHP", that is subjective. Usually it is encouraged to keep your templates and application logic separate in the Python web programming world, which can make your life easier. On the other hand, being able to write all of the code for a page in a PHP file is another definition of "easy".
Good luck.
"how easy is CGI programming in python?" Easier than C, that's for sure. Python is easier because -- simply -- it's an easier language to work with than C. First and foremost: no memory allocation-deallocation. Beyond that, the OO programming model is excellent.
Beyond the essential language simplicity, the Python WSGI standard is much easier to cope with than the CGI standard.
However, raw CGI is a huge pain when compared with the greatly simplified world of an all-Python framework (TurboGears, CherryPy, Django, whatever.)
The frameworks impose a lot of (necessary) structure. The out-of-the-box experience for a CGI programmer is that it's too much to learn. True. All new things are too much to learn. However, the value far exceeds the investment.
With Django, you're up and running within minutes. Seriously. django-admin.py startproject and you have something you can run almost immediately. You do have to design your URL's, write view functions and design page templates. All of which is work. But it's less work than CGI in C.
Django has a better architecture than PHP because the presentation templates are completely separated from the processing. This leads to some confusion (see Syntax error whenever I put python code inside a django template) when you want to use the free-and-unconstrained PHP style on the Django framework.
linking the user interface to the back-end
Python front-end (Django, for example) uses Python view functions. Those view functions can contain any Python code at all. That includes, if necessary, modules written in C and callable from Python.
That means you can compile a CLIPS module with a Python-friendly interface. It becomes something available to your Python code with the import statement.
Sometimes, however, that's ineffective because your Django pages are waiting for the CLIPS engine to finish. An alternative is to use something like a named pipe.
You have your CLIPS-based app, written entirely in C, reading from a named pipe. Your Django application, written entirely in Python, writes to that named pipe. Since you've got two independent processes, you'll max out all of your cores pretty quickly like this.
I would suggest Django, but given that you ask for something "as easy as it is with php" then you must take a look at PSP (Python Server Pages).
While Django is a complete framework for doing websites, PSP can be used in the same way than PHP, without any framework.
It is easier to write web-apps in python than it's in php. Particularly because python is not a broken language.
Pick up some web framework that supports mod_wsgi or roll out your own. WSGI apps are really easy to deploy after you get a hold from doing it.
If you want templates then genshi is about the best templating engine I've found for python and you can use it however you like.