How does the GUI testing tool PyUseCase compare to Dogtail? - python

How does the GUI testing tool PyUseCase renamed to StoryText. compare to Dogtail?
I want to hear from people who have hopefully experience in using both.
Interested in:
Maintainabilty of the testing code
How well they work against real GUI's?

Firstly: I'm the author of PyUseCase and I haven't done more than play around with Dogtail...
The tools are different in a number of respects.
Dogtail works via the accessibility interface under Gnome on Linux, while PyUseCase operates via GUI toolkits (PyGTK, Tkinter, SWT/Eclipse in the current release, plus Swing from the soon-forthcoming new release)
PyUseCase tries very hard to be usable by non-programmers. UI actions are defined in a user-defined domain language, assertions are replaced by generating and comparing plain text descriptions. It also contains a recorder.
Dogtail is a more traditional "write Python code, call APIs, assert things about what you get back" paradigm, likely a more familiar way to do things if you're used to programming and unit testing.
PyUseCase tries hard to make it very easy to change the tests en masse when the GUI changes. The testing code is super-maintainable, because there isn't any :) You get a "UI map file" instead which is just definitions.
Lastly, I'm not sure how active Dogtail is. Last time I looked it seems the last commit was in 2009, but appearances can deceive... If you want something like Dogtail, I'd suggest taking a look at the Linux Desktop Testing Project (LDTP) which is a pretty similar concept but seems a good bit more active.
PyUseCase is in any case active, to the tune of two of us working on it full time. It works pretty well on our real GUIs, but its maturity varies between the different toolkits.

Related

Python determine screenreader is enabled

I'm running a Python program with an interactive terminal output/input menu.
However, it doesn't quite work for screenreaders at the moment (it's using termios).
Is there a way to determine if a screenreader is enabled/running on a Linux based system via python?
No, there is not. More than that, Apple once tried to add such a capability, but it was considered as discriminatory as it would lead to interface differences between "accessible" and "normal" versions. I agree with this point of view, because I saw quite a bunch of websites having "special" versions for visually impaired which, of course, lacked much of the functionality of "normal" versions.
So, your (and everyone's) solution is to build accessible interfaces, regardless of the fact whether your user is using a screen reader at the moment or not.

how can ensure that a Python qt application is screen readable on all operating systems

I am developing a text with Python and Qt that can be used by blind and visually impaired people.
The text editor should be suitable for screen readers.
I do tests with the screen readers NVDA (Windows) and
Orca (Ubuntu).
Texts for screen readers should actually be saved in the "AccessibleDescription" property. The NVDA screen reader does not read the "AccessibleDescription" property. AccessibleName property only. The Orca screen reader reads the "AccessibleDescription" property.
Question:
What can I do so that my application is screen-readable on all operating systems?
Short answer
Sadly, there's probably no magic solution here.
The only reasonnable answer is just trying and testing a maximum of different combinations of OS, screen readers, GUI toolkits and components, as much as you can. That's the only way to discover what works well for your particular scope, and what doesn't.
There is no uniformity at all between OS, screen readers and GUI toolkits, and probably no combination working everywhere in all cases.
Longer answer
This isn't a peasure to give this answer but in fact, if you want to make sure your app is 100% screen reader accessible, your only way is basicly to stop using GUI toolkits and exclusively use the native components offered by the OS for which accessibility is well documented; and therefore of course develop as many different apps as OS you intent to support.
As soon as you are using a GUI toolkit, you are adding a layer, and may lose some accessibility (and often more than less) in the process.
The gap between the features offered by the native OS components and those proposed in the toolkit is often closed by emulation, and some of the emulations aren't made accessible, or simply can't be made accessible for various (more or less excusable) reasons.
That's why some components are accessible while some others aren't; why a given component can be accessible under one OS but not at all under another; and why a component can be accessible in version N of the toolkit but not the same component in version N+1.
In short, you have understood, accessibility is extremely fragile.
Sadly, GUI toolkits rarely well document accessibility of their components under the different OS they support.
That's really a shame. It has to be addressed.
Additionally, accessibility is often presented as being expert compicated features. It shouldn't be so. It should be easy for app developers to make accessible apps.
So, wwell, since there is often no documentation about accessibility, your only way is to try and test.
That's a lot of work, and there is no alternative. Try and test. That's all.
Digging deeper
There's generally two foundamental ways to design a GUI toolkit, and depending on which has been chosen, this has a dramatic impact on natural accessibility, and what kind of accessibility the toolkit can hope to achieve.
The first way is to start from a blank page as far as the OS permits, and from there, decide literally everything: what components to provide, and how they should look and behave.
By doing so, you make sure that your app will look and behave exactly the same everywhere. An user might be surprised or confused the first time because look and behavior is often not 100% conform to OS conventions or user preferences, but in the long run, he/she will always feel at home with your app regardless of the OS used.
However, more importantly, create everything from a blank page also means that everything must be defined from scratch for accessibility. It requires of course a lot of work, whether for the GUI implementers, or the app developer. Different toolkits place the responsibility cursor at different places.
Some toolkits perform better and/or go further than others, but in general they are nowhere near what the native components of the OS can provide.
The second way is to use as much as possible the components proposed by the OS, and emulate only missing features when they are really needed.
This is excellent for accessibility, since almost all native OS components are naturally accessible or only require little effort. Users will have an app that conforms to the general look of their OS, and so won't be confused or surprised by the interface and its behavior.
However, the app isn't going to look exactly the same in all computers. Sadly, for many designers and developers, this is a definitive stopper.
As the vaste majority of GUI toolkits I know of, QT is part of the first group, but isn't playing that bad in accessibility (at least under windows) if the app developer is also aware of accessibility.
In the second group, you have WXWidgets for example.

What programming language features are well suited for developing a live coding framework?

I would like to build a "live coding framework".
I should explain what is meant by "live coding framework". I'll do so by comparing live coding to traditional coding.
Generally put, in traditional programming you write code, sometimes compile it, then launch an executable or open a script in some sort of interpreter. If you want to modify your application you must repeat this process. A live coding framework enables code to be updated while the application is running and reloaded on demand. Perhaps this reloading happens each time a file containing code is changed or by some other action. Changes in the code are then reflected in the application as it is running. There is no need to close the program and to recompile and relaunch it.
In this case, the application is a windowed app that has an update/draw loop, is most likely using OpenGL for graphics, an audio library for sound processing ( SuperCollider? ) and ideally a networking lib.
Of course I have preferred languages, though I'm not certain that any of them would be well suited for this kind of architecture. Ideally I would use Python, Lua, Ruby or another higher level language. However, a friend recently suggested Clojure as a possibility, so I am considering it as well.
I would like to know not only what languages would be suitable for this kind of framework but, generally, what language features would make a framework such as this possible.
I have implemented a live coding feature in Lua as part of the ZeroBrane Studio IDE. It works exactly as you described by reloading the application when a change in the code is made. I'm working on possible improvements to modify values at run-time to avoid full reload of the application. It's a pure Lua-based solution and doesn't require any modifications to the VM.
You can see the demo of the live coding as currently implemented here: http://notebook.kulchenko.com/zerobrane/live-coding-in-lua-bret-victor-style.
In terms of language features used/required, I rely on:
the ability to interrupt/resume a running application (this is based on debug.hook and error() calls),
the ability to interact with the (unmodified) application remotely (this is done based on debug.hook, TCP interactions with select() support to detect if a new request is being sent from the host machine, as well and on coroutines to switch between the main application and the live coding module), and
the ability to inject new code into the application (this mechanism is also using co-routines, but I'm sure there are alternatives). There is also a possibility to inject just a modified fragment, but it need to be at the level of a function, and if this function is a local to some other function, you need to include that too and so on.
Clojure has pretty much everything you are likely to want as a live coding language. Main highlights:
Interactive REPL - so you can interact directly with your running program. Even when I'm doing "traditional programming" I tend to write code interactively and copy the bits I like into a source file later. Clojure is just designed to work this way - pretty much everything in your program is inspectable, modifiable and replaceable at runtime.
Great concurrency support - you can kick off concurrent background tasks trivially with code like (future (some-function)). More importantly, Clojure's STM and emphasis on high performance immutable data structures will take care of the more subtle concurrency aspects (e.g. what happens if I update a live data structure while it is in the middle of being rendered??)
Library availability - it's a JVM language so you can pull in all the audio, visual, IO or computational tools you require from the Java ecosystem. It's easy to wrap these in a line or two of Clojure so that you get a concise interface to the functions that you need
Macros - as Clojure is a homoiconic language you can take advantage of the Lisp ability to write powerful macros that extend the language. You can effectively build the exact syntax that you want to use in the live environment, and let the compiler do all the hard work of creating the complete code behind the scenes.
Dynamic typing - the benefits of this can be argued both ways, but it's certainly a huge benefit when trying to write code quickly and concisely.
Active community with a lot of cool projects - you're likely to find a lot of people interested in similar live coding techniques in the Clojure community.
A couple of links you might find interesting:
Paul Graham on Lisp - beating the averages
Live Clojure coding example with the Overtone sound synthesizer (a frontend to SuperCollider)
The only thing that’s necessary to make this work is a form of dynamic binding, e.g., message passing in Erlang or eval in many other languages.
If you have dynamic binding, then you can change the target of a message without affecting the message, or a message without affecting the target—provided that a target is defined when you try to send a message to it, and a message is defined for the targets to which you send it, when you send it.
When changing a target, all you have to do is serve messages to the previous version until the new version is in place, then do a small locked update to transition to the new version. Similarly, when changing a message, you just serve the old version till the new one is available.
Readily hot-swappable code must still be designed as such, however—the application must be modular enough that replacing the implementation of a component does not cause an interruption, and that can only come from careful programming.
It's well and good to have 'live coding' on your dev box, but a way to directly interact with a deployed server takes it a lot closer to being 'real'. For this you need a network aware REPL.
clojure provides this nicely in the form of a socket repl. This allows you to remotely attach to the running version of your code on your deployed tomcat server (for instance). You can then attach your favorite swank-enabled development tool and hack away.
Smalltalk is probably the best bet for this. As unlike the others, it has a whole IDE for live coding, not just a REPL
Tcl has such a thing already. For example, you can write a gui program that creates a separate window that has an interactive prompt. From there you can reload your code, type in new code, etc.
You can do this with any gui toolkit, though some will be much harder than others. It should be easy with python, though the indentation thing -- IMHO -- makes interactive use challenging. I'm reasonably certain most other dynamic languages can do this without too much trouble.
Look at it this way: if your toolkit lets you open more than one window, there's no reason why one of those windows can't be an interactive prompt. All you need is the ability to open a window, and some sort of "eval" command that runs code fed to it as a string.
python on google appengine has repote_api_shell.py. this is not a full live-coding suite - clojure on emacs w/ swank-clojure has had much more real-life use as far as integrating 'livecoding' into everyday development rhythms - but a lot of people don't realize this is possible in certain python environments.
$ PYTHONPATH=. remote_api_shell.py -s dustin-getz.appspot.com
App Engine remote_api shell
Python 2.7.1 (r271:86832, Jun 16 2011, 16:59:05)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)]
The db, users, urlfetch, and memcache modules are imported.
dustin-getz> import models
dustin-getz> models.BlogPost(title='a modern take on automated testing', link='https://docs.google.com/document/pub?id=1DUxQogBg45rOTK4c5_SfEHiQcvL5c207Ivcy-gDNx2s', dont_publish_feed=False).put()
dustin-getz> items = models.BlogPost.all().filter('dont_publish_feed =', False).order('-published_date').fetch(100)
dustin-getz> len(items)
58
dustin-getz> for item in items[:5]: print item.title
a modern take on automated testing
Notes: Running a startup on haskell
the [un]necessity of superstar middle management in bigcos
"everything priced above its proper value"
stages of growth as a software engineer
I'm working on a live coding feature for PyDev's Python editor. It was inspired by Bret Victor's Inventing on Principle talk, and I've implemented a program state display as well as turtle graphics. They both update as you type your Python code in Eclipse.
The project is hosted on GitHub, and I've posted a demo video, as well as a tutorial.
The main features of Python that I used were abstract syntax trees and dynamic code execution. I take the user's code, parse it into a tree, then instrument any assignment statements, loop iterations, and function calls. Once I've instrumented the tree, I execute it and display the report or draw the requested turtle graphics.
I haven't implemented the swapping feature that other answers discuss. Instead, I always run the code to completion or a time out. I envision live coding as an enhancement to test-driven development, not as a way to hack on a live application. However, I will think more about what swapping out pieces of a live application would let me do.

Looking for advice on how to develop applets for Gnome / Ubuntu

I am a linux (mostly ubuntu) user with a reasonable understanding of how the system works (although I am certainly not a linux guru!). In the past I have developed small cross-platform desktop applications in python/GTK and I delivered them to clients as self-contained filetrees, so that the only dependencies were Python itself and GTK.
Now I would like to develop a small applet for ubuntu, that I would like to release under GPL 2 or 3.
In particular these are the new steps I know I must learn in order to achieve my goal (it is very possible there are a few more that I am unaware of, though!):
Integrating with gnome: I want my application to be available as an applet in the taskbar.
Using D-bus: In particular I want my applet to use the new osd-notification framework for ubuntu, but communication with other applets is also a possible feature for a second iteration.
Packaging: I would like to setup a public PPA as soon as the application will reach alpha stage, but I also would like to use dependencies from existing packages in the official repos, rather than include the libraries again in my own package.
Of course official documentation will be my first source of knowledge, but - basing my judgment on the very useful answers that I received on another topic here on SO - I decided to turn to the SO community to collect additional advice like for example:
Are there additional steps to those I outlined before, that I have to learn in order to be able to implement my project?
Based on your own experience, would you advise me to learn those steps in advance (as the knowledge of those will influence my way of coding the core functionality) or would you consider integration with gnome / d-bus and packaging as "higher encapsulating levels" that can be added on top of core functionality afterwards (note: D-bus will be used at first just for pushing data. Input data will be retrieved with a webservice)?
Would you advise me to separate my application in two packages (back-end and front-end) or to keep it together in a single package?,
Do you know of any useful resource that you would advise me to look at, for learning any of the things that I have to?
Are you aware of any common "beginner's mistakes" that I should be aware of?
These questions are not meant to be exhaustive, though: if you feel that I am missing something from the general picture, you are more than welcomed to point me in the right direction!
PS: Should I have failed in explaining my final goal, take a look at project hamster: what I want to achieve is similar in terms of user interface (meaning: the applet should display the status and clicking on it should open the application itself, from which you could both configure the applet and perform various operations).
Well, you list python, so you'll want to have pynotify in your arsenal. It wraps DBus, and gives you a direct api for manipulating the osd-notification system.
>>> import pynotify
>>> pynotify.init("Lil' Applet")
True
>>> note = pynotify.Notification(
... pynotify.get_app_name(),
... "Lil' Applet wants you to know something's up.",
... "/usr/share/icons/Human/48x48/status/dialog-information.png")
>>> note.show()
True
This displays a notification that looks like this:
[ ] **Lil' Applet**
[ICON]
[ ] Lil' Applet wants you to know something's up.
As you already know, your first and best friend will be the code written by others - copy, paste, dissect, understand.
Luckily there are a few projects that do what you intend to achieve.
I can recommend conduit's code as a prime reference how to do things in a clean fashion. I think they also have stuff on dbus. Others to keep an eye on, would be deskbar-applet, hamster (heh), and any other app you remember having feature X. Sometimes it might require some C code deciphering though (like the applet button bit - i suggest you better take it from hamster as i was having some major time getting the thing straight)
Then the "devhelp" app will be of great assistance - it allows you to read and search in man pages fast and easy. Make sure that you also have the -doc packages for all the modules you intend to use.
For user interface i strongly suggest using glade, as that will allow you to change interface later much easier. Where you can't use glade - add and alignment box and add the widget in the box in the code.
There certainly will be quirks and things that you will learn the hard way. Should not be too hard though!
The packaging, especially the autotools will be bit of a struggle, but you will get it right. For how to do debians (and from there to PPA), you can dig in the hamster's repository history. There was once a "debian" folder.
I would suggest to start small - see if you can get a window. Then put a button on it.
You don't have to do it "right" the first time. For first time it will be ok, if something works at all.
As for the separation - i would not bother about it until you get there. Splitting up into two parts and have a core, should not be too hard later. But that all depends on your priorities.
Last thing - getting friends who know the field helps too. And one way to get new friends, is by taking part in other projects, heh.
There are some very good recommendations here already, but let me suggest that you develop your applet not so much "for Ubuntu" as "for Gnome". It doesn't take much extra effort to also make RPM packages for distributions such as Fedora, and Arch Linux packages, to name two examples. There is one major disadvantage though -- to stay compatible with Debian stable you have to stick to ancient versions of GTK and GLib, or at least make any functionality depending on newer versions optional. It's painful, but apparently Debian stable users appreciate it.
I'd also suggest setting up a source code management system somewhere as early as possible. You may not be worried about your disk crashing, but sometimes it saves you a lot of trouble just to be able to revert everything you did since the last commit.
Here's the link to the documentation on the official Gnome Panel Applet library. I don't know if it has Python bindings or not.
When I asked this question two years ago Ubuntu and Gnome were much closer to each other than they are today. At present (end of 2011) gnome adopted the gnome-shell, while Canonical decided to develop their very own UI (unity)...
Part of the tension that brought to the split specifically involved libappindicator, which makes the way I formulated this question (and probably part of the answers) obsolete.
Besides, there is now AskUbuntu on stack exchange, that would probably be a much better forum to ask about ubuntu-specific question.

Debugging a scripting language like ruby

I am basically from the world of C language programming, now delving into the world of scripting languages like Ruby and Python.
I am wondering how to do debugging.
At present the steps I follow is,
I complete a large script,
Comment everything but the portion I
want to check
Execute the script
Though it works, I am not able to debug like how I would do in, say, a VC++ environment or something like that.
My question is, is there any better way of debugging?
Note: I guess it may be a repeated question, if so, please point me to the answer.
Your sequence seems entirely backwards to me. Here's how I do it:
I write a test for the functionality I want.
I start writing the script, executing bits and verifying test results.
I review what I'd done to document and publish.
Specifically, I execute before I complete. It's way too late by then.
There are debuggers, of course, but with good tests and good design, I've almost never needed one.
Here's a screencast on ruby debugging with ruby-debug.
Seems like the problem here is that your environment (Visual Studio) doesn't support these languages, not that these languages don't support debuggers in general.
Perl, Python, and Ruby all have fully-featured debuggers; you can find other IDEs that help you, too. For Ruby, there's RubyMine; for Perl, there's Komodo. And that's just off the top of my head.
There is a nice gentle introduction to the Python debugger here
If you're working with Python then you can find a list of debugging tools here to which I just want to add Eclipse with the Pydev extension, which makes working with breakpoints etc. also very simple.
My question is, is there any better way of debugging?"
Yes.
Your approach, "1. I complete a large script, 2. Comment everything but the portion I want to check, 3. Execute the script" is not really the best way to write any software in any language (sorry, but that's the truth.)
Do not write a large anything. Ever.
Do this.
Decompose your problem into classes of objects.
For each class, write the class by
2a. Outline the class, focus on the external interface, not the implementation details.
2b. Write tests to prove that interface works.
2c. Run the tests. They'll fail, since you only outlined the class.
2d. Fix the class until it passes the test.
2e. At some points, you'll realize your class designs aren't optimal. Refactor your design, assuring your tests still pass.
Now, write your final script. It should be short. All the classes have already been tested.
3a. Outline the script. Indeed, you can usually write the script.
3b. Write some test cases that prove the script works.
3c. Runt the tests. They may pass. You're done.
3d. If the tests don't pass, fix things until they do.
Write many small things. It works out much better in the long run that writing a large thing and commenting parts of it out.
Script languages have no differences compared with other languages in the sense that you still have to break your problems into manageable pieces -- that is, functions. So, instead of testing the whole script after finishing the whole script, I prefer to test those small functions before integrating them. TDD always helps.
There's a SO question on Ruby IDEs here - and searching for "ruby IDE" offers more.
I complete a large script
That's what caught my eye: "complete", to me, means "done", "finished", "released". Whether or not you write tests before writing the functions that pass them, or whether or not you write tests at all (and I recommend that you do) you should not be writing code that can't be run (which is a test in itself) until it's become large. Ruby and Python offer a multitude of ways to write small, individually-testable (or executable) pieces of code, so that you don't have to wait for (?) days before you can run the thing.
I'm building a (Ruby) database translation/transformation script at the moment - it's up to about 1000 lines and still not done. I seldom go more than 5 minutes without running it, or at least running the part on which I'm working. When it breaks (I'm not perfect, it breaks a lot ;-p) I know where the problem must be - in the code I wrote in the last 5 minutes. Progress is pretty fast.
I'm not asserting that IDEs/debuggers have no place: some problems don't surface until a large body of code is released: it can be really useful on occasion to drop the whole thing into a debugging environment to find out what is going on. When third-party libraries and frameworks are involved it can be extremely useful to debug into their code to locate problems (which are usually - but not always - related to faulty understanding of the library function).
You can debug your Python scripts using the included pdb module. If you want a visual debugger, you can download winpdb - don't be put off by that "win" prefix, winpdb is cross-platform.
The debugging method you described is perfect for a static language like C++, but given that the language is so different, the coding methods are similarly different. One of the big very important things in a dynamic language such as Python or Ruby is the interactive toplevel (what you get by typing, say python on the command line). This means that running a part of your program is very easy.
Even if you've written a large program before testing (which is a bad idea), it is hopefully separated into many functions. So, open up your interactive toplevel, do an import thing (for whatever thing happens to be) and then you can easily start testing your functions one by one, just calling them on the toplevel.
Of course, for a more mature project, you probably want to write out an actual test suite, and most languages have a method to do that (in Python, this is doctest and nose, don't know about other languages). At first, though, when you're writing something not particularly formal, just remember a few simple rules of debugging dynamic languages:
Start small. Don't write large programs and test them. Test each function as you write it, at least cursorily.
Use the toplevel. Running small pieces of code in a language like Python is extremely lightweight: fire up the toplevel and run it. Compare with writing a complete program and the compile-running it in, say, C++. Use that fact that you can quickly change the correctness of any function.
Debuggers are handy. But often, so are print statements. If you're only running a single function, debugging with print statements isn't that inconvenient, and also frees you from dragging along an IDE.
There's a lot of good advice here, i recommend going through some best practices:
http://github.com/edgecase/ruby_koans
http://blog.rubybestpractices.com/
http://on-ruby.blogspot.com/2009/01/ruby-best-practices-mini-interview-2.html
(and read Greg Brown's book, it's superb)
You talk about large scripts. A lot of my workflow is working out logic in irb or the python shell, then capturing them into a cascade of small, single-task focused methods, with appropriate tests (not 100% coverage, more focus on edge and corner cases).
http://binstock.blogspot.com/2008/04/perfecting-oos-small-classes-and-short.html

Categories

Resources