Develop Reference

Can Locust run a script that interacts with a UI?

I'm very new to Locust and I'm having some problems. I have a python script that logs a user into our webpage and navigates to a certain point. We want to test the load of 1000s of users logging in the exact same way. I'm having trouble finding what I need in the Locust documentation, though.
Does anyone know if it's even possible to run a script that isn't a get/put action using Locust? If yes, is there a way to know the iteration that Locust is in and pass that number to the script? Ex: if it's just spawned user 234, pass 234 to the script so it can go get an id.
Does that make sense? Sorry if it's an obvious answer, I'm kind of wading through this on my own with no prior experience and I'm not really technical.
Thanks!

Read osx notification in Python

I need to launch an application on reading a Notification center notification on osx. Is there any python library for reading the notification center notifications?

Note: I did not try any of this as I do not have a machine running OSX at the moment, but from what I googled:
You could try adding an AppleScript snippet in your python code that uses System Events and Notification Center to parse all available notifications and return them to your stdout and parse the output directly in your code.
Source for getting all available notifications: https://macosxautomation.com/mavericks/notifications/01A.html. The part that should interest you is the code of the getNotificationTitles function. The bad part is that in that implementation you only get their titles, not the full notification, so you might have to parse different areas to get the full notification body.
Source for running it and getting the result: How do I embed an AppleScript in in a Python script?.
Another approach would be to use the terminal-notifier cli tool's -list argument (as a shell command inside your script) and then parse the results.
There's also a python wrapper for this tool available. The implementation contains a list() function which promises to work just the cli tool just that you wont need to spawn any shells from your own code and parse their stdouts.

I believe you will be able to do this with pyobjc. If you take a look at part of the code in macos-notifications repository, you'd find the userNotificationCenter_didDeliverNotification_ listener. This will get events the moment notifications are delivered. This does not only capture the notifications that you deliver using that application but for any notification. You could tweak the code a little and give it a try :)

Interact with python script running infinitive loop from web

I have a python script on my raspberry-pi continuously (every 5 seconds) running a loop to control the temperature of a pot with some electronics through GPIO.
I monitor temperature on a web page by having the python script write the temperature to a text file witch I request from java script and HTTP on a web page.
I would like to pass a parameter to the python script to make changes to the controlling, like change the target temperature.
What would be the better way to do this?
I'm working on a solution, where the python script is looking for parameters in a text file and then have a second python script write changes to this file. This second python script would be run by a http request from the web page.
Is this a way to go? Or am I missing a more direct way to do this.
This must be done many time before and described on the web, but I find nothing. Maybe I don't have the right terms to describe the problem.
Any hints is appreciated.
Best regards Kresten

You have to write somewhere your configuration for looping script. So file or database are possible choices but I would say that a formatted file (ini, yaml, …) is the way to go if you have a little number of parameters.

not sure about raspberry-pi but I see these solutions:
os signal (doc here
socket see Python socket server/client programming

subprocess.call does not wait for the process to complete

Per Python documentation, subprocess.call should be blocking and wait for the subprocess to complete. In this code I am trying to convert few xls files to a new format by calling Libreoffice on command line. I assumed that the call to subprocess call is blocking but seems like I need to add an artificial delay after each call otherwise I miss few files in the out directory.
what am I doing wrong? and why do I need the delay?
from subprocess import call
for i in range(0,len(sorted_files)):
args = ['libreoffice', '-headless', '-convert-to',
'xls', "%s/%s.xls" %(sorted_files[i]['filename'],sorted_files[i]['filename']), '-outdir', 'out']
call(args)
var = raw_input("Enter something: ") # if comment this line I dont get all the files in out directory
EDIT It might be hard to find the answer through the comments below. I used unoconv for document conversion which is blocking and easy to work with from an script.

It's possible likely that libreoffice is implemented as some sort of daemon/intermediary process. The "daemon" will (effectively1) parse the commandline and then farm the work off to some other process, possibly detaching them so that it can exit immediately. (based on the -invisible option in the documentation I suspect strongly that this is indeed the case you have).
If this is the case, then your subprocess.call does do what it is advertised to do -- It waits for the daemon to complete before moving on. However, it doesn't do what you want which is to wait for all of the work to be completed. The only option you have in that scenario is to look to see if the daemon has a -wait option or similar.
1It is likely that we don't have an actual daemon here, only something which behaves similarly. See comments by abernert

The problem is that the soffice command-line tool (which libreoffice is either just a link to, or a further wrapper around) is just a "controller" for the real program soffice.bin. It finds a running copy of soffice.bin and/or creates on, tells it to do some work, and then quits.
So, call is doing exactly the right thing: it waits for libreoffice to quit.
But you don't want to wait for libreoffice to quit, you want to wait for soffice.bin to finish doing the work that libreoffice asked it to do.
It looks like what you're trying to do isn't possible to do directly. But it's possible to do indirectly.
The docs say that headless mode:
… allows using the application without user interface.
This special mode can be used when the application is controlled by external clients via the API.
In other words, the app doesn't quit after running some UNO strings/doing some conversions/whatever else you specify on the command line, it sits around waiting for more UNO commands from outside, while the launcher just runs as soon as it sends the appropriate commands to the app.
You probably have to use that above-mentioned external control API (UNO) directly.
See Scripting LibreOffice for the basics (although there's more info there about internal scripting than external), and the API documentation for details and examples.
But there may be an even simpler answer: unoconv is a simple command-line tool written using the UNO API that does exactly what you want. It starts up LibreOffice if necessary, sends it some commands, waits for the results, and then quits. So if you just use unoconv instead of libreoffice, call is all you need.
Also notice that unoconv is written in Python, and is designed to be used as a module. If you just import it, you can write your own (simpler, and use-case-specific) code to replace the "Main entrance" code, and not use subprocess at all. (Or, of course, you can tear apart the module and use the relevant code yourself, or just use it as a very nice piece of sample code for using UNO from Python.)
Also, the unoconv page linked above lists a variety of other similar tools, some that work via UNO and some that don't, so if it doesn't work for you, try the others.
If nothing else works, you could consider, e.g., creating a sentinel file and using a filesystem watch, so at least you'll be able to detect exactly when it's finished its work, instead of having to guess at a timeout. But that's a real last-ditch workaround that you shouldn't even consider until eliminating all of the other options.

If libreoffice is being using an intermediary (daemon) as mentioned by #mgilson, then one solution is to find out what program it's invoking, and then directly invoke it yourself.

What is the best way of running shell commands from a web based interface?

Imagine a web application that allows a logged in user to run a shell command on the web server at the press of a button. This is relatively simple in most languages via some standard library os tools.
But if that command is long running you don't want your UI to hang. Again this is relatively easy to deal with using some sort of background process or putting the command to be executed onto a message queue (and maybe saving the output and status somewhere for later consumption). Just return quickly saving we'll run that and get back to you.
What I'd like to do is show the output of said web ui triggered shell command as it happens. So vertically scrolling text like when running in a terminal.
I have a vague idea of how I might approach this, streaming the output to a websocket perhaps and simply printing the output to screen.
What I'd like to ask is:
Are their any plugins, libraries or applications that already do this. Something I can either use or read the source of. Ideally an open source python/django or ruby/rails tool, but other stacks would be interesting too.

I'm not sure if it's what you want, but there are some web based ssh clients out there. If you care about security and really just want dynamic feedback, you could look into comet or just have a frame with its own http session that doesn't end until it's done printing.

web-based ssh client would work, into the host (there are java ssh clients out there).
Ruby has a web-based terminal:
http://tryruby.org (link to the source is at the bottom of the page).
You could also embed Ruby via jruby: http://tim.lossen.de/2007/03/jruby/applet.html
http://github.com/jruby/jruby/blob/master/samples/irb-applet.html

I haven't heard of any libraries that do this, but you'll need to setup the system command and call out to the system. You will then need to "pump" the sysout and syserr standard inputs and pipe that data back out to your web client.
As an example for this style of problem, look into code snippits of how people use ruby/python/etc to transcode a video, i.e. http://kpumuk.info/ruby-on-rails/encoding-media-files-in-ruby-using-ffmpeg-mencoder-with-progress-tracking/ - my example was taken from this blog post.
class MediaFormatException < StandardError
end
def execute_mencoder(command)
progress = nil
IO.popen(command) do |pipe|
pipe.each("r") do |line|
if line =~ /Pos:[^(]*(s*(d+)%)/
p = $1.to_i
p = 100 if p > 100
if progress != p
progress = p
print "PROGRESS: #{progress}n"
$defout.flush
end
end
end
end
raise MediaFormatException if $?.exitstatus != 0
end
I don't know if this example is pulling data from both sysout and syserr, but you will definitely need to be pulling data from both of those interfaces, typically if the buffer fills up, the executing command might hang or fail (I have experienced this with Python). This method will also look different if the only thing you do is return line to the web client - in a terminal, the progress indicator of ffmpeg/mencoder remains stationary on the bottom line, but this method will give you a long list of progress indicator updates. Pipe line out to your terminal and you'll see what I'm referring to.

So, I've tried to answer my own question with code as I couldn't find anything to quite fit the bill. Hopefully it's useful to anyone coming across the same problem.
Redbeard 0X0A pointed me in the general direction, I was able to get a stand along ruby script doing what I wanted using popen. Extending this to using EventMachine (as it provided a convenient way of writing a websocket server) and using it's inbuilt popen method solved my problem.
More details here http://morethanseven.net/2010/09/09/Script-running-web-interface-with-websockets.html and the code at http://github.com/garethr/bolt/

Certainly not the best way to run shell commands, but likely the easiest:
#!/bin/sh
echo Content-Type: text/plain
echo
/usr/bin/uptime
http://www.sente.cc/scripts/uptime.cgi

Take a look at Galaxy (online demo) or Yabi.
Except from the requirement to be able to show output during the job run, they are both excellent solutions to this! They are also both written i Python (and Yabi even on django).
They were both built with bioinformatics in mind, but really are both general job runner/workflow tools.
They will let you specify parameters in a web interface, see queued/running/finished jobs in a separate column, and after the jobs are finished, inspect details and results, or re-run the job, with possibly changed parameters.
Galaxy is the easier one to install. The Galaxy installation boils down to downloading and run "sh run.sh"), and adding your own tool boils down to creating an XML file in the line of:
<tool id="mytool" name="My Tool" version="1.0.0">
<description>Does this and that</description>
<command>somecommand --aparam $aparam</command>
<inputs>
<param name="aparam" type="text" label="A parameter"/>
</inputs>
<outputs>
<data name="outfile" format="tabular"/>
</outputs>
</tool>
... and place it in the /tools folder, and add a line in the tool_conf.xml to tell galaxy of your new tool (There you can also get rid of the bioinformatics-tools, so they don't mess up your tools menu).
Yabi is more complicated to install (see the readme file), but the process might be smooth if you are on the right kind of system. On the other hand, it allows you even do the tool configuration in the web interface, rather than as an XML file like in Galaxy.
Galaxy still is the one with the biggest community though, which is reflected in the number of features/already integrated tools (See the toolshed for shared tools/wrapper).

websocketd looks like the perfect tool for that.