Develop Reference

How to reuse completions from PathCompleter in prompt_toolkit

I am creating a REPL tool for my project that (simplified for clarity) either directly executes entered commands or (if a command ".x some/path/to/file" is entered) reads and executes them from file. My question is related to auto-completing the user input (using prompt_toolkit).
I have something like (minimum executable example):
import prompt_toolkit
from prompt_toolkit.completion import Completer, Completion
from prompt_toolkit.document import Document
from prompt_toolkit.contrib.completers import PathCompleter
class CommandCompleter(Completer):
def __init__(self):
self.path_completer = PathCompleter()
self.commands = [".x", "command1", "command2"]
def get_completions(self, document, complete_event):
if document.text.startswith(".x "):
sub_doc = Document(document.text[3:])
yield from (Completion(cmd.text, -document.cursor_position)
# ???????? ?????????????????????????
for cmd
in self.path_completer.get_completions(sub_doc, complete_event))
# ???????
else:
yield from (Completion(cmd, -document.cursor_position)
for cmd in self.commands
if cmd.startswith(document.text))
if __name__ == "__main__":
while True:
other_args = {}
input = prompt_toolkit.prompt(">>> ", completer=CommandCompleter(), **other_args)
# Do something with input (omitted)
The second if-branch (for commands) works correctly but I don't know how to properly call the PathCompleter.get_completions() method and reconstruct the Completion objects from its result (where the ???'s are) in the first branch. The trick is that I am using the completion only for a part of the input and various sub-stringing, position calculations etc. did not (yet) lead to the satisfactory behaviour (i.e. offering the paths and constructing the correct input line).
I will definitely go on searching but if anyone knows how to rewrite this, it would be very useful.
Note: yield from self.path_completer.get_completions(document, complete_event) would be used if the whole input would be just the path (and this works correctly).

Probably the following should fix it:
sub_doc = Document(document.text[3:])
yield from (Completion(completion.text, completion.start_position, display=completion.display)
for completion
in self.path_completer.get_completions(sub_doc, complete_event))
completion.text contains the text that is going to be inserted;
completion.start_position contains the place where the text is going to be inserted, relative to the cursor position (in this particular example we can take the value from the nested completer).
completion.display is the value displayed in the pop-up menu. (In this case, the whole filename, rather than only the inserted string.
Feel free to open a GitHub issue if you have any more questions.

Python network bandwidth monitor

I am developing a program in python, and one element tells the user how much bandwidth they have used since the program has opened (not just within the program, but regular web browsing while the program has been opened). The output should be displayed in GTK
Is there anything in existence, if not can you point me in the right direction. It seems like i would have to edit an existing proxy script like pythonproxy, but i can't see how i would use it.
Thanks,

For my task I wrote very simple solution using psutil:
import time
import psutil
def main():
old_value = 0
while True:
new_value = psutil.net_io_counters().bytes_sent + psutil.net_io_counters().bytes_recv
if old_value:
send_stat(new_value - old_value)
old_value = new_value
time.sleep(1)
def convert_to_gbit(value):
return value/1024./1024./1024.*8
def send_stat(value):
print ("%0.3f" % convert_to_gbit(value))
main()

import time
def get_bytes(t, iface='wlan0'):
with open('/sys/class/net/' + iface + '/statistics/' + t + '_bytes', 'r') as f:
data = f.read();
return int(data)
while(True):
tx1 = get_bytes('tx')
rx1 = get_bytes('rx')
time.sleep(1)
tx2 = get_bytes('tx')
rx2 = get_bytes('rx')
tx_speed = round((tx2 - tx1)/1000000.0, 4)
rx_speed = round((rx2 - rx1)/1000000.0, 4)
print("TX: %fMbps RX: %fMbps") % (tx_speed, rx_speed)
should be work

Well, not quiet sure if there is something in existence (written in python) but you may want to have a look at the following.
Bandwidth Monitoring (Not really an active project but may give you an idea).
Munin Monitoring (A pearl based Network Monitoring Project)
ntop (written in C/C++, based on libpcap)
Also just to give you pointers if you are looking to do something on your own, one way could be to count and store packets using sudo cat /proc/net/dev

A proxy would only cover network applications that were configured to use it. You could set, e.g. a web browser to use a proxy, but what happens when your proxy exits?
I think the best thing to do is to hook in lower down the stack. There is a program that does this already, iftop. http://en.wikipedia.org/wiki/Iftop
You could start by reading the source code of iftop, perhaps wrap that into a Python C extension. Or rewrite iftop to log data to disk and read it from Python.

Would something like WireShark (https://wiki.wireshark.org/FrontPage) do the trick? I am tackling a similar problem now, and am inclined to use pyshark, a WireShark/TShark wrapper, for the task. That way you can get capture file info readily.

Modules or functions to obtain information from a network interface in Python

I wrote a little application that I use from the terminal in Linux to keep track of the amount of data up and down that I consume in a session of Internet connection (I store the info in MongoDB). The data up and down I write by hand and read them (visually) from the monitor system, the fact is that I would like to automate more my application and make it read data consumed up and down from the interface network i use to connect to internet (in my case ppp0), but the detail is in that I does not find the way to do in Python. I guess Python have a module to import or something that lets me do what I want, but until now I have researched I have not found a way to do it.
Do you know of any module, function or similar that allows me to do in python what I want?
any example?
thanks in advance

Well I answer myself
Found in the community PyAr this recipe to me me like a glove going to do what we wanted without having to use extra commands or other applications.
Slightly modifying the code to better suit my application and add a function that comvierta of bytes to Megabytes leave it like this:
def bytestomb(b):
mb = float(b) / (1024*1024)
return mb
def bytessubidatransferidos():
interface= 'ppp0'
for line in open('/proc/net/dev', 'r'):
if interface in line:
data = line.split('%s:' % interface)[1].split()
tx_bytes = (data[8])
return bytestomb(tx_bytes)
def bytesbajadatransferidos():
interface= 'ppp0'
for line in open('/proc/net/dev', 'r'):
if interface in line:
data = line.split('%s:' % interface)[1].split()
rx_bytes = (data[0])
return bytestomb(rx_bytes)
print bytessubidatransferidos()
print bytesbajadatransferidos()

Python: Networked IDLE/Redo IDLE front-end while using the same back-end?

Is there any existing web app that lets multiple users work with an interactive IDLE type session at once?
Something like:
IDLE 2.6.4
Morgan: >>> letters = list("abcdefg")
Morgan: >>> # now, how would you iterate over letters?
Jack: >>> for char in letters:
print "char %s" % char
char a
char b
char c
char d
char e
char f
char g
Morgan: >>> # nice nice
If not, I would like to create one. Is there some module I can use that simulates an interactive session? I'd want an interface like this:
def class InteractiveSession():
''' An interactive Python session '''
def putLine(line):
''' Evaluates line '''
pass
def outputLines():
''' A list of all lines that have been output by the session '''
pass
def currentVars():
''' A dictionary of currently defined variables and their values '''
pass
(Although that last function would be more of an extra feature.)
To formulate my problem another way: I'd like to create a new front end for IDLE. How can I do this?
UPDATE: Or maybe I can simulate IDLE through eval()?
UPDATE 2: What if I did something like this:
I already have a simple GAE Python chat app set up, that allows users to sign in, make chat rooms, and chat with each other.
Instead of just saving incoming messages to the datastore, I could do something like this:
def putLine(line, user, chat_room):
''' Evaluates line for the session used by chat_room '''
# get the interactive session for this chat room
curr_vars = InteractiveSession.objects.where("chatRoom = %s" % chat_room).get()
result = eval(prepared_line, curr_vars.state, {})
curr_vars.state = curr_globals
curr_vars.lines.append((user, line))
if result:
curr_vars.lines.append(('SELF', result.__str__()))
curr_vars.put()
The InteractiveSession model:
def class InteractiveSession(db.Model):
# a dictionary mapping variables to values
# it looks like GAE doesn't actually have a dictionary field, so what would be best to use here?
state = db.DictionaryProperty()
# a transcript of the session
#
# a list of tuples of the form (user, line_entered)
#
# looks something like:
#
# [('Morgan', '# hello'),
# ('Jack', 'x = []'),
# ('Morgan', 'x.append(1)'),
# ('Jack', 'x'),
# ('SELF', '[1]')]
lines = db.ListProperty()
Could this work, or am I way off/this approach is infeasible/I'm duplicating work when I should use something already built?
UPDATE 3: Also, assuming I get everything else working, I'd like syntax highlighting. Ideally, I'd have some API or service I could use that would parse the code and style it appropriately.
for c in "characters":
would become:
<span class="keyword">for</span> <span class="var">c</span> <span class="keyword">in</span> <span class="string>"characters"</span><span class="punctuation">:</span>
Is there a good existing Python tool to do this?

I could implement something like this pretty quickly in Nevow. Obviously, access would need to be pretty restricted since doing something like this involves allowing access to a Python console to someone via HTTP.
What I'd do is create an Athena widget for the console, that used an instance of a custom subclass of code.InteractiveInterpreter that is common to all users logged in.
UPDATE: Okay, so you have something chat-like in GAE. If you just submit lines to a code.InteractiveInterpreter subclass that looks like this, it should work for you. Note that the interface is pretty similar to the InteractiveSession class you describe:
class SharedConsole(code.InteractiveInterpreter):
def __init__(self):
self.users = []
def write(self, data):
# broadcast output to connected clients here
for user in self.users:
user.addOutput(data)
class ConnectedUser(object):
def __init__(self, sharedConsole):
self.sharedConsole = sharedConsole
sharedConsole.users.append(self) # reference look, should use weak refs
def addOutput(self, data):
pass # do GAE magic to send data to connected client
# this is a hook for submitted code lines; call it from GAE when a user submits code
def gotCommand(self, command):
needsMore = self.sharedConsole.runsource(command)
if needsMore:
pass # tell the client to change the command line to a textarea
# or otherwise add more lines of code to complete the statement

The closest Python interpreter I know of to what you are looking for, in terms of interface, is DreamPie. It has separate input and output areas, much like a chat interface. Also, DreamPie runs all of the code in a subprocess. DreamPie also does completion and syntax coloring, much like IDLE, which means it doesn't just pipe input and output to/from the subprocess -- it has implemented the abstractions which you are looking for.
If you wish to develop a desktop application (not a web-app), I recommend basing your work on DreamPie and just adding the multiple-frontend functionality.
Update: For syntax highlighting (including HTML) see the Pygments project. But that is a completely different question; please ask one question at a time here.

As a proof of concept, you may be able to put something together using sockets and a command-line session.

this is likely possible with the upcoming implimentation of IPython using a 0MQ backend.

I would use ipython and screen. With this method, you would have to create a shared login, but you could both connect to the shared screen session. One downside would be that you would both appear as the same user.

What is the correct way to clean up when using PyOpenAL?

I'm looking at PyOpenAL for some sound needs with Python (obviously). Documentation is sparse (consisting of a demo script, which doesn't work unmodified) but as far as I can tell, there are two layers. Direct wrapping of OpenAL calls and a lightweight 'pythonic' wrapper - it is the latter I'm concerned with. Specifically, how do you clean up correctly? If we take a small example:
import time
import pyopenal
pyopenal.init(None)
l = pyopenal.Listener(22050)
b = pyopenal.WaveBuffer("somefile.wav")
s = pyopenal.Source()
s.buffer = b
s.looping = False
s.play()
while s.get_state() == pyopenal.AL_PLAYING:
time.sleep(1)
pyopenal.quit()
As it is, a message is printed on to the terminal along the lines of "one source not deleted, one buffer not deleted". But I am assuming the we can't use the native OpenAL calls with these objects, so how do I clean up correctly?
EDIT:
I eventually just ditched pyopenal and wrote a small ctypes wrapper over OpenAL and alure (pyopenal exposes the straight OpenAL functions, but I kept getting SIGFPE). Still curious as to what I was supposed to do here.

#relese reference to l b and s
del l
del b
del s
#now the WaveBuffer and Source should be destroyed, so we could:
pyopenal.quit()
Probably de destructor of pyopenal calls quit() before exit so you dont need to call it yourself.