The goal is to pulling real time data in the background (say every 5 seconds) and pull into the dashboard when needed. Here is my code. It kinda works but two issues I am seeing: 1. if I move st.write("TESTING!") to the end, it will never get executed because of the while loop. Is there a way to improve? I can imagine as the dashboard grows, there will be multiple pages/tables etc.. This won't give much flexibility. 2. The return px line in the async function, I am not very comfortable with it because I got it right via trial and error. Sorry for being such a newbie, but if there are better ways to do it, I would really appreciate.
Thank you!
import asyncio
import streamlit as st
import numpy as np
st.set_page_config(layout="wide")
async def data_generator(test):
while True:
with test:
px = np.random.randn(5, 1)
await asyncio.sleep(1)
return px
test = st.empty()
st.write("TESTING!")
with test:
while True:
px = asyncio.run(data_generator(test))
st.write(px[0])
From my experience, the trick to using asyncio is to create your layout ahead of time, using empty widgets where you need to display async info. The async coroutine would take in these empty slots and fill them out. This should help you create a more complex application.
Then the asyncio.run command can become the last streamlit action taken. Any streamlit commands after this wouldn't be processed, as you have observed.
I would also recommend to arrange any input widgets outside of the async function, during the initial layout, and then send in the widget output for processing. Of course you could draw your input widgets inside the function, but the layout then might become tricky.
If you still want to have your input widgets inside your async function, you'd definitely have to put them outside of the while loop, otherwise you would get duplicated widget error. (You might try to overcome this by creating new widgets all the time, but then the input widgets would be "reset" and interaction isn't achieved, let alone possible memory issue.)
Here's a complete example of what I mean:
import asyncio
import pandas as pd
import plotly.express as px
import streamlit as st
from datetime import datetime
CHOICES = [1, 2, 3]
def main():
print('\nmain...')
# layout your app beforehand, with st.empty
# for the widgets that the async function would populate
graph = st.empty()
radio = st.radio('Choose', CHOICES, horizontal=True)
table = st.empty()
try:
# async run the draw function, sending in all the
# widgets it needs to use/populate
asyncio.run(draw_async(radio, graph, table))
except Exception as e:
print(f'error...{type(e)}')
raise
finally:
# some additional code to handle user clicking stop
print('finally')
# this doesn't actually get called, I think :(
table.write('User clicked stop!')
async def draw_async(choice, graph, table):
# must send in all the streamlit widgets that
# this fn would interact with...
# this could possibly work, but layout is tricky
# choice2 = st.radio('Choose 2', CHOICES)
while True:
# this would not work because you'd be creating duplicated
# radio widgets
# choice3 = st.radio('Choose 3', CHOICES)
timestamp = datetime.now()
sec = timestamp.second
graph_df = pd.DataFrame({
'x': [0, 1, 2],
'y': [max(CHOICES), choice, choice*sec/60.0],
'color': ['max', 'current', 'ticking']
})
df = pd.DataFrame({
'choice': CHOICES,
'current_choice': len(CHOICES)*[choice],
'time': len(CHOICES)*[timestamp]
})
graph.plotly_chart(px.bar(graph_df, x='x', y='y', color='color'))
table.dataframe(df)
_ = await asyncio.sleep(1)
if __name__ == '__main__':
main()
Would something like this work?
import asyncio
import streamlit as st
async def tick(placeholder):
tick = 0
while True:
with placeholder:
tick += 1
st.write(tick)
await asyncio.sleep(1)
async def main():
st.header("Async")
placeholder = st.empty()
await tick(placeholder)
asyncio.run(main())
Related
so I've been thinking about this for a couple days now and I cant figure it out, I've searched around but couldn't find the answer I was looking for, so any help would be greatly appreciated.
Essentially what I am trying to do is call a method on a group of objects in my main thread from a separate thread, just once after 2 seconds and then the thread can exit, I'm just using threading as a way of creating a non-blocking 2 second pause (if there are other ways of accomplishing this please let me know.
So I have a pyqtplot graph/plot that updates from a websocket stream and the gui can only be updated from the thread that starts it (the main one).
What happens is I open a websocket stream fill up a buffer for about 2 seconds, make an REST request, apply the updates from the buffer to the data from the REST request and then update the data/plot as new messages come in. Now the issue is I can't figure out how to create a non blocking 2 second pause in the main thread without creating a child thread. If I create a child thread and pass the object that contains the dictionary I want to update after 2 seconds, I get issues regarding updating the plot from a different thread. So what I THINK is happening is when that new spawned thread is spawned the reference to the object I want to update is actually the object itself, or the data (dictionary) containing the update data is now in a different thread as the gui and that causes issues.
open websocket --> start filling buffer --> wait 2 seconds --> REST request --> apply updates from buffer to REST data --> update data as new websocket updates/messages come in.
Unfortunately the websocket and gui only start when you run pg.exec() and you can't break them up to start individually, you create them and then start them together (or at least I have failed to find a way to start them individually, alternatively I also tried using a separate library to handle websockets however this requires starting a thread for incoming messages as well)
This is the minimum reproducible example, sorry it's pretty long but I couldn't really break it down anymore without removing required functionality as well as preserving context:
import json
import importlib
from requests.api import get
import functools
import time
import threading
import numpy as np
import pyqtgraph as pg
from pyqtgraph.Qt import QtCore
QtWebSockets = importlib.import_module(pg.Qt.QT_LIB + '.QtWebSockets')
class coin():
def __init__(self):
self.orderBook = {'bids':{}, 'asks':{}}
self.SnapShotRecieved = False
self.last_uID = 0
self.ordBookBuff = []
self.pltwgt = pg.PlotWidget()
self.pltwgt.show()
self.bidBar = pg.BarGraphItem(x=[0], height=[1], width= 1, brush=(25,25,255,125), pen=(0,0,0,0))
self.askBar = pg.BarGraphItem(x=[1], height=[1], width= 1, brush=(255,25,25,125), pen=(0,0,0,0))
self.pltwgt.addItem(self.bidBar)
self.pltwgt.addItem(self.askBar)
def updateOrderBook(self, message):
for side in ['a','b']:
bookSide = 'bids' if side == 'b' else 'asks'
for update in message[side]:
if float(update[1]) == 0:
try:
del self.orderBook[bookSide][float(update[0])]
except:
pass
else:
self.orderBook[bookSide].update({float(update[0]): float(update[1])})
while len(self.orderBook[bookSide]) > 1000:
del self.orderBook[bookSide][(min(self.orderBook['bids'], key=self.orderBook['bids'].get)) if side == 'b' else (max(self.orderBook['asks'], key=self.orderBook['asks'].get))]
if self.SnapShotRecieved == True:
self.bidBar.setOpts(x0=self.orderBook['bids'].keys(), height=self.orderBook['bids'].values(), width=1 )
self.askBar.setOpts(x0=self.orderBook['asks'].keys(), height=self.orderBook['asks'].values(), width=1 )
def getOrderBookSnapshot(self):
orderBookEncoded = get('https://api.binance.com/api/v3/depth?symbol=BTCUSDT&limit=1000')
if orderBookEncoded.ok:
rawOrderBook = orderBookEncoded.json()
orderBook = {'bids':{}, 'asks':{}}
for orders in rawOrderBook['bids']:
orderBook['bids'].update({float(orders[0]): float(orders[1])})
for orders in rawOrderBook['asks']:
orderBook['asks'].update({float(orders[0]): float(orders[1])})
last_uID = rawOrderBook['lastUpdateId']
while self.ordBookBuff[0]['u'] <= last_uID:
del self.ordBookBuff[0]
if len(self.ordBookBuff) == 0:
break
if len(self.ordBookBuff) >= 1 :
for eachUpdate in self.ordBookBuff:
self.last_uID = eachUpdate['u']
self.updateOrderBook(eachUpdate)
self.ordBookBuff = []
self.SnapShotRecieved = True
else:
print('Error retieving order book.') #RESTfull request failed
def on_text_message(message, refObj):
messaged = json.loads(message)
if refObj.SnapShotRecieved == False:
refObj.ordBookBuff.append(messaged)
else:
refObj.updateOrderBook(messaged)
def delay(myObj):
time.sleep(2)
myObj.getOrderBookSnapshot()
def main():
pg.mkQApp()
refObj = coin()
websock = QtWebSockets.QWebSocket()
websock.connected.connect(lambda : print('connected'))
websock.disconnected.connect(lambda : print('disconnected'))
websock.error.connect(lambda e : print('error', e))
websock.textMessageReceived.connect(functools.partial(on_text_message, refObj=refObj))
url = QtCore.QUrl("wss://stream.binance.com:9443/ws/btcusdt#depth#1000ms")
websock.open(url)
getorderbook = threading.Thread(target = delay, args=(refObj,), daemon=True) #, args = (lambda : websocketThreadExitFlag,)
getorderbook.start()
pg.exec()
if __name__ == "__main__":
main()
I'm trying to build a bokeh application with streaming data that tracks multiple "strategies" as they are generated in a prisoners-dilemma agent based model. I've run into a problem trying to get my line plots NOT to connect all the data points in one line. I put together this little demo script that replicates the issue. I've read lots of documentation on line and multi_line rendering in bokeh plots, but I just haven't found something that seems to match my simple case. You can run this code & it will automatically open a bokeh server at localhost:5004 ...
from bokeh.server.server import Server
from bokeh.application import Application
from bokeh.application.handlers.function import FunctionHandler
from bokeh.plotting import figure, ColumnDataSource
from bokeh.models import Button
from bokeh.layouts import column
import random
def make_document(doc):
# Create a data source
data_source = ColumnDataSource({'step': [], 'strategy': [], 'ncount': []})
# make a list of groups
strategies = ['DD', 'DC', 'CD', 'CCDD']
# Create a figure
fig = figure(title='Streaming Line Plot',
plot_width=800, plot_height=400)
fig.line(x='step', y='ncount', source=data_source)
global step
step = 0
def button1_run():
global callback_obj
if button1.label == 'Run':
button1.label = 'Stop'
button1.button_type='danger'
callback_obj = doc.add_periodic_callback(button2_step, 100)
else:
button1.label = 'Run'
button1.button_type = 'success'
doc.remove_periodic_callback(callback_obj)
def button2_step():
global step
step = step+1
for i in range(len(strategies)):
new = {'step': [step],
'strategy': [strategies[i]],
'ncount': [random.choice(range(1,100))]}
fig.line(x='step', y='ncount', source=new)
data_source.stream(new)
# add on_click callback for button widget
button1 = Button(label="Run", button_type='success', width=390)
button1.on_click(button1_run)
button2 = Button(label="Step", button_type='primary', width=390)
button2.on_click(button2_step)
doc.add_root(column(fig, button1, button2))
doc.title = "Now with live updating!"
apps = {'/': Application(FunctionHandler(make_document))}
server = Server(apps, port=5004)
server.start()
if __name__ == '__main__':
server.io_loop.add_callback(server.show, "/")
server.io_loop.start()
My hope was that by looping thru the 4 "strategies" in the example (after clicking button2), I could stream the new data coming out of the simulation into a line plot for that one strategy and step only. But what I get is one line with all four values connected vertically, then one of them connected to the first one at the next step. Here's what it looks like after a few steps:
I noticed that if I move data_source.stream(new) out of the for loop, I get a nice single line plot, but of course it is only for the last strategy coming out of the loop.
In all the bokeh multiple line plotting examples I've studied (not the multi_line glyph, which I can't figure out and which seems to have some issues with the Hover tool), the instructions seem pretty clear: if you want to render a second line, you add another fig.line renderer to an existing figure, and it draws a line with the data provided in source=data_source for this line. But even though my for-loop collects and adds data separately for each strategy, I don't get 4 line plots, I get only one.
Hoping I'm missing something obvious! Thanks in advance.
Seems like you need a line per strategy, not a line per step. If so, here's how I would do it:
import random
from bokeh.application import Application
from bokeh.application.handlers.function import FunctionHandler
from bokeh.layouts import column
from bokeh.models import Button
from bokeh.palettes import Dark2
from bokeh.plotting import figure, ColumnDataSource
from bokeh.server.server import Server
STRATEGIES = ['DD', 'DC', 'CD', 'CCDD']
def make_document(doc):
step = 0
def new_step_data():
nonlocal step
result = [dict(step=[step],
ncount=[random.choice(range(1, 100))])
for _ in STRATEGIES]
step += 1
return result
fig = figure(title='Streaming Line Plot', plot_width=800, plot_height=400)
sources = []
for s, d, c in zip(STRATEGIES, new_step_data(), Dark2[4]):
# Generate the very first step right away
# to avoid having a completely empty plot.
ds = ColumnDataSource(d)
sources.append(ds)
fig.line(x='step', y='ncount', source=ds, color=c)
callback_obj = None
def button1_run():
nonlocal callback_obj
if callback_obj is None:
button1.label = 'Stop'
button1.button_type = 'danger'
callback_obj = doc.add_periodic_callback(button2_step, 100)
else:
button1.label = 'Run'
button1.button_type = 'success'
doc.remove_periodic_callback(callback_obj)
def button2_step():
for src, data in zip(sources, new_step_data()):
src.stream(data)
# add on_click callback for button widget
button1 = Button(label="Run", button_type='success', width=390)
button1.on_click(button1_run)
button2 = Button(label="Step", button_type='primary', width=390)
button2.on_click(button2_step)
doc.add_root(column(fig, button1, button2))
doc.title = "Now with live updating!"
apps = {'/': Application(FunctionHandler(make_document))}
server = Server(apps, port=5004)
if __name__ == '__main__':
server.io_loop.add_callback(server.show, "/")
server.start()
server.io_loop.start()
Thank you, Eugene. Your solution got me back on the right track. I played around with it a bit more and ended up with the following:
import colorcet as cc
from bokeh.server.server import Server
from bokeh.application import Application
from bokeh.application.handlers.function import FunctionHandler
from bokeh.plotting import figure, ColumnDataSource
from bokeh.models import Button
from bokeh.layouts import column
import random
def make_document(doc):
# make a list of groups
strategies = ['DD', 'DC', 'CD', 'CCDD']
# initialize some vars
step = 0
callback_obj = None
colors = cc.glasbey_dark
# create a list to hold all CDSs for active strategies in next step
sources = []
# Create a figure container
fig = figure(title='Streaming Line Plot - Step 0', plot_width=800, plot_height=400)
# get step 0 data for initial strategies
for i in range(len(strategies)):
step_data = dict(step=[step],
strategy = [strategies[i]],
ncount=[random.choice(range(1, 100))])
data_source = ColumnDataSource(step_data)
color = colors[i]
# this will create one fig.line renderer for each strategy & its data for this step
fig.line(x='step', y='ncount', source=data_source, color=color, line_width=2)
# add this CDS to the sources list
sources.append(data_source)
def button1_run():
nonlocal callback_obj
if button1.label == 'Run':
button1.label = 'Stop'
button1.button_type='danger'
callback_obj = doc.add_periodic_callback(button2_step, 100)
else:
button1.label = 'Run'
button1.button_type = 'success'
doc.remove_periodic_callback(callback_obj)
def button2_step():
nonlocal step
data = []
step += 1
fig.title.text = 'Streaming Line Plot - Step '+str(step)
for i in range(len(strategies)):
step_data = dict(step=[step],
strategy = [strategies[i]],
ncount=[random.choice(range(1, 100))])
data.append(step_data)
for source, data in zip(sources, data):
source.stream(data)
# add on_click callback for button widget
button1 = Button(label="Run", button_type='success', width=390)
button1.on_click(button1_run)
button2 = Button(label="Step", button_type='primary', width=390)
button2.on_click(button2_step)
doc.add_root(column(fig, button1, button2))
doc.title = "Now with live updating!"
apps = {'/': Application(FunctionHandler(make_document))}
server = Server(apps, port=5004)
server.start()
if __name__ == '__main__':
server.io_loop.add_callback(server.show, "/")
server.io_loop.start()
Result is just what I was looking for ...
I have method in my tkinter application. This method is used to export data into CSV from another application. The loop for export the data is very heavy. It takes days of time to complete.
I just came across the multi threading concept. Its kind of difficult to understand, I spent an entire day on this but couldn't achieve anything. Below is the code I use in my loop. Can this be handled by multiple threads without freezing my tkinter UI?
I have a Label that shows the number of records(cells) exported, in the tkinter window.
def export_cubeData(self):
exportPath = self.entry_exportPath.get()
for b in itertools.product(*(k.values())):
self.update()
if (self.flag == 0):
list1 = list()
for pair in zip(dims, b):
list1.extend(pair)
list1.append(self.box_value.get())
mdx1 = mdx.format(*temp, *list1)
try:
data = tm1.cubes.cells.execute_mdx(mdx1)
data1 = Utils.build_pandas_dataframe_from_cellset(data)
final_df = final_df.append(data1)
cellCount = tm1.cubes.cells.execute_mdx_cellcount(mdx1)
finalcellCount = finalcellCount + cellCount
self.noOfRecordsProcessed['text'] = finalcellCount
except:
pass
else:
tm.showinfo("Export Interrupted", "Data export has been cancelled")
return
final_df.to_csv(exportPath)
print(time.time() - start)
tm.showinfo("info", "Data export has been completed")
self.noOfRecordsProcessed['text'] = '0'
I've been working on this for a while and I can't find any information about adding a row to a window. I seen it done with pyside2 and qt, witch would work but the users are using multiple versions of Maya (2016 = pyside, 2017=pyside2).
I want it like adding a widget in in pyside. I done it where adding a row is a function like add row 1, add row 2, and add row 3 but the script get to long. I need to parent to rowColumnLayout and make that unique in order to delete that later. Also I have to query the textfield in each row. Maybe a for loop that adds a number to the row? I really don't know but this is what I have so far:
from maya import cmds
def row( ):
global fed
global info
item=cmds.optionMenu(mygroup, q=True, sl=True)
if item == 1:
cam=cmds.optionMenu(mygroup, q=True, v=True)
fed=cmds.rowColumnLayout(nc = 1)
cmds.rowLayout(nc=7)
cmds.text(l= cam )
cmds.text(l=u'Frame Range ')
start = cmds.textField('textField3')
cmds.text(l=u' to ')
finish = cmds.textField('textField2')
cmds.button(l=u'render',c='renderTedd()')
cmds.button(l=u'delete',c='deleteRow()')
cmds.setParent (fed)
def deleteRow ():
cmds.deleteUI(fed, layout=True)
if item == 2:
print item
global red
cam1=cmds.optionMenu(mygroup, q=True, v=True)
red = cmds.rowColumnLayout()
cmds.rowLayout(nc=7)
cmds.text(l= cam1 )
cmds.text(l=u'Frame Range ')
start = cmds.textField('textField3')
cmds.text(l=u' to ')
finish = cmds.textField('textField2')
cmds.button(l=u'render',c='renderTedd()')
cmds.button(l=u'delete',c='deleteRow2()')
cmds.setParent (red)
def deleteRow2 ():
cmds.deleteUI(red, control=True)
def cameraInfo():
info=cmds.optionMenu(mygroup, q=True, sl=True)
print info
def deleteRow ():
cmds.deleteUI(fed, control=True)
def getCamera():
layers=pm.ls(type="renderLayer")
for layer in layers:
pm.editRenderLayerGlobals(currentRenderLayer=layer)
cameras=pm.ls(type='camera')
for cam in cameras:
if pm.getAttr(str(cam) + ".renderable"):
relatives=pm.listRelatives(cam, parent=1)
cam=relatives[0]
cmds.menuItem(p=mygroup,label=str (cam) )
window = cmds.window()
cmds.rowColumnLayout(nr=10)
mygroup = cmds.optionMenu( label='Colors', changeCommand='cameraInfo()' )
getCamera()
cmds.button(l=u'create camera',aop=1,c='row ()')
cmds.showWindow( window )
This is totally doable with cmds. The trick is just to structure the code so that the buttons in each row know and can operate on the widgets in that row; once that works you can add rows all day long.
To make it work you want to do two things:
Don't use the string form of callbacks. It's never a good idea, for reasons detailed here
Do use closures to make sure your callbacks are referring to the right widgets. Done right you can do what you want without the overhead of a class.
Basically, this adds up to making a function which generates both the gui items for the row and also generates the callback functions -- the creator function will 'remember' the widgets and the callbacks it creates will have access to the widgets. Here's a minimal example:
def row_test():
window = cmds.window(title='lotsa rows')
column = cmds.columnLayout()
def add_row(cameraname) :
cmds.setParent(column)
this_row = cmds.rowLayout(nc=6, cw6 = (72, 72, 72, 72, 48, 48) )
cmds.text(l= cameraname )
cmds.text(l=u'Frame Range')
start = cmds.intField()
finish = cmds.intField()
# note: buttons always fire a useless
# argument; the _ here just ignores
# that in both of these callback functions
def do_delete(_):
cmds.deleteUI(this_row)
def do_render(_):
startframe = cmds.intField(start, q=True, v=True)
endframe = cmds.intField(finish, q=True, v=True)
print "rendering ", cameraname, "frames", startframe, endframe
cmds.button(l=u'render',c=do_render)
cmds.button(l=u'delete',c=do_delete)
for cam in cmds.ls(type='camera'):
add_row(cam)
cmds.showWindow(window)
row_test()
By defining the callback functions inside of add_row(), they have access to the widgets which get stored as start and finish. Even though start and finish will be created over and over each time the function runs, the values they store are captured by the closures and are still available when you click a button. They also inherit the value of cameraname so the rendering script can get that information as well.
At the risk of self-advertising: if you need to do serious GUI work using cmds you should check out mGui -- a python module that makes working with cmds gui less painful for complex projects.
my problem seem to be similar to This Thread however, while I think I am following the advised method, I still get a PicklingError. When I run my process locally without sending to an IPython Cluster Engine the function works fine.
I am using zipline with IPyhon's notebook, so I first create a class based on zipline.TradingAlgorithm
Cell [ 1 ]
from IPython.parallel import Client
rc = Client()
lview = rc.load_balanced_view()
Cell [ 2 ]
%%px --local # This insures that the Class and modules exist on each engine
import zipline as zpl
import numpy as np
class Agent(zpl.TradingAlgorithm): # must define initialize and handle_data methods
def initialize(self):
self.valueHistory = None
pass
def handle_data(self, data):
for security in data.keys():
## Just randomly buy/sell/hold for each security
coinflip = np.random.random()
if coinflip < .25:
self.order(security,100)
elif coinflip > .75:
self.order(security,-100)
pass
Cell [ 3 ]
from zipline.utils.factory import load_from_yahoo
start = '2013-04-01'
end = '2013-06-01'
sidList = ['SPY','GOOG']
data = load_from_yahoo(stocks=sidList,start=start,end=end)
agentList = []
for i in range(3):
agentList.append(Agent())
def testSystem(agent,data):
results = agent.run(data) #-- This is how the zipline based class is executed
#-- next I'm just storing the final value of the test so I can plot later
agent.valueHistory.append(results['portfolio_value'][len(results['portfolio_value'])-1])
return agent
for i in range(10):
tasks = []
for agent in agentList:
#agent = testSystem(agent,data) ## On its own, this works!
#-- To Test, uncomment the above line and comment out the next two
tasks.append(lview.apply_async(testSystem,agent,data))
agentList = [ar.get() for ar in tasks]
for agent in agentList:
plot(agent.valueHistory)
Here is the Error produced:
PicklingError Traceback (most recent call last)/Library/Python/2.7/site-packages/IPython/kernel/zmq/serialize.pyc in serialize_object(obj, buffer_threshold, item_threshold)
100 buffers.extend(_extract_buffers(cobj, buffer_threshold))
101
--> 102 buffers.insert(0, pickle.dumps(cobj,-1))
103 return buffers
104
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed
If I override the run() method from zipline.TradingAlgorithm with something like:
def run(self, data):
return 1
Trying something like this...
def run(self, data):
return zpl.TradingAlgorithm.run(self,data)
results in the same PicklingError.
then the passing off to the engines works, but obviously the guts of the test are not performed. As run is a method internal to zipline.TradingAlgorithm and I don't know everything that it does, how would I make sure it is passed through?
It looks like the zipline TradingAlgorithm object is not pickleable after it has been run:
import zipline as zpl
class Agent(zpl.TradingAlgorithm): # must define initialize and handle_data methods
def handle_data(self, data):
pass
agent = Agent()
pickle.dumps(agent)[:32] # ok
agent.run(data)
pickle.dumps(agent)[:32] # fails
But this suggests to me that you should be creating the Agents on the engines, and only passing data / results back and forth (ideally, not passing data across at all, or at most once).
Minimizing data transfers might look something like this:
define the class:
%%px
import zipline as zpl
import numpy as np
class Agent(zpl.TradingAlgorithm): # must define initialize and handle_data methods
def initialize(self):
self.valueHistory = []
def handle_data(self, data):
for security in data.keys():
## Just randomly buy/sell/hold for each security
coinflip = np.random.random()
if coinflip < .25:
self.order(security,100)
elif coinflip > .75:
self.order(security,-100)
load the data
%%px
from zipline.utils.factory import load_from_yahoo
start = '2013-04-01'
end = '2013-06-01'
sidList = ['SPY','GOOG']
data = load_from_yahoo(stocks=sidList,start=start,end=end)
agent = Agent()
and run the code:
def testSystem(agent, data):
results = agent.run(data) #-- This is how the zipline based class is executed
#-- next I'm just storing the final value of the test so I can plot later
agent.valueHistory.append(results['portfolio_value'][len(results['portfolio_value'])-1])
# create references to the remote agent / data objects
agent_ref = parallel.Reference('agent')
data_ref = parallel.Reference('data')
tasks = []
for i in range(10):
for j in range(len(rc)):
tasks.append(lview.apply_async(testSystem, agent_ref, data_ref))
# wait for the tasks to complete
[ t.get() for t in tasks ]
And plot the results, never fetching the agents themselves
%matplotlib inline
import matplotlib.pyplot as plt
for history in rc[:].apply_async(lambda : agent.valueHistory):
plt.plot(history)
This is not quite the same code you shared - three agents bouncing back and forth on all your engines, whereas this has on agent per engine. I don't know enough about zipline to say whether that's useful to you or not.