I am simulating meetings in conference rooms and I started with the Bank Renege example as a model.
It is working as intended - this first version just tries to put 5 meetings into 3 rooms in several different hours.
HOWEVER, I notice that the queued requests (those who don't get a conference room) accumulate in the Resource.queue list. Shouldn't they release and disappear? Is this an error on my part? I have tried
req.cancel
simpy.Interrupt
and a few other attempts, to no effect that I can tell.
RANDOM_SEED = 181106
NbrMtgs = 5 # Total number of meetings in a period
INTERVAL = 1.0 # Generate new meetings every x hours
def Schedule(env, NbrMtgs, interval, ResRooms):
#Source generates meetings
while True:
for i in range(NbrMtgs):
h=env.now
m = Meet(env, 'Meeting%02d_%02d' % (h,i), ResRooms, TimeInMtg=MtgLen)
env.process(m)
t = 1
yield env.timeout(t)
def Meet(env, name, ResRooms, TimeInMtg):
#Customer arrives, is served and leaves.
arrive = env.now
print('%7.4f %s: Here I am' % (arrive, name))
with ResRooms.request() as req:
results = yield req | env.timeout(MtgLen)
wait = env.now - arrive
if req in results:
# We got to the ResRooms
print('%7.4f %s: Waited %6.3f' % (env.now, name, wait))
#tib = random.expovariate(1.0 / TimeInMtg)
yield env.timeout(TimeInMtg)
print('%7.4f %s: Finished' % (env.now, name))
else:
# We reneged
#req.cancel() this doesnt clear queue and does give an error at last step
# no notable effect ResRooms.release(req)
#simpy.Interrupt('No rooms') still piling up. no effect
#yield env.timeout(TimeInMtg)
print('%7.4f %s: RENEGED after %6.3f' % (env.now, name, wait))
print("this req=",req) # is something like <Request() object at 0x2449892f908>
print("users",ResRooms.users)
print("queue",ResRooms.queue)
env = simpy.Environment()
# Start processes and run
ResRooms = simpy.Resource(env, capacity=3)
MtgLen = 1
Hours = 8
env.process(Schedule(env, NbrMtgs, INTERVAL, ResRooms))
env.run(until=5)
No that's not an error. If you request the resources the way you did, with the with-statement, all the request that did not get the resource will end up in the queue and wait until the resource is available again. Do you want to cancel those requests?
Related
I am just starting to work on an event simulation, and I am having some issues with the queue length. The question is how can I set an initial queue in the following code? For instance, is there a way to set the initial_queue = 200 so that when the simulation begins there are already 200 cars in the queue?
import simpy
num_of_machines = 1
env = simpy.Environment()
bcs = simpy.Resource(env, capacity=num_of_machines)
def process_client(env):
with bcs.request() as req:
i = 0
while True:
i += 1
yield req
print('Car %s starting to charge at %s' % (i, env.now))
yield env.timeout(90)
print('Car %s ending charge at %s' % (i, env.now))
print('Queue size: %s' % len(bcs.queue))
env.process(process_client(env))
env.process(process_client(env))
env.run(until=300)
Which returns:
Car 1 starting to charge at 0
Car 1 ending charge at 90
Queue size: 0
Car 2 starting to charge at 90
Car 2 ending charge at 180
Queue size: 0
Car 3 starting to charge at 180
Car 3 ending charge at 270
Queue size: 0
Car 4 starting to charge at 270
So you are on the right track, but your while loop never ends so the env.process(process_client(env)) never gets executed, but would be a unwanted recursive call anyway.
I broke your code into two parts. The first part models just one car trying to get a charge. The second creates 200 of instances of the first process. Note that we are creating these processes before the model starts. The model does not start until we the env.run(900) line executes. When the model starts, all 200 processes will try to grab the resource and 199 of them will get queue.
here is the code
"""
Simple example of starting a resource with a queue of 200 requests
programmer: Michael R. Gibbs
"""
import simpy
def charge_process(env, res_pool, car_id):
"""
Models a car waiting in a queue for a charger
and charging
"""
with res_pool.request() as req:
yield req
print('Car %s starting to charge at %s' % (car_id, env.now))
yield env.timeout(90)
print('Car %s ending charge at %s' % (car_id, env.now))
print('Queue size: %s' % len(res_pool.queue))
def load_queue(env, res_pool):
"""
create 200 charging processes.
each one will queue up for a charing station
"""
for i in range(1,201):
env.process(charge_process(env, res_pool, i))
# boot
env = simpy.Environment()
res_pool = simpy.Resource(env, capacity=1)
load_queue(env, res_pool)
env.run(900)
I have problem about Dronekit on Python
My project about receive GPS Location from Android application
and Launch drone flying to that Position, Everything work fine but problem is Drone can takeoff but drone didn't go to that Location(Testing 10+ times work only 1 time)
Here is my code (I think problem is GlobalRelative)
# Import DroneKit-Python
from dronekit import connect, VehicleMode, LocationGlobalRelative
import time
import urllib2
import simplejson
import json
import requests
#Wait for json data
response = urllib2.urlopen("http://localhost:3000/posts")
data = simplejson.load(response)
print(data)
json_string = data[0]["Information"]
gps = json.loads(json_string)
x=gps['lat']
y=gps['lon']
r = requests.delete('http://localhost:3000/posts/1')
# Connect to the Vehicle.
print("Connecting")
vehicle = connect('com4', wait_ready=False, baud=57600)#; vehicle.wait_ready(True, timeout=300)
print("Connected")
# Get some vehicle attributes (state)
print "Get some vehicle attribute values:"
print " GPS: %s" % vehicle.gps_0
print " Battery: %s" % vehicle.battery
print " Last Heartbeat: %s" % vehicle.last_heartbeat
print " Is Armable?: %s" % vehicle.is_armable
print " System status: %s" % vehicle.system_status.state
print " Mode: %s" % vehicle.mode.name # settable
# Takeoff Function
def arm_and_takeoff(tgt_altitude):
print("Arming motors")
# while not vehicle.is_armable:
# time.sleep(1)
vehicle.mode = VehicleMode("GUIDED")
vehicle.armed = True
print("Takeoff")
vehicle.simple_takeoff(tgt_altitude)
# wait reach tgt_altitude
while True:
altitude = vehicle.location.global_relative_frame.alt
if altitude >= tgt_altitude -1:
print("Altitude Reached")
break
time.sleep(1)
# Main
arm_and_takeoff(10)
# Speed
vehicle.airspeed = 7
# Go to wp
wp1 = LocationGlobalRelative(x, y, 10)
# Close vehicle object before exiting script
vehicle.mode = VehicleMode("RTL")
vehicle.close()
print("Completed")
Alternative, If I can't fix this problem I want to use MissionPlanner( I test on it , and it work) But I want to wait for GPS Location from Phone, and Launch the mission( every thing must automatic ) I have no idea how to bypass MissionPlanner
The line: wp1 = LocationGlobalRelative(x, y, 10) only assign wp1 variable with a location coordinate. You can use vehicle.simple_goto(wp1). simple_goto is a built in function in dronekit to command the vehicle to specific coordinate you can read more about it here http://python.dronekit.io/automodule.html?highlight=simple_goto#dronekit.Vehicle.simple_goto
I want to simulate customers arrivals at specific time periods (not generated according to statistical distribution). The arrival times are defined in the csv file that I loaded into pandas dataframe df:
df.head()
arrival_time start_service end_service waiting_time service_duration
09:00:20 09:01:00 09:06:00 0.40 5.00
09:01:00 09:02:20 09:04:00 1.20 1.40
This is my current code, but I don't know how to force the entities (clients) to arrive according to the schedule defined in df, e.g. at 09:00:20, then at 09:01:00, etc. I assume that I should also set the starting simulation time in the Environment, but how can I do it? (I don't need the real-time simulation):
import random
import simpy
import pandas as pd
def source(env, df, counter):
for i, row in df.iterrows():
c = client(env, 'Client%02d' % i, counter, row, time_in_bank=row["service_duration"])
env.process(c)
def client(env, name, counter, row, time_in_bank):
arrive = env.now # probably some changes to be done here
print('%s arrived at %7.4f' % (name,arrive))
with counter.request() as req:
results = yield req
wait = env.now - row["waiting_time"]
print('%s waited %6.3f' % (name, wait))
yield env.timeout(time_in_bank)
print('%s exited the office at %7.4f' % (name, env.now))
df = pd.read_csv("arrivals.csv",sep=",",header=0)
env = simpy.Environment()
counter = simpy.Resource(env, capacity=1)
env.process(source(env, df.head(), counter))
env.run()
You need to:
convert datetime objects to timestamps and work with these
Define an initial_time that you pass to Environment()
Define how long one SimPy step is, e.g. 1 step == 1 sec
Example (featuring arrow):
import arrow
import simpy
start = arrow.get('2016-11-05T00:00:00')
env = simpy.Environment(initial_time=start.timestamp)
def proc(env):
print('Proc start at', env.now, arrow.get(env.now))
yield env.timeout(10) # 10 seconds
print('Proc stop at ', env.now, arrow.get(env.now))
p = env.process(proc(env))
env.run(p)
Output:
Proc start at 1478304000 2016-11-05T00:00:00+00:00
Proc stop at 1478304010 2016-11-05T00:00:10+00:00
I was hoping someone could point me a the right direction as i work through building a queue model in Simpy. The documentation and examples on the Simpy site are amazing and i wanted to see if i could build upon an example they provided.
Essentially, i am simulating a process where a clerk has to review paperwork. The paperwork both arrives and exits his queue (i.e. is serviced) at random time intervals (much like an MM1 queue) with a first-in-first-out service discipline.
The tricky part i am finding is that I would like to model the process such that the clerk has to sleep for some period of time before returning to work. For example, if the simulation runs for "4 weeks", the clerk should sleep for 12 hours per day during that period. I cant seem to get this last caveat to function. Please see my code below, thanks!
import random
import simpy
import matplotlib.pyplot as plt
RANDOM_SEED = 123456 #Random seed
INTERVAL_RECORDS = 30.0 #review records roughly every xx hours?
T_INTER = 28 #generate records roughly every xx hours?
WEEKS = 4
SIM_TIME = WEEKS*7*24*60 #simulation time (hours?)
class record_review:
def __init__(self, env):
self.env = env
self.shift_exchange = env.event()
def record(env, interval, counter, data, t_inter):
"""Source generates records randomly"""
for i in range(1):
"""start off with x=i records in queue"""
c = review(env, 'Record%02d' % i, counter, time_in_queue=30.0)
env.process(c)
t = random.expovariate(1.0 / interval) #set up random generation rate
yield env.timeout(t)
while True:
"""continually generate records throughout simulation"""
yield env.timeout(random.randint(t_inter-2, t_inter+2)) #generate record between +/- 2 of interal
i += 1
c = review(env, 'Record%02d' % i, counter, time_in_queue=30.0)
env.process(c)
t = random.expovariate(1.0 / interval) #random generation rate
yield env.timeout(t)
def review(env, name, counter, time_in_queue):
"""Record arrives, is reviewed and exits."""
arrive = env.now
print('%7.4f %s: Record has entered queue' % (arrive, name))
with counter.request() as req:
yield req
wait = env.now - arrive #total wait time / record
tib = random.expovariate(1.0 / time_in_queue) #time in queue/review rate
yield env.timeout(tib)
data.append(wait) #monitor
print('%7.4f %s: Waited %6.3f' % (env.now, name, wait))
print('%7.4f %s: Finished' % (env.now, name))
def shift_exchange(self, env):
while True:
for i in range(SIM_TIME):
yield env.timeout(60)
self.shift_exchange = env.event()
# Setup and start the simulation
print('Batch Record Review Simulation')
random.seed(RANDOM_SEED)
env = simpy.Environment()
data = []
# Start processes and run
counter = simpy.Resource(env, capacity=1)
env.process(record(env, INTERVAL_RECORDS, counter, data, T_INTER))
env.run(until=SIM_TIME)
I'm trying to get code similar to the following example working correctly:
from multiprocessing import Process, Queue, Manager, Pool
import time
from datetime import datetime
def results_producer(the_work, num_procs):
results = Manager().Queue()
ppool = Pool(num_procs)
multiplier = 3
#step = len(the_work)/(num_procs*multiplier)
step = 100
for i in xrange(0,len(the_work), step):
batch = the_work[i:i+step]
ppool.apply_async(do_work1, args=(i,batch,results))#,callback=results.put_nowait)
return (ppool, results)
def results_consumer(results, total_work, num_procs, pool=None):
current = 0
batch_size=10
total = total_work
est_remaining = 0
while current < total_work:
size = results.qsize()
est_remaining = total_work - (current + size)
if current % 1000 == 0:
print 'Attempting to retrieve item from queue that is empty? %s, with size: %d and remaining work: %d' % (results.empty(), size, est_remaining)
item = results.get()
results.task_done()
current += 1
if current % batch_size == 0 or total_work - current < batch_size:
if pool is not None and est_remaining == 0 and size/num_procs > batch_size:
pool.apply_async(do_work2, args=(current, item, True))
else:
do_work2(current,item, False)
if current % 1000 == 0:
print 'Queue size: %d and remaining work: %d' % (size, est_remaining)
def do_work1(i, w, results):
time.sleep(.05)
if i % 1000 == 0:
print 'did work %d: from %d to %d' % (i,w[0], w[-1])
for j in w:
#create an increasing amount of work on the queue
results.put_nowait(range(j*2))
def do_work2(index, item, in_parallel):
time.sleep(1)
if index % 50 == 0:
print 'processed result %d with length %d in parallel %s' % (index, len(item), in_parallel)
if __name__ == "__main__":
num_workers = 2
start = datetime.now()
print 'Start: %s' % start
amount_work = 4000
the_work = [i for i in xrange(amount_work)]
ppool, results = results_producer(the_work, num_workers)
results_consumer(results, len(the_work), num_workers, ppool)
if ppool is not None:
ppool.close()
ppool.join()
print 'Took: %s time' % (datetime.now() - start)
And it deadlocks on the results.put_nowait call from do_work1 even though the queue is empty! Sometimes the code is able to put all the work on the queue but the results.get call from results_consumer blocks since it is apparently empty even though the work has not been consumed yet.
Additionally, I checked the programming guidelines: https://docs.python.org/2/library/multiprocessing.html and believe the above code conforms to it. Lastly the problem in this post: Python multiprocessing.Queue deadlocks on put and get seems very similar and claims to be solved on Windows (I'm running this on Windows 8.1) however the above code doesn't block due to the parent process attempting to join the child process since the logic is similar to the suggested answer. Any suggestions about the cause of the deadlock and how to fix it? Also in general, what is the best way to enable multiple producers to provide results for a consumer to process in python?