Develop Reference

I cannot delete a file with python

First off I am not too familiar with python and still currently learning also its my first time ever posting here so sorry if I am mess up with some details.
I am running experiments, and need to run multiple replicates. The issue arises when I need to start a new set of replicates, the program moves the already run experiments into a new folder and is then suppose to start the new replicates. However, what happens only on the last experiment, the environment.cfg folder is not transferred and the program crashes.
from os import listdir,chdir
import subprocess
from random import randrange,sample
from shutil import copy,copytree,rmtree
from os import mkdir,remove
import csv
import time
import shutil
test = input("Is this a test? (y/n)")
if test == "y":
text = input("What are you testing?")
testdoc=open("Testing Documentation.txt","a")
testdoc.write(text)
elif test != "y" and test != "n":
print("Please type in y or n")
test = input("Is this a test? (y/n)")
expnum = int(input("Number of Experiments: "))
exptype = int(input("Which experiment do you want to run? (1/2)"))
repnum= int(input("How many replicates do you want?"))
print(f"You want {repnum} replicates, each replicate will contain {expnum} for a total of {repnum*expnum}")
confirm = input("Is this correct? (y/n)")
if confirm == "y":
for rep in range(repnum):
if exptype == 1:
for ex in range (expnum):
num= ex + 1
mkdir('experiment_'+str(num)) #create a directory named cdir
copy('./default_files/environment.cfg','./experiment_'+str(num)+'/environment.cfg') #copy one file from one place to another
#WSIZE
env_file = open('./experiment_'+str(num)+'/environment.cfg','r')
env_file_content = []
for i in env_file:
env_file_content.append(i.split(' '))
#env_file_content = [['a','b'],['c','d']]
#access first line: env_file_content[0] ; note that index in python start from 0
#access first element in second line: env_file_content[1][0] ; note that index in python start from 0
n = num #number of resources
var0 = '100' #resource inflow
var1 = '0.01' #resource outflow
var3 = '1' # The minimum amount of resource required
reactiontype = ['not','nand','and','orn','or','andn','nor','xor','equ']
reward = ["1.0","1.0","2.0","2.0","4.0","4.0","8.0","8.0","16.0"]
#n = sample(range(10),1)[0]
out = open('./experiment_'+str(num)+'/environment.cfg','w')
for i in range(n):
out.write('RESOURCE res'+str(i)+':inflow='+var0+':outflow='+var1+'\n')
sc=0
for i in range(n):
out.write('REACTION reaction'+str(i)+' '+reactiontype[sc]+' process:resource=res'+str(i)+':value='+reward[sc]+':min='+var3+ '\n')
sc+=1
if sc==len(reactiontype):
sc = 0
out.close()
##RUN Avida from python
copy('./experiment_' + str(num) + '/environment.cfg', './')
print("starting experiment_" + str(num))
proc = subprocess.Popen(['./avida'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
# Wait for the subprocess to finish
output, error = proc.communicate()
# Close the subprocess
proc.terminate()
shutil.move('./data','./experiment_' +str(num))
#copytree('./data', './experiment_' + str(num) + '/data')
#rmtree('./data')
remove('./environment.cfg')
replicatenum = rep + 1
mkdir('replicate_'+str(replicatenum)) #create a directory named replicate_
for repl in range(expnum):
numb = repl + 1
source = './experiment_'+str(numb)
dest = './replicate_' + str(replicatenum) + '/experiment_' + str(numb)
shutil.move(source, dest)
I tried renaming the folder, however that also failed. The issue seems only to arise when the program is running as taking the same code after the program has crashed, will cause no problems.

Call python function with arguments and get returned value in autohotkey

I have a python script called "server.py" and inside it I have a function def calcFunction(arg1): ... return output How can I call the function calcFunction with arguments and use the return value in autohotkey? This is what I want to do in autohotkey:
ToSend = someString ; a string
output = Run server.py, calcFunction(ToSend) ; get the returned value from the function with ToSend as argument
Send, output ; use the returned value in autohotkey
I have looked online but nothing seems to fully answer my question. Can it even be done?

In order to send your parameters to Python, you could use arguments from within your Python script. You can do this with the sys library:
import sys
print(sys.argv[0]) # name of file
print(sys.argv[1]) # first argument
print(sys.argv[2]) # second argument...
From within your AutoHotKey script, you can send parameters to the Python script by adding them as arguments right after specifying the file name:
RunWait, server.py "This will be printed as the first argument!" "This is the second!"
Then, to get the output of the function back to AHK, you could use sys again by utilizing it's exit() function:
sys.exit(EXIT_NUMBER)
And back in AHK, you recieve the EXIT_NUMBER inside the variable ErrorLevel.
Put all together, your code should look something like this:
; AHK
RunWait, server.py "%ToSend%"
# Python
sys.exit(calcFunction(sys.argv[1]))
; AHK
MsgBox %ErrorLevel%

using python COM server, ahk can really calls python functions. directly.
you use it like this: MsgBox % pythonComServer.toUppercase("hello world")
simple example: return uppercased string
use the python part from How to program hotstrings in python like in autohotkey and use this for ahk part:
call python function uppercase.ahk
#NoEnv ; Recommended for performance and compatibility with future AutoHotkey releases.
#SingleInstance, force
SendMode Input ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir% ; Ensures a consistent starting directory.
SetBatchLines, -1
#KeyHistory 0
ListLines Off
pythonComServer:=ComObjCreate("Python.stringUppercaser")
;or
; pythonComServer:=ComObjCreate("{C70F3BF7-2947-4F87-B31E-9F5B8B13D24F}") ;use your own CLSID
MsgBox % pythonComServer.toUppercase("hello world")
Exitapp
f3::Exitapp
customized version: (math) use SymPy to simplify Expression
read this first to understand: How to program hotstrings in python like in autohotkey
sympy com server.py
from sympy import simplify, Number, N
from sympy.parsing.sympy_parser import standard_transformations, implicit_multiplication_application, convert_xor
from sympy.parsing.sympy_parser import parse_expr
from decimal import Decimal
from winsound import MessageBeep
transformations = standard_transformations + (implicit_multiplication_application, convert_xor)
def removeTrailingZerosFromNum(num):
dec = Decimal(str(num))
tup = dec.as_tuple()
delta = len(tup.digits) + tup.exponent
digits = ''.join(str(d) for d in tup.digits)
if delta <= 0:
zeros = abs(tup.exponent) - len(tup.digits)
val = '0.' + ('0' * zeros) + digits
else:
val = digits[:delta] + ('0' * tup.exponent) + '.' + digits[delta:]
val = val.rstrip('0')
if val[-1] == '.':
val = val[:-1]
if tup.sign:
return '-' + val
return val
def removeTrailingZerosFromExpr(operatorObject):
if operatorObject.args:
return type(operatorObject)(*[removeTrailingZerosFromExpr(i) for i in operatorObject.args])
else:
try:
return Number(removeTrailingZerosFromNum(operatorObject))
except:
return operatorObject
def removeTrailingZerosFromExprOrNumber(operatorObject):
try:
return removeTrailingZerosFromNum(operatorObject)
except:
return removeTrailingZerosFromExpr(operatorObject)
class BasicServer:
# list of all method names exposed to COM
_public_methods_ = ["parExprN"]
#staticmethod
def parExprN(clipBak):
parsed = parse_expr(clipBak, transformations=transformations)
simplified = simplify(N(parsed))
finalStr = str(removeTrailingZerosFromExprOrNumber(simplified))
MessageBeep(-1)
return finalStr.replace("**", "^")
if __name__ == "__main__":
import sys
if len(sys.argv) < 2:
print("Error: need to supply arg (""--register"" or ""--unregister"")")
sys.exit(1)
else:
import win32com.server.register
import win32com.server.exception
# this server's CLSID
# NEVER copy the following ID
# Use "print(pythoncom.CreateGuid())" to make a new one.
myClsid="{4530C817-6C66-46C8-8FB0-E606970A8DF6}"
# this server's (user-friendly) program ID, can be anything you want
myProgID="Python.SimplifyExpr"
import ctypes
def make_sure_is_admin():
try:
if ctypes.windll.shell32.IsUserAnAdmin():
return
except:
pass
exit("YOU MUST RUN THIS AS ADMIN")
if sys.argv[1] == "--register":
make_sure_is_admin()
import pythoncom
import os.path
realPath = os.path.realpath(__file__)
dirName = os.path.dirname(realPath)
nameOfThisFile = os.path.basename(realPath)
nameNoExt = os.path.splitext(nameOfThisFile)[0]
# stuff will be written here
# HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID\${myClsid}
# HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID\{c2467d33-71c5-4057-977c-e847c2286882}
# and here
# HKEY_LOCAL_MACHINE\SOFTWARE\Classes\${myProgID}
# HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Python.SimplifyExpr
win32com.server.register.RegisterServer(
clsid=myClsid,
# I guess this is {fileNameNoExt}.{className}
pythonInstString=nameNoExt + ".BasicServer", #sympy com server.BasicServer
progID=myProgID,
# optional description
desc="(math) SymPy simplify Expression",
#we only want the registry key LocalServer32
#we DO NOT WANT InProcServer32: pythoncom39.dll, NO NO NO
clsctx=pythoncom.CLSCTX_LOCAL_SERVER,
#this is needed if this file isn't in PYTHONPATH: it tells regedit which directory this file is located
#this will write HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID\{4530C817-6C66-46C8-8FB0-E606970A8DF6}\PythonCOMPath : dirName
addnPath=dirName,
)
print("Registered COM server.")
# don't use UseCommandLine(), as it will write InProcServer32: pythoncom39.dll
# win32com.server.register.UseCommandLine(BasicServer)
elif sys.argv[1] == "--unregister":
make_sure_is_admin()
print("Starting to unregister...")
win32com.server.register.UnregisterServer(myClsid, myProgID)
print("Unregistered COM server.")
else:
print("Error: arg not recognized")
to register:
python "sympy com server.py" --register
sympy com client.ahk
#NoEnv ; Recommended for performance and compatibility with future AutoHotkey releases.
#SingleInstance, force
SendMode Input ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir% ; Ensures a consistent starting directory.
SetBatchLines, -1
#KeyHistory 0
ListLines Off
sympyComServer:=ComObjCreate("Python.SimplifyExpr")
;or
; pythonComServer:=ComObjCreate("{4530C817-6C66-46C8-8FB0-E606970A8DF6}") ;use your own CLSID
; clipboard:=sympyComServer.parExprN("1+3*7")
clipboard:=sympyComServer.parExprN("1/3 + 1/2")
$#s::
clipboard:=sympyComServer.parExprN(clipboard)
return
f3::Exitapp

How to write data to a file every 10 seconds

I'm a JS dev trying to learn a bit of Python while working on a Raspberry-Pi3 project that reads data from a Bluetooth temperature sensor.
I need to write the data to my file.txt every 10 seconds, how could I do that please? I found similar topic here (Run certain code every n seconds ), but I don't know how to make it work in my current scenario.
#!/usr/bin/env python3
import argparse
import re
import logging
import sys
import time
from btlewrap import available_backends, BluepyBackend, GatttoolBackend, PygattBackend
from mitemp_bt.mitemp_bt_poller import MiTempBtPoller, \
MI_TEMPERATURE, MI_HUMIDITY, MI_BATTERY
def valid_mitemp_mac(mac, pat=re.compile(r"4C:65:A8:[0-9A-F]{2}:[0-9A-F]{2}:[0-9A-F]{2}")):
"""Check for valid mac adresses."""
if not pat.match(mac.upper()):
raise argparse.ArgumentTypeError('The MAC address "{}" seems to be in the wrong format'.format(mac))
return mac
def poll(args):
"""Poll data from the sensor."""
backend = _get_backend(args)
poller = MiTempBtPoller(args.mac, backend)
line1 = "Temperature: {}".format(poller.parameter_value(MI_TEMPERATURE))
line2 = "Humidity: {}".format(poller.parameter_value(MI_HUMIDITY))
print("Getting data from Mi Temperature and Humidity Sensor")
print("FW: {}".format(poller.firmware_version()))
print("Name: {}".format(poller.name()))
print("Battery: {}".format(poller.parameter_value(MI_BATTERY)))
print(line1)
print(line2)
f = open('file.txt', 'w')
f.write("%s \n %s \n" % (line1, line2))
f.close()
def _get_backend(args):
"""Extract the backend class from the command line arguments."""
if args.backend == 'gatttool':
backend = GatttoolBackend
elif args.backend == 'bluepy':
backend = BluepyBackend
elif args.backend == 'pygatt':
backend = PygattBackend
else:
raise Exception('unknown backend: {}'.format(args.backend))
return backend
def list_backends(_):
"""List all available backends."""
backends = [b.__name__ for b in available_backends()]
print('\n'.join(backends))
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--backend', choices=['gatttool', 'bluepy', 'pygatt'], default='gatttool')
parser.add_argument('-v', '--verbose', action='store_const', const=True)
subparsers = parser.add_subparsers(help='sub-command help', )
parser_poll = subparsers.add_parser('poll', help='poll data from a sensor')
parser_poll.add_argument('mac', type=valid_mitemp_mac)
parser_poll.set_defaults(func=poll)
parser_scan = subparsers.add_parser('backends', help='list the available backends')
parser_scan.set_defaults(func=list_backends)
args = parser.parse_args()
if args.verbose:
logging.basicConfig(level=logging.DEBUG)
if not hasattr(args, "func"):
parser.print_help()
sys.exit(0)
args.func(args)
if __name__ == '__main__':
main()

You can use the time module to pause the program for 10 seconds on each iteration;
from time import sleep
def func(n):
print(n+1)
for i in range(5):
func(i)
sleep(10)
>1
>2
>3
>4
>5
# (every 10 seconds)
However this will block the rest of the program running, although a simple multi-threading script to call the writing function would suffice.
And in relation to the code you are using, insert the sleep call within the poll function and wrap what you have there. If you want to loop the program 10 times then;
def poll(args):
"""Poll data from the sensor."""
for _ in range(10):
# code things
f = open('file.txt', 'a') # << Use append here or you will keep overwriting file contents
f.write('hello')
f.close()
sleep(10)
Or if you want it to run forever until you KeyboardInterrupt or exit somehow:
def poll(args):
"""Poll data from the sensor."""
while True:
# code things
f = open('file.txt', 'a') # << Use append here or you will keep overwriting file contents
f.write('hello')
f.close()
sleep(10)

you need some kind of loop that polls your sensor - I do not see one glancing over your code. You got while and for loops in JS as well - look them up in http://docs.python.org/3/tutorial if you are unsure about the syntax.
store the time you wrote to a variable , sleep a bit poll the next value, check if 10s passed, write if, else not. (or simply sleep 10s between polls if you do not want intermediate values printed
Readup about loops:
for statement
looping techniques
import time
def poll():
return time.time(), 42
last_write = None # when did we record to file last?
# loop for as long as you want - while True would loop endlessly
for _ in range(7):
t,c = poll() # call poll() to get time and temperature from mocked data reader
# check if enough time has passed
if last_write is None or (t-last_write) > 2: # check if new reading needed
with open("t.txt","a") as f:
f.write(f"{t} {c}\n")
last_write=t
print("in file ", t,c)
else:
print("just output ", t,c)
time.sleep(0.7) # sleep some
Output:
in file 1552978725.5224085 42 # ...25.5
just output 1552978726.2232893 42 # ...26.2 - not 2s passed
just output 1552978726.9241226 42 # ...26.9 - not 2s passed
in file 1552978727.6249442 42 # ...27.6 - 2.1s passed
just output 1552978728.3259027 42 # ...28.3 - not 2s passed
just output 1552978729.0267787 42 # ...29.0 - not 2s passed
in file 1552978729.7275977 42 # ...29.7 - 2.1s passed
More remarks:
use with open(filename,mode) as f: and scope the file opeations below it - it will autoclose your file after scope and handle exceptions by closing the filehandle as well.
Using mode "w" will truncate the file before writing to it - you might want to use append instead: reading and writing files

monitor function name error using simpy

very new to python and trying to use a simpy script I found online to queue times. I am getting a name error when I use "Monitor" - which I thought was part of SimPy. Is there somewhere else I should be importing monitor from?
Thanks in advance for the help!
See below:
#!/usr/bin/env python
from __future__ import generators
import simpy
from multiprocessing import Queue, Process
from random import Random,expovariate,uniform
# MMC.py simulation of an M/M/c/FCFS/inft/infty queue
# 2004 Dec updated and simplified
# $Revision: 1.1.1.5 $ $Author: kgmuller $ $Date: 2006/02/02 13:35:45 $
"""Simulation of an M/M/c queue
Jobs arrive at random into a c-server queue with
exponential service-time distribution. Simulate to
determine the average number and the average time
in the system.
- c = Number of servers = 3
- rate = Arrival rate = 2.0
- stime = mean service time = 1.0
"""
__version__='\nModel: MMC queue'
class Generator(Process):
""" generates Jobs at random """
def execute(self,maxNumber,rate,stime):
##print "%7.4f %s starts"%(now(), self.name)
for i in range(maxNumber):
L = Job("Job "+`i`)
activate(L,L.execute(stime),delay=0)
yield hold,self,grv.expovariate(rate)
class Job(Process):
""" Jobs request a gatekeeper and hold it for an exponential time """
def execute(self,stime):
global NoInSystem
arrTime=now()
self.trace("Hello World")
NoInSystem +=1
m.accum(NoInSystem)
yield request,self,server
self.trace("At last ")
t = jrv.expovariate(1.0/stime)
msT.tally(t)
yield hold,self,t
yield release,self,server
NoInSystem -=1
m.accum(NoInSystem)
mT.tally(now()-arrTime)
self.trace("Geronimo ")
def trace(self,message):
if TRACING:
print "%7.4f %6s %10s (%2d)"%(now(),self.name,message,NoInSystem)
TRACING = 0
print __version__
c = 3
stime = 1.0
rate = 2.0
print "%2d servers, %6.4f arrival rate,%6.4f mean service time"%(c,rate,stime)
grv = Random(333555) # RV for Source
jrv = Random(777999) # RV for Job
NoInSystem = 0
m=Monitor()
mT=Monitor()
msT=Monitor()
server=Resource(c,name='Gatekeeper')
initialize()
g = Generator('gen')
activate(g,g.execute(maxNumber=10,rate=rate,stime=stime),delay=0)
simulate(until=3000.0)
print "Average number in the system is %6.4f"%(m.timeAverage(),)
print "Average time in the system is %6.4f"%(mT.mean(),)
print "Actual average service-time is %6.4f"%(msT.mean(),)

You are currently receiving a name error because Monitor hasn't currently been defined within your script. In order to use the Monitor from within simpy, you will need to either change import simpy to from simpy import Monitor or append simpy.Monitor for the locations that you are currently using the Monitor function.
Ex:
#!/usr/bin/env python
from __future__ import generators
from simpy import Monitor
Or (lines 71-73):
m=simpy.Monitor()
mT=simpy.Monitor()
msT=simpy.Monitor()

How can I clear a line in console after using \r and printing some text?

For my current project, there are some pieces of code that are slow and which I can't make faster. To get some feedback how much was done / has to be done, I've created a progress snippet which you can see below.
When you look at the last line
sys.stdout.write("\r100%" + " "*80 + "\n")
I use " "*80 to override eventually remaining characters. Is there a better way to clear the line?
(If you find the error in the calculation of the remaining time, I'd also be happy. But that's the question.)
Progress snippet
#!/usr/bin/env python
import time
import sys
import datetime
def some_slow_function():
start_time = time.time()
totalwork = 100
for i in range(totalwork):
# The slow part
time.sleep(0.05)
if i > 0:
# Show how much work was done / how much work is remaining
percentage_done = float(i)/totalwork
current_running_time = time.time() - start_time
remaining_seconds = current_running_time / percentage_done
tmp = datetime.timedelta(seconds=remaining_seconds)
sys.stdout.write("\r%0.2f%% (%s remaining) " %
(percentage_done*100, str(tmp)))
sys.stdout.flush()
sys.stdout.write("\r100%" + " "*80 + "\n")
sys.stdout.flush()
if __name__ == '__main__':
some_slow_function()
Consoles
I use ZSH most of the time, sometimes bash (and I am always on a Linux system)

Try using the ANSI/vt100 "erase to end of line" escape sequence:
sys.stdout.write("\r100%\033[K\n")
Demonstration:
for i in range(4):
sys.stdout.write("\r" + ("."*i*10))
sys.stdout.flush()
if i == 3:
sys.stdout.write("\rDone\033[K\n")
time.sleep(1.5)
Reference: https://en.wikipedia.org/wiki/ANSI_escape_code#CSI_sequences

This is what I use
from msvcrt import putch, getch
def putvalue(value):
for c in str(value):
putch(c)
def overwrite(value):
""" Used to overwrite the current line in the command prompt,
useful when displaying percent or progress """
putvalue('\r'+str(value))
from time import sleep
for x in xrange(101):
overwrite("Testing Overwrite.........%s%% complete" % x)
sleep(.05)