Develop Reference

Writing sensor read out with python while script runs in tmux

apologies if this question has been asked somewhere before, but I couldn't find a solution / info i was looking for.
So: I am running a raspberry pi 0W with a python script reading 4 temperature sensors monitoring some stuff at home. I am reading these sensors every 2 minutes and write the output into a CSV (for the time being sufficient for me). When I ssh my PI and run my script in TMUX, I realised that when calling my "Sensordata.csv" it only updates, once I close the script in my TMUX session. Ideally I want my "Sensordata.CSV" file update after each polling cycle.
I believe its not the code, as my code opens, writes and closes the file while running in the shell normally. Hope someone can help me out =)
import time
import csv
from datetime import datetime
def get_temp(dev_file):
f = open(dev_file,"r")
contents = f.readlines()
f.close()
index = contents[-1].find("t=")
if index != -1 :
temperature = contents[-1][index+2:]
cels =float(temperature)/1000
return cels
time_for_csv=time.asctime( time.localtime(time.time()))
f=open("Sensordata.csv", "a")
c=csv.writer(f)
if __name__ =="__main__":
while True:
dateTimeObj = datetime.now()
timestampStr = dateTimeObj.strftime("%d-%b-%Y (%H:%M:%S.%f)")
temp = get_temp("//sys/bus/w1/devices/28-0301a2799ec4/w1_slave")
temp2 = get_temp("//sys/bus/w1/devices/28-0301a2790081/w1_slave")
temp3 = get_temp("//sys/bus/w1/devices/28-012020be268d/w1_slave")
tempout = get_temp("//sys/bus/w1/devices/28-00000b0f6540/w1_slave")
print('Current Timestamp : ', timestampStr, "28-00000b0f6540 - Outside Sensor", tempout)
print('Current Timestamp : ', timestampStr, "28-0301a2799ec4 - Floorheating out", temp)
print('Current Timestamp : ', timestampStr, "28-0301a2790081 - Floorheating in", temp2)
print('Current Timestamp : ', timestampStr, "28-012020be268d - Room Sensor", temp3)
f = open("Sensordata.csv", "a")
c = csv.writer(f)
c.writerow([timestampStr, temp, temp2, temp3, tempout])
f.close()
time.sleep(120)

I don't think that Tmux is part of the issue.
Try removing these 3 lines of code in your script:
time_for_csv=time.asctime( time.localtime(time.time()))
f=open("Sensordata.csv", "a")
c=csv.writer(f)
time_for_csv is unused.
When opening a file for writing or appending, a safer way is to use the following:
with open("Sensordata.csv", "a") as f:
c=csv.writer(f)
c.writerow([timestampStr, temp, temp2, temp3, tempout])
The file object will always be closed even if an exception is raised. You do not have to close it explicitly. See With statement in Python
I am guessing that your opening a file object in the middle of the script leaves a file open, and then you re-define f after the if __name__ == "__main__. Leaving file objects open can have unpredictable results like the one you're experiencing.

Python String Slicing with Paramiko Output

Hi I'm building a python app that will simplify managing multiple Raspberry Pi's on my network. The idea is that it will speed up basic tasks like updates on the Pi's by issuing the commands to the Pi via the paramiko python module. I can save commands to the application and then run them via a simple shortcut rather than having to login and type them out.
I've hit a bit of a stumbling block in that my commands are running and I'm getting the output but because of the way I'm using paramiko every time a command runs I'm getting the full console output i.e. all the stuff the appears when you login over SSH in my output. To slim that down and only display the output I'm interested in (the result of the command I've run). I'm trying to use the string slicing method.
My code will check the output initially for the presence of :~$ using string.find() as the first intstance it will find will be just before the command runs I'm using that as the start point for the slice. At the minute I'm only looking to get rid of the output before the :~$. I'll probably then pass the result of the first slice to a new string and then look for user# in that string as the end point for the slice.
The issue I'm having is that once I've sliced the string I get no output whatsoever. Sometimes I'll get an output and that does appear to be correct in that its what I'm expecting to happen but most of the time I'm getting no output and sometime if I check the length of the string it will come back to say there are a number of characters in there but I get no output when I try to print the variable (rem1).
Below is my code, I'm issuing a command and then looping the script repeatedly until I either pick up on a second :~$ in the output indicating the command has finised running or until there is nothing in the output variable. On each loop the contents of output is transfered to final_output which I use as the full string to manipulate at the end.
This may be a long winded way of doing what I want to do but honestly I couldn't think of another way round it as I kept running into issues when trying to use interactive commands like apt full-upgrade
def interactive_command(connect, ip_addr, port, uname, passwd, admin, issue_cmd):
affirmative = {'yes', 'y', 'Y', 'Yes', 'ye', 'Ye', ''}
connect.connect(ip_addr, port, uname, passwd)
channel = connect.invoke_shell()
channel.set_combine_stderr(True)
stdin = channel.makefile('wb')
channel.send(issue_cmd + '\n')
if admin != -1:
print('Sudo detected. Attempting to elevate privileges...')
channel.send(passwd + '\n')
else:
pass
fin_check = {':~$', ':~ $'}
ser_check = {'[Y/n]', '(Y/I/N/O/D/Z) [default=N] ?'}
counter = 0
prompt_count = 0
running = True
monitoring = False
final_output = ''
while running:
stdin
counter = counter + 1
#print('Loop Counter: ' + str(counter))
# allow for 10 while loops to be completed before raising an error.
if counter == 10:
print('Unable to establish whether the command has finished running')
running = False
print('Counter: ' + str(counter))
print('Prompt Count: ' + str(prompt_count))
#print('Prompt Count: ' + str(prompt_count))
sleep(2)
if channel.recv_ready():
output = channel.recv(65535).decode('utf-8')
else:
running = False
monitoring = False
print('No output detected. Printing last output.')
break
final_output = final_output + output
#print(len(output))
search = str(output)
finish = -1
if search == -1 or finish == -1:
for y in ser_check:
search = str(output).find(y)
if search != -1:
print(output)
resp = input('Response?: ') + '\n'
channel.send(resp)
running = False
monitoring = True
next
for x in fin_check:
if x in output:
prompt_count = prompt_count + 1
#print('Prompt Count: ' + str(prompt_count))
if prompt_count == 2:
finish = 1
break
else:
finish = -1
if finish != -1:
print('Command Finished')
#print('Prompt Count: ' + str(prompt_count))
running = False
if monitoring:
pass
else:
# print(output)
break
next
sleep(3)
while monitoring:
output = channel.recv(65535).decode('utf-8')
final_output = final_output + output
for x in fin_check:
if x in output:
prompt_count = prompt_count + 1
#print('Prompt Count: ' + str(prompt_count))
if prompt_count == 2:
finish = 1
else:
finish = -1
if finish != -1:
print('Command Finished')
#print('Prompt Count: ' + str(prompt_count))
running = False
monitoring = False
#print(output)
next
if prompt_count == 2:
print('Prompt Count Reached. Command Finished')
break
sleep(3)
print()
#print(final_output)
for c in fin_check:
start_point = final_output.find(c)
if start_point == -1:
pass
else:
#print(len(final_output))
#print(start_point)
start_point = start_point + len(c)
#print(start_point)
next
rem1 = final_output[start_point:]
print(len(rem1))
print(rem1)
print()
complete = input('Are you finished with this connection? ')
if complete in affirmative:
complete = 'Y'
return complete

How to I make this program write the changes into the original or a new copy of the input file?

This program works for me except the changes that are made to the input .srt (subtitle) file are displayed but NOT saved back in the input file or a new file. Can anyone please tell me what is missing? Thanks. Stuart.
code follows:
#! /usr/bin/python
import sys;
# Patching SRT files to make them readable on Samsung TVs
# It basically inserts blank subtitles when silence should occur.
seqNo=1
try:
subs = open(sys.argv[1])
except:
print "Please provide subtile file to process"
sys.exit(1)
while True:
srtSeqNo=subs.readline();
try:
begin,arrow,end=subs.readline().rstrip('\n\r').split(" ")
except:
break
srtText = subs.readline();
again = subs.readline();
while len(again.strip('\n\r')) > 0:
srtText = srtText + again
again = subs.readline()
print "%d\n%s --> %s\n%s" % (seqNo, begin, end, srtText)
seqNo = seqNo + 1
print "%d\n%s --> %s\n%s\n" % (seqNo, end, end, " ")
seqNo = seqNo + 1

Python refresh file from disk

I have a python script that calls a system program and reads the output from a file out.txt, acts on that output, and loops. However, it doesn't work, and a close investigation showed that the python script just opens out.txt once and then keeps on reading from that old copy. How can I make the python script reread the file on each iteration? I saw a similar question here on SO but it was about a python script running alongside a program, not calling it, and the solution doesn't work. I tried closing the file before looping back but it didn't do anything.
EDIT:
I already tried closing and opening, it didn't work. Here's the code:
import subprocess, os, sys
filename = sys.argv[1]
file = open(filename,'r')
foo = open('foo','w')
foo.write(file.read().rstrip())
foo = open('foo','a')
crap = open(os.devnull,'wb')
numSolutions = 0
while True:
subprocess.call(["minisat", "foo", "out"], stdout=crap,stderr=crap)
out = open('out','r')
if out.readline().rstrip() == "SAT":
numSolutions += 1
clause = out.readline().rstrip()
clause = clause.split(" ")
print clause
clause = map(int,clause)
clause = map(lambda x: -x,clause)
output = ' '.join(map(lambda x: str(x),clause))
print output
foo.write('\n'+output)
out.close()
else:
break
print "There are ", numSolutions, " solutions."

You need to flush foo so that the external program can see its latest changes. When you write to a file, the data is buffered in the local process and sent to the system in larger blocks. This is done because updating the system file is relatively expensive. In your case, you need to force a flush of the data so that minisat can see it.
foo.write('\n'+output)
foo.flush()

I rewrote it to hopefully be a bit easier to understand:
import os
from shutil import copyfile
import subprocess
import sys
TEMP_CNF = "tmp.in"
TEMP_SOL = "tmp.out"
NULL = open(os.devnull, "wb")
def all_solutions(cnf_fname):
"""
Given a file containing a set of constraints,
generate all possible solutions.
"""
# make a copy of original input file
copyfile(cnf_fname, TEMP_CNF)
while True:
# run minisat to solve the constraint problem
subprocess.call(["minisat", TEMP_CNF, TEMP_SOL], stdout=NULL,stderr=NULL)
# look at the result
with open(TEMP_SOL) as result:
line = next(result)
if line.startswith("SAT"):
# Success - return solution
line = next(result)
solution = [int(i) for i in line.split()]
yield solution
else:
# Failure - no more solutions possible
break
# disqualify found solution
with open(TEMP_CNF, "a") as constraints:
new_constraint = " ".join(str(-i) for i in sol)
constraints.write("\n")
constraints.write(new_constraint)
def main(cnf_fname):
"""
Given a file containing a set of constraints,
count the possible solutions.
"""
count = sum(1 for i in all_solutions(cnf_fname))
print("There are {} solutions.".format(count))
if __name__=="__main__":
if len(sys.argv) == 2:
main(sys.argv[1])
else:
print("Usage: {} cnf.in".format(sys.argv[0]))

You take your file_var and end the loop with file_var.close().
for ... :
ga_file = open(out.txt, 'r')
... do stuff
ga_file.close()
Demo of an implementation below (as simple as possible, this is all of the Jython code needed)...
__author__ = ''
import time
var = 'false'
while var == 'false':
out = open('out.txt', 'r')
content = out.read()
time.sleep(3)
print content
out.close()
generates this output:
2015-01-09, 'stuff added'
2015-01-09, 'stuff added' # <-- this is when i just saved my update
2015-01-10, 'stuff added again :)' # <-- my new output from file reads
I strongly recommend reading the error messages. They hold quite a lot of information.
I think the full file name should be written for debug purposes.

Python - Changing File Object within function

Sorry - My questions is how can I change a file object within a function from a different function?
I've been trying to work out this error in my first python script for too long now, Dr Google and the forums aren't helping me too much, but I'm hoping you can.
I have a looping function that generates alot of data and I would like to output it to a text file, and create a new text file after the third loop.
I have 2 functions defined, one to create the data hashes, the other to create the new files.
The new files are being created as expected (aaa.txt, baa.txt...etc) but the "hashit" function only ever writes to the first file (aaa.txt) even though the others are being created.
I have tried fo.close() fo.flush(), as well as referencing fo in the functions but can't seem to make it work. Also I've moved the fo.write from the function to the main body.
I have included a cut down version of the code that I've been using to troubleshoot this issue, the real one has several more loops increasing the string length.
Thanks in advance
import smbpasswd, hashlib
base = '''abcdefghijklmnopqrstuvwxyz '''
# base length 95
print(base)
baselen = len(base)
name = 'aaa.txt'
fo = open(name, "w")
print "Name of the file: ", fo.name
print "Closed or not : ", fo.closed
print "Opening mode : ", fo.mode
print "Softspace flag : ", fo.softspace
pw01 = 0
pw02 = 0
pw03 = 0
def hashit(passwd):
#2
# Need to install module
# sudo apt-get install python-smbpasswd
hex_dig_lm = smbpasswd.lmhash(passwd)
hex_dig_ntlm = smbpasswd.nthash(passwd)
#print '%s:%s' % smbpasswd.hash(passwd)
hash_md5 = hashlib.md5(passwd)
hex_dig_md5 = hash_md5.hexdigest()
print(passwd)
print(hex_dig_lm)
print(hex_dig_ntlm)
print(hex_dig_md5)
hashstring = passwd +","+ hex_dig_lm +","+ hex_dig_md5 + '\n'
fo.write(hashstring);
def newfile(name):
fo.flush()
fo = open(name, "a")
print("-------newfile------")
print "Name of the file: ", fo.name
print "Closed or not : ", fo.closed
print('NewFile : ' + name)
raw_input("\n\nPress the enter key to exit.")
# add 3rd digit
while (pw03 < baselen):
pwc03 = base[pw03]
name = pwc03 + 'aa.txt'
fo.close
newfile(name);
pw03 += 1
while (pw02 < baselen):
pwc02 = base[pw02]
pw02 += 1
while (pw01 < baselen):
pwc01 = base[pw01]
pw01 += 1
passwd = pwc03 + pwc02 + pwc01
hashit(passwd);
else:
pw01 = 0
else:
pw02 = 0
else:
pw03 = 0

In your newfile() function, add this line first:
global fo