First of all some context:
Four MPR121 Breakout Boards (https://www.sparkfun.com/products/9695) connected via i2C to a Raspberry Pi 2. A python script reads the data from the four boards and sends it to pure data with pdsend.
At the moment I have managed to get all the data I need to print nicely on the terminal. However, I am not sure how to get the same in pure data as I am getting text messages only (something like "print: .join(map(str print: diff3))")
I am pretty sure I need to change the os.system line to accomodate for the variables but I can't find how to do this.
Thank you very much in advance.
def send2Pd (message=' '):
os.system("echo '" + message + "' | pdsend 3000");
while True:
diff1 = [cap1.baseline_data(i)-cap1.filtered_data(i) for i in range(12)]
print 'Diff1:', '\t'.join(map(str, diff1))
send2Pd ('.join(map(str, diff1));')
diff2 = [cap2.baseline_data(i)-cap2.filtered_data(i) for i in range(12)]
print 'Diff2:', '\t'.join(map(str, diff2))
send2Pd ('.join(map(str, diff2));')
diff3 = [cap3.baseline_data(i)-cap3.filtered_data(i) for i in range(12)]
send2Pd ('.join(map(str, diff3));')
print 'Diff3:', '\t'.join(map(str, diff3))
diff4 = [cap4.baseline_data(i)-cap4.filtered_data(i) for i in range(12)]
print 'Diff4:', '\t'.join(map(str, diff4))
send2Pd ('.join(map(str, diff4));')
time.sleep(0.1)
If I understand correctly, you just need to remove the quotes from the arguments you pass to send2Pd, does something like send2Pd('\t'.join(map(str, diff1))) work?
OK sorted.
This code gives me one line for each cap in pure data.
However, it does seem very resource hungry (the usage monitor marks around 50%) and that is without running the pd patch. Is there a simple way of making this more efficient?
Thank you!
import os
import sys
import time
import captouch as MPR121
# Create MPR121 instances
cap1 = MPR121.MPR121()
cap2 = MPR121.MPR121()
cap3 = MPR121.MPR121()
cap4 = MPR121.MPR121()
# Initialize communication with all 4 MPR121s
cap1.begin( 0x5a )
cap2.begin( 0x5b )
cap3.begin( 0x5d )
cap4.begin( 0x5c )
# Define send2Pd function
def send2Pd (message=' '):
os.system("echo '" + message + "' | pdsend 3000");
# Start loop
while True:
# Filtered and baseline data are commented out
# filtered = [cap1.filtered_data(i) for i in range(12)]
# print 'Filt:', '\t'.join(map(str, filtered))
# base = [cap1.baseline_data(i) for i in range(12)]
# print 'Base:', '\t'.join(map(str, base))
# Difference for all 4 boards are calculated, printed in terminal and sent to Pure Data
diff1 = [cap1.baseline_data(i)-cap1.filtered_data(i) for i in range(12)]
print 'Diff1:', '\t'.join(map(str, diff1))
send2Pd ('diff1: ' + '\t'.join(map(str, diff1)) + ';')
diff2 = [cap2.baseline_data(i)-cap2.filtered_data(i) for i in range(12)]
print 'Diff2:', '\t'.join(map(str, diff2))
send2Pd ('diff2: ' + '\t'.join(map(str, diff2)) + ';')
diff3 = [cap3.baseline_data(i)-cap3.filtered_data(i) for i in range(12)]
print 'Diff3:', '\t'.join(map(str, diff3))
send2Pd ('diff3: ' + '\t'.join(map(str, diff3)) + ';')
diff4 = [cap4.baseline_data(i)-cap4.filtered_data(i) for i in range(12)]
print 'Diff4:', '\t'.join(map(str, diff4))
send2Pd ('diff4: ' + '\t'.join(map(str, diff4)) + ';')
# Short pause before repeating loop
time.sleep(0.1)
Related
I have the following python script. In it, I am iterating through a CSV file which has rows and rows of loyalty cards. In many cases, there is more than one entry per card. I am currently looping through each row, then using loc to find all other instances of the card in the current row, so I can combine them together to post to an API. What I'd like to do however, is when that post is done, remove all the rows I've just merged, so that way the iteration doesn't hit them again.
That's the part I'm stuck on. Any ideas? Essentially I want to remove all the rows in card_list from csv before I go for the next iteration. That way even though there might be 5 rows with the same card number, I only process that card once. I tried by using
csv = csv[csv.card != row.card]
At the end of the loop, thinking it might re-generate the dataframe without any rows with a card matching the one just processed, but it didn't work.
import urllib3
import json
import pandas as pd
import os
import time
import pyfiglet
from datetime import datetime
import array as arr
for row in csv.itertuples():
dt = datetime.now()
vouchers = []
if minutePassed(time.gmtime(lastrun)[4]):
print('Getting new token...')
token = get_new_token()
lastrun = time.time()
print('processing ' + str(int(row.card)))
card_list = csv.loc[csv['card'] == int(row.card)]
print('found ' + str(len(card_list)) + ' vouchers against this card')
for row in card_list.itertuples():
print('appending card ' + str(int(row.card)) + ' voucher ' + str(row.voucher))
vouchers.append(row.voucher)
print('vouchers, ', vouchers)
encoded_data = json.dumps({
"store_id":row.store,
"transaction_id":"11111",
"card_number":int(row.card),
"voucher_instance_ids":vouchers
})
print(encoded_data)
number += 1
r = http.request('POST', lcs_base_path + 'customer/auth/redeem-commit',body=encoded_data,headers={'x-api-key': api_key, 'Authorization': 'Bearer ' + token})
response_data = json.loads(r.data)
if (r.status == 200):
print (str(dt) + ' ' + str(number) + ' done. processing card:' + str(int(row.card)) + ' voucher:' + str(row.voucher) + ' store:' + str(row.store) + ' status: ' + response_data['response_message'] + ' request:' + response_data['lcs_request_id'])
else:
print (str(dt) + ' ' + str(number) + 'done. failed to commit ' + str(int(row.card)) + ' voucher:' + str(row.voucher) + ' store:' + str(row.store) + ' status: ' + response_data['message'])
new_row = {'card':row.card, 'voucher':row.voucher, 'store':row.store, 'error':response_data['message']}
failed_csv = failed_csv.append(new_row, ignore_index=True)
failed_csv.to_csv(failed_csv_file, index=False)
csv = csv[csv.card != row.card]
print ('script completed')
print (str(len(failed_csv)) + ' failed vouchers will be saved to failed_commits.csv')
print("--- %s seconds ---" % (time.time() - start_time))
First rule of thumb is never alternate what you are iterating on. Also, I think you are doing it wrong with itertuples. Let's do groupby:
for card, card_list in csv.groupby('card'):
# card_list now contains all the rows that have a specific cards
# exactly like `card_list` in your code
print('processing, card)
print('found', len(card_list), 'vouchers against this card')
# again `itertuples` is over killed -- REMOVE IT
# for row in card_list.itertuples():
encoded_data = json.dumps({
"store_id": card_list['store'].iloc[0], # same as `row.store`
"transaction_id":"11111",
"card_number":int(card),
"voucher_instance_ids": list(card_list['voucher']) # same as `vouchers`
})
# ... Other codes
Here is my list, So models [0], names[0], mac_addresses[0] would all be required to calculate my mac address that I need
models = ["MR18","MR32", "MR18"]
names = ["David", "Bob", "Frank"]
mac_addresses = ["00:18:0A:2C:3D:5F", "00:18:0A:2d:3c:5F", "00:18:0A:2A:3B:5F"]
These are the functions that should run depending on which if statement is True.
def calc18(mac_address, name, mr):
#Mac Address Breakdown
print(name)
mac_calc = mac_address[:2]
mac_extractor_front = mac_address[2:6]
mac_extractor_back = mac_address[8:]
flag = True
First_Pos_Hex = first_hex_calc(mac_calc, mr, flag)
#Initial Mac Addresses
list_2_4.append(mac_address)
list_5.append(First_Pos_Hex + mac_extractor_front + mr_18_5ghz + mac_extractor_back)
flag = False
#First Time Calculation hex updated
hex_updater = first_hex_calc(mac_calc, mr, flag)
list_2_4.append(hex_updater + mac_extractor_front + mr_18_24ghz + mac_extractor_back)
list_5.append(hex_updater + mac_extractor_front + mr_18_5ghz + mac_extractor_back)
#Update self, after appending mac addresses
for i in range(15):
counter = i + 1
hex_updater = hex_calc(hex_updater, mr)
list_2_4.append(hex_updater + mac_extractor_front + mr_18_24ghz + mac_extractor_back)
list_5.append(hex_updater + mac_extractor_front + mr_18_5ghz + mac_extractor_back)
print(str(counter) + ") 2.4ghz: " + list_2_4[i] + "\t" + str(counter) + ") 5 Ghz: " + list_5[i] )
for i in range(len(list_2_4)):
writer(name, mac_address, list_2_4[i], list_5[i], i)
def calc32(mac_address, name):
#Mac Address Breakdown
mac_calc = mac_address[15:17]
mac_extractor_front = mac_address[:6]
mac_extractor_back = mac_address[8:15]
#Initial Mac Addresses
list_2_4.append(mac_extractor_front + mr_32_24ghz + mac_extractor_back + mac_calc)
list_5.append(mac_extractor_front + mr_32_5ghz + mac_extractor_back + mac_calc)
#Update self, after appending mac addresses
for i in range(15):
counter = i + 1
mac_calc = hex_calc(mac_calc, mr)
list_2_4.append(mac_extractor_front + mr_32_24ghz + mac_extractor_back + mac_calc)
list_5.append(mac_extractor_front + mr_32_5ghz + mac_extractor_back + mac_calc)
print(str(counter) + ") 2.4ghz: " + list_2_4[i] + "\t" + str(counter) + ") 5 Ghz: " + list_5[i] )
writer(name, mac_address, list_2_4[i], list_5[i], i)
Now I have this for-loop should iterate through each position in models, which would then check the if statements and execute a specific function
so, the first iteration models[0] which has "MR18" stored inside, then would check the if statements and execute calc18(),
after calc18() function finishes its execution would return to the for-loop and run the next iteration which would be models[1] which has "MR32", then again would check the if statements and execute calc32(). Then when calc32() finishes executing move on to models[2]
But in my case when I run the code, models[2] which represents "MR18" runs through the if statement, completely ignoring models[0] and models[1]
for num, mod in enumerate(models):
print(mod)
if mod == "MR18":
print("I have entered")
calc18(mac_addresses[num], names[num], mod)
if mod == "MR32":
print("I have entered 32")
calc32(mac_addresses[num], names[num])
If this is still confusing please let me know, I'm not sure if pictures are allowed, I can draw a visual example if that's allowed :(
As your code is not runnable on my system, I run a simpler version of code to illustrate that your code should indeed go through each element of models.
models = ["MR18","MR32", "MR18"]
names = ["David", "Bob", "Frank"]
mac_addresses = ["00:18:0A:2C:3D:5F", "00:18:0A:2d:3c:5F", "00:18:0A:2A:3B:5F"]
def calc18(mac_address, name, mr):
print("calc18 running")
print(mac_address)
print(name)
print(mr)
def calc32(mac_address, name):
print("calc32 running")
print(mac_address)
print(name)
for num, mod in enumerate(models):
print(num)
print(mod)
if mod == "MR18":
print("I have entered")
calc18(mac_addresses[num], names[num], mod)
if mod == "MR32":
print("I have entered 32")
calc32(mac_addresses[num], names[num])
produces:
0
MR18
I have entered
calc18 running
00:18:0A:2C:3D:5F
David
MR18
1
MR32
I have entered 32
calc32 running
00:18:0A:2d:3c:5F
Bob
2
MR18
I have entered
calc18 running
00:18:0A:2A:3B:5F
Frank
MR18
The code does go through each element of the list. The problem shouldn't be with the for loop.
I see writer in your code, of which you might be writing to a file. Do check if it is set to overwrite mode or append mode.
I have a wireless sensor node that transmits wirelessly to a receiver connected to my computer. I use this command to see the data in real time
sudo stdbuf -o0 ./pip_sense.v2 l l | stdbuf -o0 grep -P "TX:03(376|004)
I am new to Python, but I was able to develop a piece of code as show below to capture the stdout and takes frames from a camera connected to the same computer. That is when a packet (data) is received, at that timestamp, a frame will be captured.
It is working fine, except the fact that its running slow. I am running my sensor at a rate of 10 Hz (transmitting every 0.1 sec.) but the code does what is required at a slower rate. I know that is because I am using cv2.imwrite() (I commented out this part and that helped in reducing the delay). But also, as discussed in this post and its said (as I understand) that using shell=True can invoke extra shell process and thus increase the delay.
Also, looking at this, Popen() seems to be causing delays as well. A solution is proposed in the second link, but I do not really understand how to modify it to work for me.
#!/usr/bin/env python
from subprocess import Popen, PIPE
import time
import sys
import cv2
import os
import csv
import argparse
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
from collections import OrderedDict
from datetime import datetime
PIP_CMD = 'sudo stdbuf -o0 ./pip_sense.v2 l l | stdbuf -o0 grep -P "TX:03(376|004)"'
def run(command):
process = Popen(command, stdout=PIPE, shell=True)
while True:
line = process.stdout.readline().rstrip()
if not line:
break
yield line
print ('Starting...')
def createFolder(directory):
try:
if not os.path.exists(directory):
os.makedirs(directory)
except OSError:
print ('Error: Creating directory. ' + directory)
createFolder('/home/piptag/pip-test/capture_webcam/EXPERIMENTS')
file1 = open('/home/piptag/pip-test/capture_webcam/EXPERIMENTS/3376.txt','w')
file1.write("Time Stamp \t TX ID \t RSSI \tSEQ NO \tCAP \tX-Axis \tY-Axis \tZ-Axis \t X \t Y \t Z \n") # Header line
file1.write("-------------------------------------------------------------------------------------------------------------------------------------\n")
file2 = open('/home/piptag/pip-test/capture_webcam/EXPERIMENTS/3004.txt','w')
file2.write("Time Stamp \t TX ID \t RSSI \tSEQ NO \tX-Axis \tY-Axis \tZ-Axis \t X \t Y \t Z \n") # Header line
file2.write("-------------------------------------------------------------------------------------------------------------------------------------\n")
dirname = "/home/piptag/pip-test/capture_webcam/EXPERIMENTS/"
def save_webcam(dirname):
cam = cv2.VideoCapture(-1) # was getting error (V4L: can't open camera by index 0), thus changes it to -1
jpg_quality = 75
frame_number = 0
if cam.isOpened():
ret_val, img = cam.read()
else:
ret_val = False
timestamp = int(TS)
path = dirname + str(tx) + str("-") + str(timestamp) + ".jpg"
cv2.imwrite(path, img, [int(cv2.IMWRITE_JPEG_QUALITY), jpg_quality])
frame_number = frame_number + 1
##this is the end of the camera program
def twos_comp(val, bits):
"""compute the 2's compliment of int value val"""
if (val & (1 << (bits - 1))) != 0: # if sign bit is set e.g., 8bit: 128-255
val = val - (1 << bits) # compute negative value
return val # return positive value as is
#########################################MAIN###################################
if __name__ == "__main__":
for line in run(PIP_CMD):
raw_data = line.split('\t')
if len(raw_data) > 1:
TS = raw_data[0][3:]
tx = raw_data[3].split(':')[-1]
rssi = float(raw_data[4][5:])
crc_ok = True if (raw_data[5] != b'BAD CRC') else False
no_drop = True if (raw_data[1] == b'Drop:0') else False
# If CRC check is ok and no drop, process the packet
if crc_ok and no_drop:
data = raw_data[-1].split(':')[-1].split()
cat = ""
for i in data:
cat += str(i)
if tx == '03376':
save_webcam(dirname)
print data
CapStatus=data[1]
if CapStatus == '50':
Cap='0'
elif CapStatus == '51':
Cap='1'
SEQNO1=str(int((data[2]),16))
x_axis1=data[3]+data[4]
y_axis1=data[5]+data[6]
z_axis1=data[7]+data[8]
TX1=tx
x1 = twos_comp(int(x_axis1,16), 16) * 0.0039 #* 9.80665 #the value is multiplied by 0.004 as the ADXL345 reports data as 4mg per 1lsb
y1 = twos_comp(int(y_axis1,16), 16) * 0.0039 #* 9.80665 #the value is multiplied by 0.004 as the ADXL345 reports data as 4mg per 1lsb
z1 = twos_comp(int(z_axis1,16), 16) * 0.0039 #* 9.80665 #the value is multiplied by 0.004 as the ADXL345 reports data as 4mg per 1lsb
st1 = str(TS) + "\t "+ "{:<5}".format(str (TX1)) + "\t" + "{:<10}".format(str (rssi)) + "{:<5}".format(SEQNO1) + "\t" + "{:<5}".format(str (Cap)) + "\t" + "{:<5}".format(str (x_axis1)) + "\t" + "{:<5}".format(str (y_axis1)) + "\t"+ "{:<5}".format(str(z_axis1)) + "\t" + "{:<5}".format(str(x1)) + "\t" + "{:<5}".format(str(y1)) + "\t" + "{:<5}".format(str(z1)) +"\n"
file1.write(st1)
elif tx == '03004':
save_webcam(dirname)
print data
SEQNO2=str(int((data[1]),16))
x_axis2=data[2]+data[3]
y_axis2=data[4]+data[5]
z_axis2=data[6]+data[7]
TX2=tx
x2 = twos_comp(int(x_axis2,16), 16) * 0.0039 #* 9.80665 #the value is multiplied by 0.004 as the ADXL345 reports dataas 4mg per 1lsb
y2 = twos_comp(int(y_axis2,16), 16) * 0.0039 #* 9.80665 #the value is multiplied by 0.004 as the ADXL345 reports data as 4mg per 1lsb
z2 = twos_comp(int(z_axis2,16), 16) * 0.0039 #* 9.80665 #the value is multiplied by 0.004 as the ADXL345 reports data as 4mg per 1lsb
st2 = str(TS) + "\t "+ "{:<5}".format(str (TX2)) +"\t "+ "{:<10}".format(str (rssi)) + "{:<5}".format(SEQNO2) + "\t" + "{:<5}".format(str (x_axis2)) + "\t"+ "{:<5}".format(str (y_axis2)) + "\t"+ "{:<5}".format(str(z_axis2))+ "\t"+"{:<5}".format(str(x2)) + "\t" + "{:<5}".format(str(y2))+ "\t"+ "{:<5}".format(str(z2)) +"\n"
file2.write(st2)
file1.close()
file2.close()
I appreciate any help
Thank you
Update:
I got this advice from a friend and it worked.
Calling the cam = cv2.VideoCapture(-1) outside the save_webcam function (as a global variable) reduced the delays a lot.
In this case, the camera will be turned on one time and then capturing images will be done each time save_webcam is called...hence, turning on and off the camera was slowing the process.
However, I would like to enhance the program more, as it get stuck sometimes. So, anyone understands what is in this post and how my stdout part should be modified, then please comment here. I appreciate this.
Why does my \n not work and how can i make it work
x = "text1"
c = "text2"
b=[]
for i in range(5):
xx=(c + " | " + x + "\n")
b.append(xx)
print (b)
if you just want to print out text instead of list then use str.join
# ...
print(''.join(b))
# or like this:
text = ''.join([(c + " | " + x + "\n") for _ in range(5)])
print(text)
# or python 3.6 style
text = ''.join(f"{c} | {x}\n" for _ in range(5))
This code works as expected. Output:
Loading
Loading.
Loading..
Loading...
Code:
done = False
count = 0
while not done:
print '{0}\r'.format("Loading"),
time.sleep(0.25)
print '{0}\r'.format("Loading."),
time.sleep(0.25)
print '{0}\r'.format("Loading.."),
time.sleep(0.25)
print '{0}\r'.format("Loading..."),
time.sleep(0.25)
count += 1
if count == 5:
done = True
And this code doesn't. Output:
Loading.
Loading...
Code:
done = False
count = 0
while not done:
print '{0}\r'.format("Loading"),
time.sleep(0.125)
print '{0}\r'.format("Loading."),
time.sleep(0.125)
print '{0}\r'.format("Loading.."),
time.sleep(0.125)
print '{0}\r'.format("Loading..."),
time.sleep(0.125)
count += 1
if count == 5:
done = True
Why does the time function seem to skip every second print statement if it is lower than 0.25?
Reason
Depending on the platform, Python buffers the output to different degrees.
For example, on Mac OSX there is no output at all even for your version with 0.25 seconds sleep.
Flushing manually
Flushing manually should work:
import sys
import time
done = False
count = 0
while not done:
for n in range(4):
print '{0}\r'.format("Loading" + n * '.'),
sys.stdout.flush()
time.sleep(0.125)
print ' ' * 20 + '\r',
count += 1
if count == 5:
done = True
You need to flush the output with sys.stdout.flush(). You also need to print empty spaces to make the dots "going back and forth":
print ' ' * 20 + '\r',
More minimal and cleaned up
This is shortened and a bit more general in terms of the shown text:
import sys
import time
text = 'Loading'
for _ in range(5):
for n in range(4):
print '{0}\r'.format(text + n * '.'),
sys.stdout.flush()
time.sleep(0.25)
nspaces = len(text) + n
print ' ' * nspaces + '\r',
Running unbuffered from the commandline
You can remove the line:
sys.stdout.flush()
if you run your script with the -u option:
python -u script_name.py
Note: This will have an effect on all print statements.