Develop Reference

Use Raspberry Pi 4B Terminal Command in Python Application

I am currently working on a small Python Application which turns a Raspberry Pi 4B into a 48 Channel Audio-Recorder. Basics work, but during Recording, I need a log file which tells me when recording started, which ALSA warnings occurred and when recording stopped.
The recorder can be started with this terminal command:
pi#raspberrypi:~ $ rec -q -t caf --endian little --buffer 96000 -c 48 -b 24 /home/pi/myssd-one/Aufnahmen/test.caf 2>&1 | tee /home/pi/myssd-one/Aufnahmen/logging.log
this records audio in the test.caf file and writes ALSA warnings to logging.log
So far so good.
The Python Program (which should run on a touchscreen with GUI so recording can easily started and stopped) takes care of variable audio-filenames (date-time-stamp) and controls an LED to show that recording is running.
This part of the code takes care of switching on and off:
#!/usr/bin/env python
from tkinter import *
import shlex
import os
import subprocess
import tkinter.font
import datetime
from gpiozero import LED
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BCM)
GPIO.setup(11, GPIO.OUT)
def ledToggle():
if led.is_lit:
led.off()
my_env = os.environ.copy()
my_env['AUDIODRIVER'] = 'alsa'
my_env['AUDIODEV'] = 'hw:KTUSB,0'
ledButton["text"] = "Turn Recorder on"
print ("recorder stops")
subprocess.Popen(['sudo', 'pkill', '-SIGINT', 'rec'], env = my_env, shell = FALSE, stdout=subprocess.PIPE)
else:
led.on()
my_env = os.environ.copy()
my_env['AUDIODRIVER'] = 'alsa'
my_env['AUDIODEV'] = 'hw:KTUSB,0'
ledButton["text"] = "Turn Recorder off"
print ("recorder starts")
##reference statement command line: "rec -q -t caf --endian little --buffer 96000 -c 48 -b 24 /home/pi/myssd-one/Aufnahmen/test.caf 2>&1 | tee /home/pi/myssd-one/Aufnahmen/logging.log"
command_line = shlex.split("rec '-q' '-t' 'caf' '--buffer' '96000' '-c 48' '-b 24' '/home/pi/myssd-one/Aufnahmen/test.caf' '"2>&1 | tee"' '/home/pi/myssd-one/Aufnahmen/logging.log'")
p1 = subprocess.Popen(command_line, env = my_env, shell = False, stdout=subprocess.PIPE)
I am trying to move the original command line statement into the subprocess.Popen command, to no success yet. The part where routing to the log file is done, fails. It looks as the initiating sox-application 'rec' tries to interpret it as part of its own parameter list, instead of interpreting it as a redirection of stdout and stderr to the log file. I appreciate some guidance in this issue.
Variable Filenames for audio files is already done, but for simplicity taken out of this code snippet.

Thanks Mark, I dived into this command line along your hint that it only can run with shell=True and this implied that it had to be written as a full statement without separating commas and escape quotes. Now it works. Actually, the shlex.split() becomes obsolete.

I cannot list up the BLE devices in the neighbourhood of my Raspberry Pi (python, btmgmt)

I want to scan the ble devices in the environment of my Raspberry, by using a python script called from a cron script.
But when I do this in cron (I mean I added to sudo crontab -e), I allways end up with an empty list.
when I am logged in as pi user - btmgmt works (only) fine with su permissions:
pi#Pluto:~ $ btmgmt find
Unable to start discovery. status 0x14 (Permission Denied)
pi#Pluto:~ $ sudo btmgmt find
Discovery started
hci0 type 7 discovering on
hci0 dev_found: 77:F8:D7:8A:1E:E5 type LE Random rssi -83 flags 0x0000
...
so in my python script I wrote:
flog.write("P01:\r\n")
out = subprocess.Popen(['sudo', '/usr/bin/btmgmt', 'find'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
stdout, stderr = out.communicate()
flog.write("stderr: " + str(stderr) + "\r\n")
cDvc = stdout.split('\n')
flog.write("Amount of lines = " + str(len(cDvc)) + "\r\n")
for line in cDvc:
line = line + '\r\n'
if debugflag:
print(line)
flog.write(line)
..
Running this script from the shell prompt works fine.. in the log-file (flog) I get: ...
P01:
stderr: None
Amount of lines = 40
Discovery started
hci0 type 7 discovering on
hci0 dev_found: 70:D0:FD:74:34:AC type LE Random rssi -59 flags 0x0000
AD flags 0x1a
..
Running this same script as a crontab -e line: no devices show up & I cannot find cause:
...
P01:
stderr: None
Amount of lines = 1
P02:
...
Can anyone help me out here?

If you use the BlueZ DBus API to get the information then you will not need to use sudo. It also avoids you having to use btmgmt as I am not sure it was intended for it to be scripted in that way
The documentation for the DBus API is available at:
https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/adapter-api.txt
https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc/device-api.txt
The pydbus library is very helpful for accessing the BlueZ DBus API: https://pypi.org/project/pydbus/
Some useful things to know to get you started:
The Dbus service for bluez is called 'org.bluez'
The default Bluetooth adapter normally has '/org/bluez/hci0' as its DBus object path.
BlueZ/DBus has an object manager which stores information about devices
I did the following script to test the idea:
from datetime import datetime
import os
import pydbus
from gi.repository import GLib
discovery_time = 60
log_file = '/home/pi/device.log'
# Create an empty log file
def write_to_log(address, rssi):
if os.path.exists(log_file):
open_mode = 'a'
else:
open_mode = 'w'
with open(log_file, open_mode) as dev_log:
now = datetime.now()
current_time = now.strftime('%H:%M:%S')
dev_log.write(f'Device seen[{current_time}]: {address} # {rssi} dBm\n')
bus = pydbus.SystemBus()
mainloop = GLib.MainLoop()
class DeviceMonitor:
def __init__(self, path_obj):
self.device = bus.get('org.bluez', path_obj)
self.device.onPropertiesChanged = self.prop_changed
print(f'Device added to monitor {self.device.Address}')
def prop_changed(self, iface, props_changed, props_removed):
rssi = props_changed.get('RSSI', None)
if rssi is not None:
print(f'\tDevice Seen: {self.device.Address} # {rssi} dBm')
write_to_log(self.device.Address, rssi)
def end_discovery():
"""Handler for end of discovery"""
mainloop.quit()
adapter.StopDiscovery()
def new_iface(path, iface_props):
"""If a new dbus interfaces is a device, add it to be monitored"""
device_addr = iface_props.get('org.bluez.Device1', {}).get('Address')
if device_addr:
DeviceMonitor(path)
# BlueZ object manager
mngr = bus.get('org.bluez', '/')
mngr.onInterfacesAdded = new_iface
# Connect to the DBus api for the Bluetooth adapter
adapter = bus.get('org.bluez', '/org/bluez/hci0')
adapter.DuplicateData = False
# Iterate around already known devices and add to monitor
mng_objs = mngr.GetManagedObjects()
for path in mng_objs:
device = mng_objs[path].get('org.bluez.Device1', {}).get('Address', [])
if device:
DeviceMonitor(path)
# Run discovery for discovery_time
adapter.StartDiscovery()
GLib.timeout_add_seconds(discovery_time, end_discovery)
print('Finding nearby devices...')
try:
mainloop.run()
except KeyboardInterrupt:
end_discovery()

I have exact the same issue. I need to use a raspberry pi to check if some specific bluetooth devices are alive nearby and to send a heartbeat to monitoring service.
I was not getting any output from sudo btmgmt find when the command was executed in a cron job like * * * * * sudo btmgmt find > /tmp/ble_devices.txt, neither if I was using python to capture the output from Popen call. So I asked myself if I could executed it into another screen, and it worked.
My solution is quite hackish. I did the following:
Installed the screen tool on raspberry pi: sudo apt install screen
Created a screen for running the scan command: screen -S blescan
Detached myself from screen ctrl+a+d
Created a shell script in /home/pi/scan_job with the content:
#!/bin/bash
cd <to python project> && ./<file to be executed>
Made it executable chmod +x /home/pi/scan_job
Set cronjob to execute the file in blescan screen:
*/10 * * * * screen -S blescan -X screen '/home/pi/scan_job'

Bad Display Name when running Python script on boot with Touch Screen

Attempting to run a Python script on boot on Raspberry Pi 3B+ 1GB RAM, Raspbian, with a SunFounder 10" Touch Screen, - .log file returns "Bad display name'
Python script is 100% functional when run via Terminal / executable script / Thonny etc. Attempted to run at boot first via rc.local - created a service, enabled service, daemon-reload... etc. Did not work.
Tried to run as crontab, same result - .log output from crontab shows "Bad display name". Thought it was lack of Display Environment imported and declared within the Python script, so I added that - but on boot returns the same result.
This is the Python Script I'm using
#!/usr/bin/env python3
import RPi.GPIO as GPIO
import os
import sys
import webbrowser
import time
import subprocess
from pynput import keyboard
from Xlib.display import Display
#GPIO Readout
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
#GPIO Header Setup
header = 2
GPIO.setup(header, GPIO.IN)
#Omxplayer Commands
Loop = 'omxplayer -b --loop --no-osd -o hdmi /home/pi/Videos/PlanetEarth.mp4 > /dev/null'
Donation = 'omxplayer -b --no-osd -o hdmi /home/pi/Videos/Cartoon.mp4 > /dev/null'
KillPlayer = 'pkill omxplayer.bin'
KillForm = 'pkill chromium'
#Set Display Environment
new_env = dict(os.environ)
new_env['DISPLAY'] = ':0.0'
#Form Handling Required Below
#If Donation is successful, Stop Looping Video, Open Form in Chromium, Wait 60 seconds, Close Chromium, Restart Loop
def PullDownSuccess():
subprocess.Popen(KillPlayer, env=new_env, shell=True)
time.sleep(2)
webbrowser.open('<url>')
time.sleep(60)
subprocess.Popen(KillForm, env=new_env, shell=True)
time.sleep(2)
subprocess.Popen(Loop, env=new_env, shell=True)
#Inception
subprocess.Popen(Loop, env=new_env, shell=True)
#Terminate Loop with Escape Key or Manually Initiate Donation Success
def on_press(key):
if key == keyboard.Key.ctrl:
PullDownSuccess()
if key == keyboard.Key.esc:
subprocess.Popen(KillPlayer, shell=True)
#Keyboard Listener Module
with keyboard.Listener(
on_press=on_press) as listener:
listener.join()
#Donation Successful Do:
while True:
header_state = GPIO.input(header)
if header_state == GPIO.HIGH:
PullDownSuccess()
I am currently attempting to run this script on Boot via crontab with this line:
#reboot (/bin/sleep 10; /usr/bin/python3 /home/pi/Custom_Scripts/<script>.py > /home/pi/Custom_Scripts/cronjoblog 2>&1)
The error log file for crontab returns the following:
raise error.DisplayNameError(display)
Xlib.error.DisplayNameError: Bad display name ""
This error only exists when attempting to run script on boot up. Is the Display overriding the boot display permissions on Boot Up? What is keeping the script from running on the Display on boot up, but not when remotely executed? Thank you for your consideration.
Update: Still no solution. Display environment returns ":0.0' ... so far I have tried to remove
> /dev/null from #Omxplayer Commands
Replacing crontab startup line to:
DISPLAY=":0" /usr/bin/python3 /home/pi/Custom_Scripts/<script>.py
and
DISPLAY=":0.0" /usr/bin/python3 /home/pi/Custom_Scripts/<script>.py
And any possible combination of these.
Confirmed the script is not waiting for any background processes as I have added delay (time.sleep) up to 30 seconds, as well as returning IP addresses, etc.
Returns either Bad Display Name still OR "Can't connect to display ":0": b'Invalid MIT-MAGIC-COOKIE-1 key"
Still looking for a solution if anyone has one.
EDIT:
Fixed using /LXDE-pi/autostart. Answer below.

Try to add DISPLAY environment variable before calling your script:
DISPLAY=":0" /usr/bin/python3 /home/pi/Custom_Scripts/<script>.py

DISPLAY="/dev/null" /usr/bin/python3 /home/pi/Custom_Scripts/<script>.py
Have you tried a longer sleep?

If you try to get your ip adress during your startup script with (for example) :
import socket
def get_ip_address():
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.connect(("8.8.8.8", 80))
return s.getsockname()[0]
you will get an errror like:
File "/usr/lib/python2.7/socket.py", line 228, in meth
return getattr(self._sock,name)(*args)
error: [Errno 101] Network is unreachable
This is because at reboot network is not necessarily available. In order to make it available
see https://raspberrypi.stackexchange.com/questions/45769/how-to-wait-for-networking-on-login-after-reboot :
sudo raspi-config
then select option 3 Boot Options
then select option 4 Wait for network at boot
Once you have done that networking should not be an issue at reboot.
You get also check for other ways to enforce network setup before running your script take a look at :
https://askubuntu.com/questions/3299/how-to-run-cron-job-when-network-is-up

If there are unforeseen ramifications for using this method, I will update this thread accordingly. I fixed this issue just by adding two lines to the autostart file which resides in /etc/sdg/lxsession/LXDE-pi/autostart.
sudo nano /etc/xgd/lxsession/LXDE-pi/autostart
Add these two lines, do not modify the existing code.
sleep 5
#/usr/bin/python3 /home/pi/Custom_Scripts/<script>.py
My autostart file looks like this:
#lxpanel --profile LXDE-pi
#pcmanfm --desktop --profile LXDE-pi
#xscreensaver -no-splash
point-rpi
sleep 5
#usr/bin/python3 /home/pi/Custom_Scripts/<script>/py
Ctrl + O to write file
sudo reboot

Continuous Python script and restart if ended

I am making a script that reads the output from a 433 MHz receiver over i2c on Arduino to an Raspberry Pi. I have tried to run start it from rc.local, but it seems like after a couple of days the script ends/stops/breaks/halts/is killed (?).
I have tried to run the following script from cron to determine if the runs or not and start the Python script if it has stopped. But it seems to not detect a running script and starts a new script every time it runs and so finally crashing the system.
#!/bin/bash
until <path to Python script>; do
sleep 1
done
exit(0)
I have also tried to replace the always true while statement with a statement that lets the script run for a minute and restart the script with cron each minute, but that also results in that the script does not end and a new process is started.
Do anyone have any idea of how I either can make the script stable or able to restart. I.e. Always running all time until infinity! :-)
Python script:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
#Run in correct folder
import os
os.system("<path to script-folder>")
import sys
import MySQLdb as mdb
from smbus import SMBus
import RPi.GPIO as GPIO
import datetime
import time
addr = 0x10
intPin = 4
bus = SMBus(1)
def readData():
val = bus.read_byte(addr)
raw = val << 24
val = bus.read_byte(addr)
raw = raw | (val << 16)
val = bus.read_byte(addr)
raw = raw | (val << 8)
val = bus.read_byte(addr)
raw = raw | val
# Tidstämpel
ts = time.time()
date = datetime.datetime.fromtimestamp(ts).strftime('%Y-%m-%d %H:%M:%S')
try:
con = mdb.connect('<server>', '<User>', '<password>', '<DB>')
con.autocommit(True)
cur = con.cursor()
cur.execute("INSERT INTO <DB statement>") # Data is stored as integers
except mdb.Error, e:
errorLog = open('<path to log>', 'a')
errorlog.write("Error %d: %s" % (e.args[0],e.args[1]) + "\n")
errorlog.close()
sys.exit(1)
finally:
if con:
cur.close()
con.close()
while True:
GPIO.setmode(GPIO.BCM)
GPIO.setup(intPin,GPIO.IN)
GPIO.wait_for_edge(intPin, GPIO.RISING)
readData()

use linux screen to run the python script it will always run in backend until you stop the script.
http://www.tecmint.com/screen-command-examples-to-manage-linux-terminals/

Consider using daemontools. It's small and will ensure your code runs...forever.

Python script not runnig on boot - raspberry

I made a python script on my raspberry - /home/pi/bin/script.py:
#!/usr/bin/python
from urllib2 import urlopen
from time import sleep
FILE = "publicip"
SITE = "http://ipecho.net/plain"
DELAY = 60 # seconds
def get_ip():
while True:
# wait for DELAY seconds
sleep(DELAY)
# get my current public ip
try:
ip = urlopen(SITE).read()
except IOError:
continue
# compare with the one in file
file_value = open(FILE).read()
if ip != file_value: # if they are not equal
open(FILE, "w").write(ip) # update the ip in file
if __name__ == "__main__":
get_ip()
It's propose is to get my public ip and store it in a file.
I need this script run in a loop so it can update the file as soon
the ip changes.
If the power fail I want it to run when the raspberry restarts. So,
I updated the /etc/rc.local file:
#!/bin/sh -e
/home/pi/bin/script.py
exit 0
After that I used sudo reboot to restart the raspberry. I'm using PuTTY
from a windows computer to connect to the raspberry. After logging in again
I used ps -e | grep script.py to see if my script was runnig but it was
not. Then I runned the script manually and it worked!
What would you do to solve this problem?

An alternative for running in the cron or init, it to use a userspace monitor.
This tutorial is great showing supervisor.
It is really easy to use.
apt-get install supervisor
service supervisor restart
add to /etc/supervisor/conf.d/ip_update.conf
[program:ip_update]
command=/home/pi/bin/script.py
autostart=true
autorestart=true
stderr_logfile=/var/log/ip_update.err.log
stdout_logfile=/var/log/ip_update.out.log
and you can still use supervisorctl to manage it:
$ supervisorctl
> restart ip_update

First verify your script execution's permission, if it's have a execution permission. After that, you need use & after command of script in (runs to infinite loops), try:
#!/bin/sh
/home/pi/bin/script.py &
More details in raspbian documentation.

Your other option is to use cron
sudo crontab -e will open the crontab for you
you can set your script to run as often as you like and if you put in the entry:
#reboot /home/pi/bin/script.py
it should run during the boot sequence
other non numeric options are:
#yearly Run once a year, "0 0 1 1 *".
#annually (same as #yearly)
#monthly Run once a month, "0 0 1 * *".
#weekly Run once a week, "0 0 * * 0".
#daily Run once a day, "0 0 * * *".
#midnight (same as #daily)
#hourly Run once an hour, "0 * * * *"
The standard entry is:
# Minute Hour Day of Month Month Day of Week Command
0 * * * * /home/pi/bin/script.py
#Once an hour on the hour
* * * * * /home/pi/bin/script.py
#Every minute
Edit:
with reference to your comment, performing it with cron, means that you should take out the timing within your code, as that is what cron is doing. So you would end up up with something like:
#!/usr/bin/python
from urllib2 import urlopen
FILE = "publicip"
SITE = "http://ipecho.net/plain"
def get_ip():
try:
ip = urlopen(SITE).read()
except IOError:
continue
# compare with the one in file
file_value = open(FILE).read()
if ip != file_value: # if they are not equal
open(FILE, "w").write(ip) # update the ip in file
if __name__ == "__main__":
get_ip()
With reference to your existing code, I notice that you are never closing the file, just an endless loop of opening it with read and then opening it with write, I'm not sure quite how python will handle that offhand but it certainly isn't good practice.