Develop Reference

Python remote connection to Raspberry via SSH and mosquitto

I want add remote connection feature to my Raspberry PI device.
Program was written in python 3.5.
I use mosquitto to send command from my PC to Raspberry program which listen special "command_topic"
Command:
./remote.exp password ssh -R 19999:localhost:22 login#ip_host
remote.exp code:
#!/usr/bin/expect
set timeout 20
set cmd [lrange $argv 1 end]
set password [lindex $argv 0]
eval spawn $cmd
expect "assword:"
send "$password\r";
interact
I use class Process from module multiprocessing which runs
subprocess.checkout_output(command)
and when I run my program normally
"python3 main.py"
and then I use
mosquitto_pub -t "command_topic" -m "command"
it works. I get 19999 port on my PC but when I run my program in background
"nohup python3 main.py &"
I cannot get port and I don't know how to solve this?

Executing remote script locally using subprocess - Remote script uses vars from ~/.profile

I am executing a script that resides on the remote server.
This bash script makes use of a variable.
This variable is defined in ~/.profile.
For this purpose lets say it
$MYVAR=/a/b/c
So on remote server, or even ssh to remote and I execute
echo $MYVAR returns /a/b/c as you would expect.
But if I execute the remote script locally using python subprocess, the script fails. It fails as the script uses the $MYVAR which translates as soemthing incorrect.
This is because I am executing it via SSH, the ~./profile must not be getting loaded and instead it is using some other profile.
see here https://superuser.com/questions/207200/how-can-i-set-environment-variables-for-a-remote-rsync-process/207262#207262
Here is the command executed from a python script
ssh = subprocess.Popen(['ssh', '%s' % env.host, 'cd /script/dir | ./myscript arg1 arg2'],shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
My question is how can I run a script locally, that will ssh to remote, load the users ~/.profile then execute a bash script.

you can use paramiko module to run the scripts on remote server from locally.
http://www.paramiko.org/
Once installed , you can run the command as shown in example below
import paramiko
command=" 'ssh', '%s' % env.host, 'cd /script/dir | ./myscript arg1 arg2' "
ssh=paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('192.18.1.26',port=22,username='root',password='defassult') #This will connect to remote server
stdin,stdout,stderr=ssh.exec_command(command) #This will execute the command on remote server
output=stdout.readlines()
print '\n'.join(output)

The easiest solution for this was to create a sort of wrapper script like so
#!/bin/bash
. /users/me/.profile
cd /dir/where/script/exists/
exec script LIVE "$#"
So now then create a method in the python script to scp the wrapper script to tmp dir
scp wrapper user#remote:/tmp
Now ssh command in the question becomes
subprocess.Popen(['ssh', '%s' % env.host, env.installPatch],shell=False, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
env.installPatch translates to:
cd /tmp; ./wrapper 'patch_name'
Now the .profile is loaded and the patch script has the correct variable vals.
With using exec I get all the output back from the patch o/p and can write to file.
This was the cleanest solution for my case.

Python script under daemontools does not get signal 15

I have a python script which I want to run as a daemon under daemontools. Usually the service running under deamontools runs in foreground and if it goes down, daemontools will restart it. The service should also handle signals as described here.
My program catches SIGINT and SIGALRM and when the signals are received program exits.
When the program is run from command line and kill -1 <pid> and kill -15 <pid> is signaled, the signal handler is run, which causes program to exit, and corresponding logs are printed.
But when the program is run under daemontools, and svc -d /service/myrogram is executed, the program neither exits nor logs are printed.
I am using following run script to run the program under daemontools
#!/bin/sh
exec 2>&1
/usr/bin/python /home/test/sigtest.py
I am wondering why kill -15 <pid> works while svc -d /service/myrogram does not seems to deliver the signal to the python program.
The python script I am working with is:
from pyudev import Context, Monitor, MonitorObserver
import signal, sys, time
def print_device_event(device):
print ('New event - {0.action}: {0.device_path}'.format(device))
if __name__ == '__main__':
context = Context()
monitor = Monitor.from_netlink(context)
observer = MonitorObserver(monitor, callback=print_device_event, name='monitor-observer')
print ("Processing started")
observer.start()
def handler(signum, frame):
print("Received Signal: %d"%signum)
observer.send_stop()
print("Exiting.")
signal.signal(signal.SIGTERM, handler)
signal.signal(signal.SIGHUP, handler)
try:
while observer.is_alive():
observer.join(timeout=1.0)
except (KeyboardInterrupt, SystemExit):
print("Caught KeyboardInterrupt, exiting")
observer.send_stop()

I am using following run script to run the program under daemontools
#!/bin/sh
exec 2>&1
/usr/bin/python /home/test/sigtest.py
And there's your error. You're forking a child process to run the python program in. Stop your daemon from forking. Daemons do not need to, and should not, fork to run in a child process, be they running under daemontools, daemontools-encore, runit, s6, perp, or even systemd.
The standard operation of the shell, remember, is to fork and run commands in child processes.
For best results, you should acclimate yourself to writing run scripts using something other than the shell, that chain loads daemon programs. Here's your run script using Laurent Bercot's execline instead of /bin/sh as the script interpreter:
#!/command/execlineb -PW
fdmove -c 2 1
/usr/bin/python /home/test/sigtest.py
execline doesn't have any of the overhead of all of the things that even non-interactive shells do, including parsing startup "rc" files. And it comes with a useful suite of chain loading utilities for doing many of the things that one might want to do in run scripts.
But even execline does things that you don't need in a run script. Hence the -P option to turn off argument pushing, which has no utility for run scripts. Here's your run script using an even more barebones script interpreter, my nosh program:
#!/usr/local/bin/nosh
fdmove -c 2 1
/usr/bin/python /home/test/sigtest.py
Finally, here's your run script for /bin/sh, corrected:
#!/bin/sh
exec 2>&1
exec /usr/bin/python /home/test/sigtest.py
Further reading
https://superuser.com/a/708384/38062
Russ Allbery (2013-12-18). Bug#727708: upstart proposed policy in Debian. debian-ctte.
Jonathan de Boyne Pollard (2001). "Don't fork() in order to 'put the dæmon into the background'.". Mistakes to avoid when designing Unix dæmon programs. Frequently Given Answers.

process is not going background

I am running the python script shown below. The script does a ssh to the remote machine and runs a c program in the background. But on running the python script I get the following output:
This above means that a.out was run and the pid is [1] 2115 .
However whn I login to the remote machine and check for a.out via 'ps' command I dont see it.
Another observation is that when i add the delay statement in the python script thread.sleep(20) like , and while the script is still runnuing,
if I check in the remote machine, a.out is active.
#!/usr/bin/python
import HostMod #where ssh function is wrote
COMMAND_PROMPT1 = '[#$] '
p = HostMod.HostModule()
obj1=p.HostLogin('10.200.2.197', 'root', 'newnet') #ssh connection to remote system
obj1.sendline ('./a.out > test.txt &') #sending program to remote machine to executethere
obj1.expect (COMMAND_PROMPT1)
print obj1.before
#a.out program
int main()
{
while(1)
{
sleep(10);
}
return 0;
}

please try giving absolute path of ./a.out

Try using nohup
...
obj1.sendline ('nohup ./a.out > test.txt &') #sending program to remote machine to executethere
but you should really not use a shell to invoke commands over ssh. The ssh protocol has builtin support for running commands. I am not sure how your HostMod module works, but you could try this from your shell (would be easy to port to use subprocess):
ssh somehost nohup sleep 20
<Ctrl+C>
ssh somehost
ps ax | grep sleep
And you should see your sleep process still running. This method does not instantiate a shell, which is much more reliable, since you may or may not have control over which shell is run, what is in your ~/.(...)rc files, etc. All in all, much more portable.

Putting command in the background with Fabric does not work on some hosts

For testing purposes, I am running the following command, with plain ssh command line tool:
ssh user#host "nohup sleep 100 >> /tmp/xxx 2>&1 < /dev/null &"
This is working as expected, in all my hosts: a sleep process is created in the background, and the ssh finishes immediately.
I am trying to implement this functionality in python using Fabric. I end up doing a run call. This is what the Fabric logging is reporting:
[user#host] run: nohup sleep 100 >> /tmp/xxx 2>&1 < /dev/null &
Which is exactly what I expect. But if I check the processes which are running in my host, sleep 100 is not one of them. Worse yet: the problem happens only on some of my hosts.
I also added some more info to show what process has been created, by appending a "\necho $!" to the command to be run by Fabric. This is what was reported:
[user#host] run: nohup sleep 100 >> /tmp/xxx 2>&1 < /dev/null &
echo $!
[user#host] out: 30935
I am running out of ideas on how to debug this, since Fabric is reporting that the process has been created, but I see no process running in the other end. The syslog reports that an ssh session is being opened and closed:
Dec 6 09:12:09 host sshd[2835]: Accepted publickey for user from 67.133.172.14 port 37732 ssh2
Dec 6 09:12:09 host sshd[2838]: pam_unix(sshd:session): session opened for user user by (uid=0)
Dec 6 09:12:10 host sshd[2838]: pam_unix(sshd:session): session closed for user user
Could I somehow increase the amount of logging that the ssh daemon is producing, so that I can see at least what command is being requested via ssh?
I know that Fabric has some issues with running commands in the background, but that does not seem to be my problem. Could there be other issues with Fabric / ssh / background processes?
EDIT
I have installed dtach on all my systems. The version packaged in Ubuntu 8.04 is too old, and does not allow calling dtach -n over ssh (problems with terminal), so I had to download and compile the dtach sources. After doing that, I was able to run my command like this, with Fabric:
[user#host] run: dtach -n /tmp/Y sleep 100 >> /tmp/xxx 2>&1
This is working fine in all hosts. But this does not fit my scenario, because:
dtach creates two processes: one for dtach itself, another for the process being run.
I can not get the pid of the process being started

You are probably bumping into the infamous Fabric issue #395. The easiet workaround for these problems is to run your task with pty=False.