Develop Reference

Log output of background process to a file

I have time consuming SNMP walk task to perform which I am running as a background process using Popen command. How can I capture the output of this background task in a log file. In the below code, I am trying to do snampwalk on each IP in ip_list and logging all the results to abc.txt. However, I see the generated file abc.txt is empty.
Here is my sample code below -
import subprocess
import sys
f = open('abc.txt', 'a+')
ip_list = ["192.163.1.104", "192.163.1.103", "192.163.1.101"]
for ip in ip_list:
cmd = "snmpwalk.exe -t 1 -v2c -c public "
cmd = cmd + ip
print(cmd)
p = subprocess.Popen(cmd, shell=True, stdout=f)
p.wait()
f.close()
print("File output - " + open('abc.txt', 'r').read())
the sample output from the command can be something like this for each IP -
sysDescr.0 = STRING: Software: Whistler Version 5.1 Service Pack 2 (Build 2600)
sysObjectID.0 = OID: win32
sysUpTimeInstance = Timeticks: (15535) 0:02:35.35
sysContact.0 = STRING: unknown
sysName.0 = STRING: UDLDEV
sysLocation.0 = STRING: unknown
sysServices.0 = INTEGER: 72
sysORID.4 = OID: snmpMPDCompliance
I have already tried Popen. But it does not logs output to a file if it is a time consuming background process. However, it works when I try to run background process like ls/dir. Any help is appreciated.

The main issue here is the expectation of what Popen does and how it works I assume.
p.wait() here will wait for the process to finish before continuing, that is why ls for instance works but more time consuming tasks doesn't. And there's nothing flushing the output automatically until you call p.stdout.flush().
The way you've set it up is more meant to work for:
Execute command
Wait for exit
Catch output
And then work with it. For your usecase, you'd better off using an alternative library or use the stdout=subprocess.PIPE and catch it yourself. Which would mean something along the lines of:
import subprocess
import sys
ip_list = ["192.163.1.104", "192.163.1.103", "192.163.1.101"]
with open('abc.txt', 'a+') as output:
for ip in ip_list:
print(cmd := f"snmpwalk.exe -t 1 -v2c -c public {ip}")
process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE) # Be wary of shell=True
while process.poll() is None:
for c in iter(lambda: process.stdout.read(1), ''):
if c != '':
output.write(c)
with open('abc.txt', 'r') as log:
print("File output: " + log.read())
The key things to take away here is process.poll() which checks if the process has finished, if not, we'll try to catch the output with process.stdout.read(1) to read one byte at a time. If you know there's new lines coming, you can switch those three lines to output.write(process.stdout.readline()) and you're all set.

In Python, how do I get user's remote IP (their last hop) if they're connected over SSH?

I want to detect if the user is connected over SSH. In a term, the "env" command shows SSH_CONNECTION line. Accessed in Python in one of two ways:
#python:
import os
print os.getenv("SSH_CONNECTION") #works
print os.environ.get("SSH_CONNECTION") #works
But, if the user has ran my program using SUDO (as they will need to), env$ dooesn't show SSH_CONNECTION. So Python can't see it:
#sudo python:
import os
print os.getenv("SSH_CONNECTION") #not set
print os.environ.get("SSH_CONNECTION") #not set
The aim is to achieve the following:
#Detect if user is over remote IP
lRemoteIP="" #Is set if user on SSH
lStr=os.environ.get("SSH_CONNECTION") #Temp var
if lStr: lRemoteIP=lStr.split()[0].split("=")[1] #Store user's lasthop IP
#Later on in the code, for multiple purposes:
if lRemoteIP: pass #Do stuff (or not) depending on if they're on SSH
How do I retrieve SSH_CONNECTION environment variable under SUDO, when its not present in env$ ?
Or more precisely: how can I detect if the current session is via SSH when sudo?
I'm not a natural at Linuxy-type things, so be gentle with me...
[EDIT:] METHOD 2: Giving up on env$, I've tried the following:
pstree -ps $$ | grep "sshd("
If it returns anything then it means that the SSH daemon sits above the session. Ergo, it's a SSH connection. And the results are showing me the PIDs of the SSH daemons. Results of the pstree cmd:
init(1)---sshd(xxx)---sshd(xxx)---sshd(xxx)---bash(xxx)-+-grep(xxx)
But I'm struggling to get a src IP from the PID. Any ideas on this avenue?
[EDIT] METHOD 3: /run/utmp contains details of SSH logins. In python:
import os
import sys
lStr=open("/var/run/utmp").read().replace('\x00','') #Remove all those null values which make things hard to read
#Get the pseudo-session ID (pts) minus the /dev/ that it starts with:
lCurSess=os.ttyname(sys.stdout.fileno()).replace('/dev/','')
#Answer is like pts/10 (pseudo-term session number 10)
#Search lStr for pts/10
lInt=lStr.find(lCurSess.replace('/dev/',''))
#Print /var/utmp starting with where it first mentions current pts:
print lStr[lInt:]
So far, so good. This gives the following results (I've changed the IP and username to USERNAME)
pts/10/10USERNAME\x9e\x958Ym\xb2\x05\x0 74\x14pts/10s/10USERNAME192.168.1.1\xbf\x958Y\xf8\xa3\r\xc0\xa88\x01
So, when it comes to extracting the IP from the file, there's some bumf inbetween the occurances of pts/10 and the IP. What's the best way to parse it, given that (I reckon) the precise distance from the match to the IP will be different under different circumstances?

The OpenSSH daemon writes an entry to /var/run/utmp with the current terminal, the IP and the name of the user. Check the output of the w or who commands that parse /var/run/utmp.
It's just a question of getting the current terminal (similar to the tty command) and extracting the information you want.
Use pyutmp like this:
from pyutmp import UtmpFile
import os
import sys
for utmp in UtmpFile():
if os.ttyname(sys.stdout.fileno()) == utmp.ut_line:
print '%s logged from %s on tty %s' % (utmp.ut_user, utmp.ut_host, utmp.ut_line)
Then filter by using ut_pid field to parse the /proc/ut_pid/cmdline file which should contain:
sshd: ut_user [priv]

GOT IT AT LAST!!!
The "last" command has list of users and their IPs!! So simple.
It has "still logged in" marked against sessions. Filter by these
And then filter by current pts ID
To get the IP for the current SSH session in Python, do this:
import os,sys,subprocess
(out, err) = subprocess.Popen(['last | grep "still logged in" | grep "' + os.ttyname(sys.stdout.fileno()).replace('/dev/','') + '"'], stdout=subprocess.PIPE, shell=True).communicate()
RemoteIP=out.split()[2].replace(":0.0","") #Returns "" if not SSH
For readability, across multiple lines:
import os,sys,subprocess
pseudoTermID = os.ttyname(sys.stdout.fileno()).replace('/dev/','')
cmdStr = 'last | grep "still logged in" | grep "'+pseudoTermID+'"'
sp = subprocess.Popen([cmdStr], stdout=subprocess.PIPE, shell=True)
(out, err) = sp.communicate()
RemoteIP = out.split()[2].replace(":0.0","") #Returns "" if not SSH

is it possible to write entire python output to a text or csv file?

i'm new to python and working on traceroute so I was wondering if it's possible to write entire python traceroute result output to txt and Csv file? any idea how can i achieve that because i can't find any proper one which shows how to do it.
thanks

import subprocess
with open("hostlist.txt", "r") as hostlist, open("results.txt", "a") as output:
for host in hostlist:
host = host.strip()
print "Tracing", host
trace = subprocess.Popen(["tracert", "-w", "100", host], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while True:
hop = trace.stdout.readline()
if not hop: break
print '-->', hop.strip()
output.write(hop)
# When you pipe stdout, the doc recommends that you use .communicate()
# instead of wait()
# see: http://docs.python.org/2/library/subprocess.html#subprocess.Popen.wait
trace.communicate()
I am reading host details from hostlist and writing o/p in a file

how to read command output from serial device using python

I have an embedded linux device and here's what I would like to do using python:
Get the device console over serial port. I can do it like this:
>>> ser = serial.Serial('/dev/ttyUSB-17', 115200, timeout=1)
Now I want to run a tail command on the embedded device command line, like this:
# tail -f /var/log/messages
and capture the o/p and display on my python >>> console.
How do I do that ?

Just open the file inside python and keep readign from it. If needed be, in another thread:
>>> ser = serial.Serial('/dev/ttyUSB-17', 115200, timeout=1)
>>> output = open("/var/log/messages", "rb")
And inside any program loop, just do:
data = output.read()
print(data)
If you want it to just go printing on the console as you keep doing other stuff, type
in something like:
from time import sleep
from threading import Thread
class Display(Thread):
def run(self):
while True:
data = self.output.read()
if data: print(data)
sleep(1)
t = Display()
t.output = output
t.start()

very first you need to get log-in into the device.
then you can run the specified command on that device.
note:command which you are going to run must be supported by that device.
Now after opening a serial port using open() you need to find the login prompt using Read() and then write the username using write(), same thing repeat for password.
once you have logged-in you can now run the commands you needed to execute

how to get console output from a remote computer (ssh + python)

I have googled "python ssh". There is a wonderful module pexpect, which can access a remote computer using ssh (with password).
After the remote computer is connected, I can execute other commands. However I cannot get the result in python again.
p = pexpect.spawn("ssh user#remote_computer")
print "connecting..."
p.waitnoecho()
p.sendline(my_password)
print "connected"
p.sendline("ps -ef")
p.expect(pexpect.EOF) # this will take very long time
print p.before
How to get the result of ps -ef in my case?

Have you tried an even simpler approach?
>>> from subprocess import Popen, PIPE
>>> stdout, stderr = Popen(['ssh', 'user#remote_computer', 'ps -ef'],
... stdout=PIPE).communicate()
>>> print(stdout)
Granted, this only works because I have ssh-agent running preloaded with a private key that the remote host knows about.

child = pexpect.spawn("ssh user#remote_computer ps -ef")
print "connecting..."
i = child.expect(['user#remote_computer\'s password:'])
child.sendline(user_password)
i = child.expect([' .*']) #or use i = child.expect([pexpect.EOF])
if i == 0:
print child.after # uncomment when using [' .*'] pattern
#print child.before # uncomment when using EOF pattern
else:
print "Unable to capture output"
Hope this help..

You might also want to investigate paramiko which is another SSH library for Python.

Try to send
p.sendline("ps -ef\n")
IIRC, the text you send is interpreted verbatim, so the other computer is probably waiting for you to complete the command.