Develop Reference

Unable to read file with python

I'm trying to read the content of a file with python 3.8.5 but the output is empty, I don't understand what I'm doing wrong.
Here is the code:
import subprocess
import os
filename = "ls.out"
ls_command = "ls -la"
file = open(filename, "w")
subprocess.Popen(ls_command, stdout=file, shell=True)
file.close()
# So far, all is ok. The file "ls.out" is correctly created and filled with the output of "ls -la" command"
file = open(filename, "r")
for line in file:
print(line)
file.close()
The output of this script is empty, it doesn't print anything. I'm not able to see the content of ls.out.
What is not correct here ?

Popen creates a new process and launches it but returns immediately. So the end result is that you've forked your code and have both processes running at once. Your python code in executing faster than the start and finish of ls. Thus, you need to wait for the process to finish by adding a call to wait():
import subprocess
import os
filename = "ls.out"
ls_command = "ls -la"
file = open(filename, "w")
proc = subprocess.Popen(ls_command, stdout=file, shell=True)
proc.wait()
file.close()
file = open(filename, "r")
for line in file:
print(line)
file.close()

Popen merely starts the subprocess. Chances are the file is not yet populated when you open it.
If you want to wait for the Popen object to finish, you have to call its wait method, etc; but a much better and simpler solution is to use subprocess.check_call() or one of the other higher-level wrappers.
If the command prints to standard output, why don't you read it drectly?
import subprocess
import shlex
result = subprocess.run(
shlex.split(ls_command), # avoid shell=True
check=True, text=True, capture_output=True)
line = result.stdout

python readlines not working during incron

I'm trying to call a python script through incron:
/data/alucard-ops/drop IN_CLOSE_WRITE /data/alucard-ops/util/test.py $#/$#
but I cant seem to read from the file passed. Here is the script:
#!/usr/bin/env /usr/bin/python3
import os,sys
logfile = '/data/alucard-ops/log/'
log = open(logfile + 'test.log', 'a')
log.write(sys.argv[1] + "\n")
log.write(str(os.path.exists(sys.argv[1])) + "\n")
datafile = open(sys.argv[1], 'r')
log.write('Open\n')
data = datafile.readlines()
log.write("read\n")
datafile.close()
The output generated by the script:
/data/alucard-ops/drop/nsco-20180219.csv
True
Open
It seems to stop at the readlines() call. I dont see any errors in the syslog.
Update: It seems that i can use a subprocess to cat the file and it retrieves the contents. But, when i decode it, data.decode('utf-8') I'm back to nothing in the variable.

I ended up using watchdog instead.

send string between python script

I want to send 'hello world' to a script in python already running in ubuntu.
The script that's always running is this one (part of it):
print("$ echo 'foobar' > {0}".format(get_ttyname()))
print("$ echo 'foobar' > /proc/{0}/fd/0".format(os.getpid()))
sys.stdin.readline()
it throws the pid of the running process so I can send stuff by console with:
echo 'hello script!' > /proc/PID/fd/0
It will print it in the console! but I can't send \x15 or EOF or anything to break sys.stdin.readline() and do some other stuff in my script, for example:
def f(e):
print 'we already read:',s
while True:
s = sys.stdin.readline()
print 'we break the readline'
f(s)
.....blablabla some other stuff, and then we return to the top of the while to keep reading...
Does anyone know how to do it? The script that send the string will not always be running, but the script that receives the info will be always running.
PROBLEM SOLVED!
Thank's to Rafael this is the solution:
Reader:
import os
import sys
path = "/tmp/my_program.fifo"
try:
os.mkfifo(path)
except OSError:
pass
fifo = open(path, "r")
while True:
for line in fifo:
linea = line
print "Received: " + linea,
fifo.close()
if linea =='quit':
break
fifo = open(path, "r")
Sender:
# -*- coding: utf-8 -*-
import os
path = "/tmp/my_program.fifo"
fifo = open(path, "w")
fifo.write("Hello Wordl!!\n")
fifo.close()

Since you obviously don't have a problem with being limited to a Unix system, you can use named pipes to communicate with the program. Very unix-y way to work.
Python provides the os.mkfifo function to ease creating named pipes; otherwise they work just like files.

Write to a text file that is read by the already running program. The two can interact via this file. For example, these two programs simultaneously read and write to an initially empty text file.
already.py
# executed 1st
import time
while True:
text = open('file.txt').read()
print 'File contents: ' + text
time.sleep(5)
program.py
# executed 2nd
import time
while True:
text = open('file.txt', 'a')
text.write(raw_input('Enter data: '))
text.close()
time.sleep(5)

Read file and copy to standard output.

I'm trying to write a python program that will read input and copy it to standard output (with no alterations). I've been told that it needs to operate as a Python version of the Unix cat function. If a file cannot be opened, an error message needs to be printed, and then the program needs to continue processing any additional files. I am a complete beginner, and have tried my best to scrape something together with my limited knowledge. Here is what I have so far:
from sys import argv, stdout, stdin, stderr
if len(argv) == 1:
try:
stdout.write(raw_input(' ') + '\n')
except:
stderr.write ('sorry' + '\n')
quit()
else:
for filename in argv[1:]:
try:
filehandle + open(filename)
except IOError:
stderr.write('Sorry, could not open', filename + '\n')
continue
f = filehandle.read()
stdout.write(f)
I am not quite sure where to go from here.. does anyone have any advice/am I on the right track even a little bit? Please and thank you!

This function will copy the specified file to the console line by line (in case you later on decide to give it the ability to use the -n command line option of cat)
def catfile(fn):
with open(fn) as f:
for line in f:
print line,
It can be called with the filename once you have established the file exists.

How to redirect stderr of a program that is run using os.system by a third-party python library

I use external library, like this:
from some_lib import runThatProgram
infile = '/tmp/test'
outfile = '/tmp/testout'
runThatProgram(infile, outfile)
while runThatProgram is:
def runThatProgram(infile, outfile):
os.system("%s %s > %s" % ('thatProgram', infile, outfile))
The problem is that 'thatProgram' returns lots of stuff on STDERR, I want to redirect it to a file, but I cannot edit runThatProgram code because it is in third party lib!

To illustrate what Rosh Oxymoron said, you can hack the code like this :
from some_lib import runThatProgram
infile = '/tmp/test'
outfile = '/tmp/testout 2>&1'
runThatProgram(infile, outfile)
with this, it will call
thatProgram /tmp/test > /tmp/testout 2>&1
that will redirected stderr (2) to stdout (1), and everything will be logged in your outfile.

To elaborate on using subprocess, you can open it, give it a pipe and then work from there so
import subprocess
program = "runthatprogram.py".split()
process = subprocess.Popen(program, stdout = subprocess.PIPE, stderr = open('stderr','w')) #stderr to fileobj
process.communicate()[0] #display stdout