Develop Reference

Make python script write output in .txt file after force quit

I have a Python script running on a server through SSH with the following command:
nohup python3 python_script.py >> output.txt
It was running for a very long time and (probably) created useful output, so I want to force it to stop now. How do I make it write the output it has so far, into the output.txt file?
The file was automatically created when I started running the script, but the size is zero (nothing has been written in it so far).

as Robert said in his comment, check that the output you are expecting to go to the file is actually making it there and not stderr. If the process is already running and has been for a long time without any response or writes into your output file, I think there are 3 options:
It is generating outout but it's not going where you are expecting (Roberts response)
It is generating output but it's buffered and for some reason has yet to be flushed
It's hasn't generated any output
Option 3 is easy: wait longer. Options 1 & 2 are a little bit tricky. If you are expecting a significant amount of output from this process, you could check the memory consumption of the python instance running your script and see if it's growing or very large. Also you could use lsof to see if it has the file open and to get some idea what it's doing with it.
If you find that your output appears to be going somewhere like /dev/null, take a look at this answer on redirecting output for an existing process.
In the unlikely event that you have a huge buffer that hasn't been flushed, you could try using ps to get the PID and then use kill -STOP [PID] to pause the process and see where you can get using GDB.
Unless it would be extremely painful, I would probably just restart the whole thing, but add periodic flushing to the script, and maybe some extra status reporting so you can tell what is going on. It might help too if you could post your code, since there may be other options available in your situation depending on how the program is written.

Python program to capture output of console to file for every 10 seconds

I have a 3rd party software which i run on Linux platform, this gives me a output for every 10 seconds. Is there any way in python to capture the screen contents thrown after every 10 seconds to a file a via python.
I do not want to run this 3rd party software command via sub process.

You don't need a python script to redirect output from a program to a file. The Linux bash already gives you the option to redirect output of running programs to files with the ">" or ">>" operators.
Source: Redirect all output to file
But since you want a pythonic solution, you could just use Popen.communicate to start your program and dump the output of its stdout/stderr to a variable in a python script that you can then use to write to a file.
Source: How can I capture the stdout output of a child process?

Receiving streaming output from ssh connection in Python

I am running a script remotely on a server via SSH and Python. This script looks through log files and returns some information based on each log entry it encounters. The problem I am running into is that although my script spits out the information as soon as it hits each log entry, my local machine needs to wait for the entire process to finish before it can read the lines from the ssh connection's stdout.
ssh = subprocess.Popen(cmd.split(' '), stdout=subprocess.PIPE)
with open(remote_ip+'.hits', 'w') as f:
for line in ssh.stdout:
print line
Essentially this code prints all of the results all at once, at the end. I was wondering if there was a way for me to print out the contents of stdout as it was being produced on the server. Sorry if this is unclear, if something is ambiguous I will do my best to clarify it. Thanks!
EDIT: I should also add that I would preferably like to do this without any external packages, just built-in modules from 2.6 if possible.

How do I read terminal output in real time from Python?

I am writing a Python program which runs a virtual terminal. Currently I am launching it like so:
import pexpect, thread
def create_input(child, scrollers, textlength=80, height=12):
while 1:
newtext = child.readline()
print newtext
child = pexpect.spawn("bash", timeout=30000)
thread.start_new_thread(create_input,(child))
This works, and I can send commands to it via child.send(command). However, I only get entire lines as output. This means that if I launch something like Nano or Links, I don't receive any output until the process has completed. I also can't see what I'm typing until I press enter. Is there any way to read the individual characters as bash outputs them?

You would need to change the output of whatever program bash is running to be unbuffered instead of line buffering. A good amount of programs have a command line option for unbuffered output.
The expect project has a tool called unbuffer that looks like it can give you all bash output unbuffered. I have personally never used it, but there are other answers here on SO that also recommend it: bash: force exec'd process to have unbuffered stdout

The problem is lies in something else. If you open an interactive shell normally a terminal window is opened that runs bash, sh, csh or whatever. See the word terminal!
In the old days, we connected a terminal to a serial port (telnet does the same but over ip), again the word terminal.
Even a dumb terminal respond to ESC codes, to report its type and to set the cursor position, colors, clear screen etc.
So you are starting a subprocess with interactive output, but there is no way in telling that shell and subprocesses are to a terminal in this setup other than with bash startup parameters if there are any.
I suggest you enable telnetd but only on localhost (127.0.0.1)
Within your program, make a socket and connect to localhost:telnet and look up how to emulate a proper terminal. If a program is in line mode you are fine but if you go to full screen editing, somewhere you will need an array of 80x24 or 132x24 or whatever size you want to store its characters, color. You also need to be able to shift lines up in that array.
I have not looked but I cannot imagine there is no telnet client example in python, and a terminal emu must be there too!
Another great thing is that telnet sessions clean up if the the ip connection is lost, eliminating ghost processes.
Martijn

What would happen if I abruptly close my script while it's still doing file I/O operations?

here's my question: I'm writing a script to check if my website running all right, the basic idea is to get the server response time and similar stuff every 5 minutes or so, and the script will log the info each time after checking the server status. I know it's no good to close the script while it's in the middle of checking/writing logs, but I'm curious if there are lots of server to check and also you have to do the file I/O pretty frequently, what would happen if you abruptly close the script?
OK, here's an example:
while True:
DemoFile = open("DemoFile.txt", "a")
DemoFile.write("This is a test!")
DemoFile.close()
time.sleep(30)
If I accidentally close the script while this line DemoFile.write("This is a test!") is running, what would I get in the DemoFile.txt? Do I get "This i"(an incomplete line) or the complete line or the line not even added?
Hopefully somebody knows the answer.

According to the python io documentation, buffering is handled according to the buffering parameter to the open function.
The default behavior in this case would be either the device's block size or io.DEFAULT_BUFFER_SIZE if the block size can't be determined. This is probably something like 4096 bytes.
In short, that example will write nothing. If you were writing something long enough that the buffer was written once or twice, you'd have multiples of the buffer size written. And you can always manually flush the buffer with flush().
(If you specify buffering as 0 and the file mode as binary, you'd get "This i". That's the only way, though)
As #sven pointed out, python isn't doing the buffering. When the program is terminated, all open file descriptors are closed and flushed by the operating system.