I need to write a temporary file to a n*x machine using python3 so that I can read it from the command line.
import tempfile
import subprocess
from os import path
string = 'hi *there*'
# run markdown server-side
tfile = tempfile.NamedTemporaryFile(mode='w+', suffix='.txt', prefix='prove-math-')
tfile.write(string)
fpath = tfile.name
markdown_path = path.join(LIB_DIR, 'Markdown.pl')
command = [markdown_path, fpath]
completed_process = subprocess.run(command, check=True, stdout=subprocess.PIPE)
string = completed_process.stdout.decode()
tfile.close()
print(string)
The output should be '<p>hi <em>there</em></p>', but the actual output is '\n', which suggests to me that Markdown.pl read the contents of the file as '\n'.
Use,
file_obj.flush()
In your case, you'll have to use
tfile.flush()
It'll write to the file on being called upon!
Related
I'm trying to read the content of a file with python 3.8.5 but the output is empty, I don't understand what I'm doing wrong.
Here is the code:
import subprocess
import os
filename = "ls.out"
ls_command = "ls -la"
file = open(filename, "w")
subprocess.Popen(ls_command, stdout=file, shell=True)
file.close()
# So far, all is ok. The file "ls.out" is correctly created and filled with the output of "ls -la" command"
file = open(filename, "r")
for line in file:
print(line)
file.close()
The output of this script is empty, it doesn't print anything. I'm not able to see the content of ls.out.
What is not correct here ?
Popen creates a new process and launches it but returns immediately. So the end result is that you've forked your code and have both processes running at once. Your python code in executing faster than the start and finish of ls. Thus, you need to wait for the process to finish by adding a call to wait():
import subprocess
import os
filename = "ls.out"
ls_command = "ls -la"
file = open(filename, "w")
proc = subprocess.Popen(ls_command, stdout=file, shell=True)
proc.wait()
file.close()
file = open(filename, "r")
for line in file:
print(line)
file.close()
Popen merely starts the subprocess. Chances are the file is not yet populated when you open it.
If you want to wait for the Popen object to finish, you have to call its wait method, etc; but a much better and simpler solution is to use subprocess.check_call() or one of the other higher-level wrappers.
If the command prints to standard output, why don't you read it drectly?
import subprocess
import shlex
result = subprocess.run(
shlex.split(ls_command), # avoid shell=True
check=True, text=True, capture_output=True)
line = result.stdout
I'm trying to check reverse lookup of IP address (argument). and then write the result to txt file.
How I can check if the IP address (argument) is already registered in the file? If so, I need to get out of the script.
My script:
import sys, os, re, shlex, urllib, subprocess
cmd = 'dig -x %s #192.1.1.1' % sys.argv[1]
proc = subprocess.Popen(shlex.split(cmd), stdout=subprocess.PIPE)
out, err = proc.communicate()
# Convert to list of str lines
out = out.decode().split('\n')
# Only write the line containing "PTR"
with open("/tmp/test.txt", "w") as f:
for line in out:
if "PTR" in line:
f.write(line)
If the file is not too large you could do:
with open('file.txt','r') as f:
content = f.read()
if ip in content:
sys.exit(0)
Now if the file is big and you want to avoid possible memory problems you could use mmap like so:
import mmap
with open("file.txt", "r+b") as f:
# memory-map the file, size 0 means whole file
mm = mmap.mmap(f.fileno(), 0)
if mm.find(ip) != -1:
sys.exit(0)
The mmap.find(string[, start[, end]]) is well documented here.
Something like:
otherIps = [line.strip() for line in open("<path to ipFile>", 'r')]
theIp = "192.168.1.1"
if theIp in otherIps:
sys.exit(0)
otherIps contains a list of the ip addresses on ipFile, then you need to check if theIp is already on otherIps, if so, exit the script.
I would like a Python script to prompt me for a string, but I would like to use Vim to enter that string (because the string might be long and I want to use Vim's editing capability while entering it).
You can call vim with a file path of your choice:
from subprocess import call
call(["vim","hello.txt"])
Now you can use this file as your string:
file = open("hello.txt", "r")
aString = file.read()
Solution:
#!/usr/bin/env python
from __future__ import print_function
from os import unlink
from tempfile import mkstemp
from subprocess import Popen
def callvim():
fd, filename = mkstemp()
p = Popen(["/usr/bin/vim", filename])
p.wait()
try:
return open(filename, "r").read()
finally:
unlink(filename)
data = callvim()
print(data)
Example:
$ python foo.py
This is a big string.
This is another line in the string.
Bye!
I'm executing a .py file, which spits out a give string. This command works fine
execfile ('file.py')
But I want the output (in addition to it being shown in the shell) written into a text file.
I tried this, but it's not working :(
execfile ('file.py') > ('output.txt')
All I get is this:
tugsjs6555
False
I guess "False" is referring to the output file not being successfully written :(
Thanks for your help
what your doing is checking the output of execfile('file.py') against the string 'output.txt'
you can do what you want to do with subprocess
#!/usr/bin/env python
import subprocess
with open("output.txt", "w+") as output:
subprocess.call(["python", "./script.py"], stdout=output);
This'll also work, due to directing standard out to the file output.txt before executing "file.py":
import sys
orig = sys.stdout
with open("output.txt", "wb") as f:
sys.stdout = f
try:
execfile("file.py", {})
finally:
sys.stdout = orig
Alternatively, execute the script in a subprocess:
import subprocess
with open("output.txt", "wb") as f:
subprocess.check_call(["python", "file.py"], stdout=f)
If you want to write to a directory, assuming you wish to hardcode the directory path:
import sys
import os.path
orig = sys.stdout
with open(os.path.join("dir", "output.txt"), "wb") as f:
sys.stdout = f
try:
execfile("file.py", {})
finally:
sys.stdout = orig
If you are running the file on Windows command prompt:
python filename.py >> textfile.txt
The output would be redirected to the textfile.txt in the same folder where the filename.py file is stored.
The above is only if you have the results showing on cmd and you want to see the entire result without it being truncated.
The simplest way to run a script and get the output to a text file is by typing the below in the terminal:
PCname:~/Path/WorkFolderName$ python scriptname.py>output.txt
*Make sure you have created output.txt in the work folder before executing the command.
Use this instead:
text_file = open('output.txt', 'w')
text_file.write('my string i want to put in file')
text_file.close()
Put it into your main file and go ahead and run it. Replace the string in the 2nd line with your string or a variable containing the string you want to output. If you have further questions post below.
file_open = open("test1.txt", "r")
file_output = open("output.txt", "w")
for line in file_open:
print ("%s"%(line), file=file_output)
file_open.close()
file_output.close()
using some hints from Remolten in the above posts and some other links I have written the following:
from os import listdir
from os.path import isfile, join
folderpath = "/Users/nupadhy/Downloads"
filenames = [A for A in listdir(folderpath) if isfile(join(folderpath,A))]
newlistfiles = ("\n".join(filenames))
OuttxtFile = open('listallfiles.txt', 'w')
OuttxtFile.write(newlistfiles)
OuttxtFile.close()
The code above is to list all files in my download folder. It saves the output to the output to listallfiles.txt. If the file is not there it will create and replace it with a new every time to run this code. Only thing you need to be mindful of is that it will create the output file in the folder where your py script is saved. See how you go, hope it helps.
You could also do this by going to the path of the folder you have the python script saved at with cmd, then do the name.py > filename.txt
It worked for me on windows 10
I'm in my 2nd week of Python and I'm stuck on a directory of zipped/unzipped logfiles, which I need to parse and process.
Currently I'm doing this:
import os
import sys
import operator
import zipfile
import zlib
import gzip
import subprocess
if sys.version.startswith("3."):
import io
io_method = io.BytesIO
else:
import cStringIO
io_method = cStringIO.StringIO
for f in glob.glob('logs/*'):
file = open(f,'rb')
new_file_name = f + "_unzipped"
last_pos = file.tell()
# test for gzip
if (file.read(2) == b'\x1f\x8b'):
file.seek(last_pos)
#unzip to new file
out = open( new_file_name, "wb" )
process = subprocess.Popen(["zcat", f], stdout = subprocess.PIPE, stderr=subprocess.STDOUT)
while True:
if process.poll() != None:
break;
output = io_method(process.communicate()[0])
exitCode = process.returncode
if (exitCode == 0):
print "done"
out.write( output )
out.close()
else:
raise ProcessException(command, exitCode, output)
which I've "stitched" together using these SO answers (here) and blogposts (here)
However, it does not seem to work, because my test file is 2.5GB and the script has been chewing on it for 10+mins plus I'm not really sure if what I'm doing is correct anyway.
Question:
If I don't want to use GZIP module and need to de-compress chunk-by-chunk (actual files are >10GB), how do I uncompress and save to file using zcat and subprocess in Python?
Thanks!
This should read the first line of every file in the logs subdirectory, unzipping as required:
#!/usr/bin/env python
import glob
import gzip
import subprocess
for f in glob.glob('logs/*'):
if f.endswith('.gz'):
# Open a compressed file. Here is the easy way:
# file = gzip.open(f, 'rb')
# Or, here is the hard way:
proc = subprocess.Popen(['zcat', f], stdout=subprocess.PIPE)
file = proc.stdout
else:
# Otherwise, it must be a regular file
file = open(f, 'rb')
# Process file, for example:
print f, file.readline()