I am writing a script to clean up my desktop, moving files based on file type. The first step, it would seem, is to ls -1 /Users/user/Desktop (I'm on Mac OSX). So, using Python, how would I run a command, then write the output to a file in a specific directory? Since this will be undocumented, and I'll be the only user, I don't mind (prefer?) if it uses os.system().
You can redirect standard output to any file using > in command.
$ ls /Users/user/Desktop > out.txt
Using python,
os.system('ls /Users/user/Desktop > out.txt')
However, if you are using python then instead of using ls command you can use os.listdir to list all the files in the directory.
path = '/Users/user/Desktop'
files = os.listdir(path)
print files
After skimming the python documentation to run shell command and obtain the output you can use the subprocess module with the check_output method.
After that you can simple write that output to a file with the standard Python IO functions: File IO in python.
To open a file, you can use the f = open(/Path/To/File) command. The syntax is f = open('/Path/To/File', 'type') where 'type' is r for reading, w for writing, and a for appending. The commands to do this are f.read() and f.write('content_to_write'). To get the output from a command line command, you have to use popen and subprocess instead of os.system(). os.system() doesn't return a value. You can read more on popen here.
Related
I'm using this code os.system("dcapgen.exe C:\\Users\\folder\\a.dcap") in my python script to run this command dcapgen.exe C:\Users\folder\a.dcap. This command generates .txt file in its current directory. I want to use this generated .txt file further in my code. How to do this? I don't want command log output. Thanks!
I'm new to python programming.
Try the open method: here.
It is especially handy in Python that you can surround this method with a with clause.
Let's assume you have a simple C program that writes "Hello World" to a text file, called text.txt, as follows:
#include <stdio.h>
#include <stdlib.h>
int main(){
FILE * fp = fopen ("text.txt","w");
fprintf(fp, "Hello world\n");
fclose(fp);
return 0;
}
Compiling the C program will give you an executable, in our case it will be called a.out. The command ./a.out will run the executable and print to file.
Now let's assume we have a Python script in the same folder. In order to execute the C program AND read from file you'll have to do something like this
import os
# Run the C generated executable
os.system('./a.out')
# At this point a `text.txt` file exists, so it can be accessed
# The "r" option means you want to read. "w" or "a" for write and append respectively
# The file is now accesible with assigned name `file`
with open("text.txt", "r") as file:
file.read() # To get the full content of the file
file.readline() # To read a single line
for line in file: # Handy way to traverse the file line by line
print(line)
EDIT(s)
If you want to follow through keep in mind that:
This is a step-by-step Linux approach. I know little about Windows, but I am assuming executables are ran a bit differently. I'm sure you know how.
If you have your files across different folders you'll need the path to the files to be specified, not just the file name.
Use subprocess.run to run the command and capture STDOUT to get the output from the command:
proc = subprocess.run(['dcapgen.exe', 'C:\\Users\\folder\\a.dcap'], stdout=subprocess.PIPE, text=True)
Now you can get the STDOUT from the stdout attribute:
proc.stdout
You might need to strip-off CR-LF from end:
proc.stdout.rstrip()
Edit:
If you're using Python 2.7, you can use subprocess.check_output:
out = subprocess.check_output(['dcapgen.exe', 'C:\\Users\\folder\\a.dcap'])
This cannot be done because .txt file that will be generated is not getting traced in command prompt. we can only get cmd log output using subprocess which is being suggested by other answers. so, we can get access to the generated file by using its path and full name which we know because its naming follows some pattern and location is fixed, like this
generated_text_file = original_file_name+"_some_addition.txt"
with open(generated_text_file, 'r') as file1:
s = file1.read().replace('/n','')
I am trying to tail a file's content through python. The code I am using is as below
#! /usr/bin/python
import subprocess
import os.path
# Get the file path
filepath = os.path.join(baseDir,"filename.*" + uniqueId)
# Call subprocess and get last 10 lines from file
spTailFile = subprocess.Popen(["tail", "-10", filepath ], stdout=subprocess.PIPE)
tailOutput = spTailFile.communicate()[0]
print tailOutput
The above code throws an error as below:
tail: cannot open `/hostname/user/app/filename.*39102'
I see output if I execute the tail command with the filepath directly in bash.
tail -10 /hostname/user/app/filename.*39102
Why is subprocess passing an extra backtick (`) when executing the tail command?
Update:
I ended up using glob to find the file as #cdarke had suggested and then passing it to the Popen cmd.
Bash extends the '*', Popen not.
Two possibilities:.
1. Do it within your script and pass a filename without '*'.
2. Create a Bash script and call this from python.
I would like to start out by saying any help is greatly appreciated. I'm new to Python and scripting in general. I am trying to use a program called samtools view to convert a file from .sam to a .bam I need to be able do what this BASH command is doing in Python:
samtools view -bS aln.sam > aln.bam
I understand that BASH commands like | > < are done using the subprocess stdin, stdout and stderr in Python. I have tried a few different methods and still can't get my BASH script converted correctly. I have tried:
cmd = subprocess.call(["samtools view","-bS"], stdin=open(aln.sam,'r'), stdout=open(aln.bam,'w'), shell=True)
and
from subprocess import Popen
with open(SAMPLE+ "."+ TARGET+ ".sam",'wb',0) as input_file:
with open(SAMPLE+ "."+ TARGET+ ".bam",'wb',0) as output_file:
cmd = Popen([Dir+ "samtools-1.1/samtools view",'-bS'],
stdin=(input_file), stdout=(output_file), shell=True)
in Python and am still not getting samtools to convert a .sam to a .bam file. What am I doing wrong?
Abukamel is right, but in case you (or others) are wondering about your specific examples....
You're not too far off with your first attempt, just a few minor items:
Filenames should be in quotes
samtools reads from a named input file, not from stdin
You don't need "shell=True" since you're not using shell tricks like redirection
So you can do:
import subprocess
subprocess.call(["samtools", "view", "-bS", "aln.sam"],
stdout=open('aln.bam','w'))
Your second example has more or less the same issues, so would need to be changed to something like:
from subprocess import Popen
with open('aln.bam', 'wb',0) as output_file:
cmd = Popen(["samtools", "view",'-bS','aln.sam'],
stdout=(output_file))
You can pass execution to the shell by kwarg 'shell=True'
subprocess.call('samtools view -bS aln.sam > aln.bam', shell=True)
In the Linux kernel, I can send a file to the printer using the following command
cat file.txt > /dev/usb/lp0
From what I understand, this redirects the contents in file.txt into the printing location. I tried using the following command
>>os.system('cat file.txt > /dev/usb/lp0')
I thought this command would achieve the same thing, but it gave me a "Permission Denied" error. In the command line, I would run the following command prior to concatenating.
sudo chown root:lpadmin /dev/usb/lp0
Is there a better way to do this?
While there's no reason your code shouldn't work, this probably isn't the way you want to do this. If you just want to run shell commands, bash is much better than python. On the other hand, if you want to use Python, there are better ways to copy files than shell redirection.
The simplest way to copy one file to another is to use shutil:
shutil.copyfile('file.txt', '/dev/usb/lp0')
(Of course if you have permissions problems that prevent redirect from working, you'll have the same permissions problems with copying.)
You want a program that reads input from the keyboard, and when it gets a certain input, it prints a certain file. That's easy:
import shutil
while True:
line = raw_input() # or just input() if you're on Python 3.x
if line == 'certain input':
shutil.copyfile('file.txt', '/dev/usb/lp0')
Obviously a real program will be a bit more complex—it'll do different things with different commands, and maybe take arguments that tell it which file to print, and so on. If you want to go that way, the cmd module is a great help.
Remember, in UNIX - everything is a file. Even devices.
So, you can just use basic (or anything else, e.g. shutil.copyfile) files methods (http://docs.python.org/2/tutorial/inputoutput.html#reading-and-writing-files).
In your case code may (just a way) be like that:
# Read file.txt
with open('file.txt', 'r') as content_file:
content = content_file.read()
with open('/dev/usb/lp0', 'w') as target_device:
target_device.write(content)
P. S. Please, don't use system() call (or similar) to solve your issue.
under windows OS there is no cat command you should usetype instead of cat under windows
(**if you want to run cat command under windows please look at: https://stackoverflow.com/a/71998867/2723298 )
import os
os.system('type a.txt > copy.txt')
..or if your OS is linux and cat command didn't work anyway here are other methods to copy file..
with grep:
import os
os.system('grep "" a.txt > b.txt')
*' ' are important!
copy file with sed:
os.system('sed "" a.txt > sed.txt')
copy file with awk:
os.system('awk "{print $0}" a.txt > awk.txt')
I am having a problem using my command/commands with one instance of os.system.
Unfortunately I have to use os.system as I have no control over this, as I send the string to the os.system method. I know I should really use subprocess module for my case, but that ain't an option.
So here is what I am trying to do.
I have a string like below:
cmd = "export BASE_PATH=`pwd`; export fileList=`python OutputString.py`; ./myscript --files ${fileList}; cp outputfile $BASE_PATH/.;"
This command then gets sent to the os.system module like so
os.system(cmd)
unfortunately when I consult my log file I get something that looks like this
os.system(r"""export BASE_PATH=/tmp/bla/bla; export fileList=; ./myscript --files ; cp outputfile /.;""")
As you can see BASE_PATH seems to be working but then when I call it with the cp outputfile /.
I get a empty string
Also with my fileList I get a empty string as fileList=python OutputString.py should print out a file list to this variable.
My thoughts:
Are these bugs due to a new process for each command? Hence I loose the variable in BASE_PATH in the next command.
Also for I not sure why fileList is empty.
Is there a solution to my above problem using os.system and my command string?
Please Note I have to use os.system module. This is out of my control.