Develop Reference

grep bash command into Python script

There is a bash command, I am trying to convert the logic into python. But I don't know what to do, I need some help with that.
bash command is this :
ls
ls *
TODAY=`date +%Y-%m-%d`
cd xx/xx/autotests/
grep -R "TEST_F(" | sed s/"TEST_F("/"#"/g | cut -f2 -d "#" | while read LINE
The logic is inside a directory, reads the filename one by one, includes all sub-folders, then lists the file matches. Any help here will be much appreciated
I tried something like follows, but it is not what I would like to have. And there are some subfolders inside, which the code is not reading the file inside them
import fnmatch
import os
from datetime import datetime
time = datetime.now()
dir_path = "/xx/xx/autotests"
dirs = os.listdir(dir_path)
TODAY = time.strftime("%Y-%m-%d")
filesOfDirectory = os.listdir(dir_path)
print(filesOfDirectory)
pattern = "TEST_F("
for file in filesOfDirectory:
if fnmatch.fnmatch(file, pattern):
print(file)

Use os.walk() to scan the directory recursively.
Open each file, loop through the lines of the file looking for lines with "TEST_F(". Then extract the part of the line after that (that's what sed and cut are doing).
for root, dirs, files in os.walk(dir_path):
for file in files:
with open(os.path.join(root, file)) as f:
for line in f:
if "TEST_F(" in line:
data = line.split("TEST_F(")[1]
print(data)

Python's os.system(command) and dashes inside a string

I want to run some command with os.system().
I have prepared string "command" (includes some dashes, slashes) and try to do:
print(command)
os.system(command)
My output:
"C:\Program Files (x86)\SimInTech\bin\mmain.exe" "Aux Systems.prt" /saveas "Aux Systems.xprt" /exit
"C:\Program" не является внутренней или внешней
командой, исполняемой программой или пакетным файлом.
("C:\Program" is not an inner or outter command, executable program or package file.)
If I copy+paste in a command line this (output of print(command)):
"C:\Program Files (x86)\SimInTech\bin\mmain.exe" "Aux Systems.prt" /saveas "Aux Systems.xprt" /exit
It works fine.
So, what is the problem and how to avoid it?
Some details of how I prepare my "command" string:
import sys
import os
import codecs
simintech = '"C:\\Program Files (x86)\\SimInTech\\bin\\mmain.exe"'
argfile = sys.argv[1:len(sys.argv)]
file = ' '.join(argfile)
file = file.strip()
filepath, filename = os.path.split(file)
if filepath !='':
filepath = filepath+"\\"
oldname = filename
newname = filename.replace('.prt','.xprt')
command = simintech + ' "' + filepath +oldname+'" /saveas "'+filepath+newname+'" /exit'
Explanation:
I run my script with filename as an argument.
Filename could contain spaces, so I collect all args and put it in one string using ' '.join(argfile)
After that I separate flepath and filename (if filename was full - with absolute path).
And finally I form my command with path to my program, dash, fullname of a file, option \saveas, again fullname with path but with other file extension, dash and option \exit.

You should not use os.system, but subprocess.call:
import subprocess
subprocess.call(["C:\Program Files (x86)\SimInTech\bin\mmain.exe", "Aux Systems.prt", "/saveas", "Aux Systems.xprt", "/exit"]
It will make sure that all spaces are escaped properly and will call the according program with the according arguments. (First of list is program, rest is arguments)

Uncompyle6 convert pyc to py file python 3 (Whole directory)

I have 200 pyc files I need to convert in a folder. I am aware of converting pyc to py files through uncompyle6 -o . 31.pyc however as I have so many pyc files, this would take a long period of time. I've founds lots of documentation but not much in bulk converting to py files. uncompyle6 -o . *.pyc was not supported.
Any idea on how I can achieve this?

Might not be perfect but it worked great for me.
import os
import uncompyle6
your_directory = ''
for dirpath, b, filenames in os.walk(your_directory):
for filename in filenames:
if not filename.endswith('.pyc'):
continue
filepath = dirpath + '/' + filename
original_filename = filename.split('.')[0]
original_filepath = dirpath + '/' + original_filename + '.py'
with open(original_filepath, 'w') as f:
uncompyle6.decompile_file(filepath, f)

This is natively supported by uncompyle6
uncompyle6 -ro <output_directory> <python_directory>
-r tells the tool to recurse into sub directories.
-o tells the tool to output to the given directory.

In operating systems with shell filename expansion, you might be able to use the shell's file expansion ability. For example:
uncompyle6 -o /tmp/unc6 myfiles/*.pyc
If you need something fancier or more control, you could always write some code that does the fancier expansion. Here is the above done in POSIX shell filtering out the single file myfiles/huge.pyc:
cd myfiles
for pyc in *.pyc; do
if [[ $pyc != huge.pyc ]] ; then
uncompyle -o /tmp/unc $pyc
fi
done
Note: It seems this question was also asked in Issue on output directory while executing commands with windows batch command "FOR /R"

thank you for the code, extending it to recursively call, nested sub directories, save as uncompile.py, in the directory to be converted, to run in command prompt type "python uncomple.py" would convert pyc to py in current working directory, with error handling and if rerun skips (recovery) files checking existing py extension match
import os
import uncompyle6
#Use current working directory
your_directory = os.getcwd()
#function processing current dir
def uncompilepath(mydir):
for dirpath, b, filenames in os.walk(mydir):
for d in b:
folderpath = dirpath + '/' + d
print(folderpath)
#recursive sub dir call
uncompilepath(folderpath)
for filename in filenames:
if not filename.endswith('.pyc'):
continue
filepath = dirpath + '/' + filename
original_filename = filename.split('.')[0]
original_filepath = dirpath + '/' + original_filename + '.py'
#ignore if already uncompiled
if os.path.exists(original_filepath):
continue
with open(original_filepath, 'w') as f:
print(filepath)
#error handling
try:
uncompyle6.decompile_file(filepath, f)
except Exception:
print("Error")
uncompilepath(your_directory)

execution of .jar file from python doesn't work

I am trying to minify multiple js files by yuicompressor.jar called from python.
Here is the code:
import os, subprocess
root = r"c:\myfloder"
minifier = '"c:\yuicompressor-2.4.7.jar"'
for path, subdirs, files in os.walk(root):
for name in files:
fullFile = os.path.join(path, name)
if name.endswith(".js"):
minifyCommand = ' --disable-optimizations '+'"'+fullFile+'"' +" -o "+'"'+fullFile+"min"+'"'
subprocess.call(["java","-jar",minifier,minifyCommand])
print "done"
The only problem is that it does't do the job, I mean the output files are not created. I have printed that and put it into command line and everything worked. Any idea?

Running a Bash command on files with Python

So I'm creating a script that needs to go through all the files on a server and run each of those files names followed by the command "ll" and then take the output of that command and print it to a txt file.
example:
folder/filename.txt ll
output: SoMETHINGSomethingSomethingother - This is sent to a output.txt file
folder/subfolder/filename3.txt ll
output: SoMETHINGSomethingSomethingother - This is sent to a output.txt file
This is what I have so far:
import os
with open("output.txt", "w") as a:
for path, subdirs, files in os.walk(r'C:\Users\user\Desktop\Test_Py'):
for filename in files:
f = os.path.join(filename)
m = f + ' ll'
a.write(str(m) + os.linesep)
So what I'm trying to figure out now is how to make the file names that are printed out run with the "ll" command. So far this code will write the names of all the files in that folder and its subfolders in to my output.txt file.
Does anybody have any ideas?

Use os.system():
import os
with open("output.txt", "w") as a:
for path, subdirs, files in os.walk(r'C:\Users\user\Desktop\Test_Py'):
for filename in files:
f = os.path.join(filename)
m = f + ' ll > output.txt'
os.system(m)
This will send only the standard output to the output.txt file. If you want to send error messages to output.txt as well, use m = f + ' ll > output.txt 2>&1' instead.
Explanation: os.system(command_string) will execute the command command_string in your system as if you typed that command into a terminal. The > operator is standard in Windows and Linux to redirect standard output from a command into a file. The 2>&1 extra argument at the end is the only not-so-clear part: it redirects standard error to the same place where standard output is going. See more about this last part here.

In order to run the files with the "ll" command you can use subprocess module available in python.
Your revised code will be:-
import os
import subprocess
import shlex
with open("output.txt", "w") as a:
for path, subdirs, files in os.walk(r'C:\Users\user\Desktop\Test_Py'):
for filename in files:
f = os.path.join(filename)
m = f + ' ll'
cmd_args = shlex.split(m)
output = subprocess.check_output(cmd_args)
a.write(output + os.linesep)