Develop Reference

Need to pass argument from sys.argv[1] as input to glob() in python

I have a situation where i need to take argument from command-line and use that string (expression) I need to print files based on that regex.
I want to use glob to parse my string as i can pass regex to filter.
excerpt from python file:
dated = sys.argv[1]
files = glob.glob(dated)
This throws me empty list
> python analysis.py <some_expression>
[]
However, if I give any value manually:
dated = '*.xlsx' # example sake
files = glob.glob(dated)
print(files)
it prints:
[<list of files conforming to the required filter>]
It's obvious that the CLI arguments that it prints above, but I want the CLI argument to work properly.
I tested manually if the arguments are actually testing and it worked, so the sys.argv[1] is working but the results are not getting parsed in the glob.glob()
any ideas if I am missing something somewhere?

The issue her is not in Python, but in the shell that invokes it. Most shells I know (definitely all Linux shells) perform glob expansion before passing arguments to the executable they spawn (your Python script, in this case). This means that, at most, sys.argv[1] would contain the first file matching the glob expression you pass, and anyway, applying glob on it would not do any good.
For example, if your work directory has files a.xlsx, b.xlsx and c.xlsx, and you invoke you code using:
python mycode.py *.xlsx
Then the shell will actually glob the argument you specified, and will pass the results to your script, making the following true:
sys.argv[1:] == [`a.xlsx`, `b.xlsx`, `c.xlsx`]
In fact, instead of explicitly invoking glob, you can simply iterate on sys.argv[1:].

Loop an existing script

I'm using a script from a third party I can't modify or show (let's call it original.py) which takes a file and produces some calculations. At the end it ouputs a result (using the print statment).
Since I have many files I decided to make a second script that gets all wanted files and runs them through the original.py
1st get list of all files to run
2nd run each file through the original.py
3rd obtain results from each file
I have the 1st and 2nd step. However, the end result only saves the calculations from the last file it read.
import sys
import original
import glob
import os
fn=str(sys.argv[1])
for filename in sys.argv[1:]:
print(filename)
ficheiros = [f for f in glob.glob(fn)]
for ficheiro in ficheiros:
original.file = bytes(ficheiro,'utf-8')
original.function()
To summarize:
Knowing I can't change the original script (which is made with a print statement) how can I obtain the results for each loop? Is there a better way than using a for loop?.
The first script can be invoked with python original.py
It requires the file to be changed manually inside the script in the original.file line.
This script outputs the result in the console and I redirect it with: python original.py > result.txt
At the moment when I try to run my script, it reads all the correct files in the folder but only returns the results for the last file.
#
(I tried to reformulate the question hopefully it's easier to understand)
#
The problem is due to a mistake in the ````ficheiros = [f for f in glob.glob(fn)]`````it's only reading one file, hence only outputting one result.
Thanks for the time.sleep() trick in the comments.
Solved:
I changed the initial part to:
fn=str(sys.argv[1])
ficheiros= []
for filename in sys.argv[1:]:
ficheiros.append(filename)
#print(filename)
and now it correctly reads all the files and it outputs all the results

Depending on your operating system there are different ways to take what is printed to the console and append it to a file.
For example on Linux, you could run this file that calls original.py for every file python yourfile.py >> outputfile.txt, which will then effectively save everything that is printed into outputfile.txt.
The syntax is similar for Windows.

I'm not quite sure what you're asking, but you could try one of these:
Either redirecting all output to a file for later use, by running the script like so: python secondscript.py > outfilename.txt
Or, and this might or might not work for you, redefining the print command to a function that outputs the result how you want, eg:
def print(x):
with open('outfile.txt','w') as f:
f.write('example: ' + x)
If you choose the second option, I recommend saving the old print function (oldprint = print) so you can restore and use the regular print later.

I don't know if I got exactly what you want. You have a first script named original.py which takes some arguments and returns things in the form of print statements and you would like to grab these prints statements in your scripts to do things?
If so, a solution could be the subprocess module:
Let's say that this is original.py:
print("Hi, I'm original.py")
print("print me!")
And this is main.py:
import subprocess
script_path = "original.py"
print("Executing ", script_path)
process = subprocess.Popen(["python3", script_path], stdout=subprocess.PIPE)
for line in process.stdout:
print(line.decode("utf8"))
You can easily add more arguments in the Popen call like ["arg1", "arg2",] etc.
Output:
Executing original.py
Hi, I'm original.py
print me!
and you can grab the lines in the main.py to do what you want with them.

Pass the result of Python script to VB.net

I have a python file:
myFile.py
def get_value(data):#pass in data as input parameter
output = process(data)#function to process data
return output
Here, output can be a float number or a string.
I want to call this python script from VB.NET. I searched the web and someone suggested
import System.Diagnostics
Process.Start("C:\python " & "myFile.py")
I am not sure if it is correct. Furthermore, it does not receive output in the python file.
What should I do?
Thank you.

If your python script is just like that your function is never called and therefore doesn't return anything.
I don't know what kind of data you plan to return, if it's simple stuff the perhaps just print it with print(get_value(data)) in the end of the file and capture the printed lines with the VB script?

Interpolate variables into external programs call (subprocess.call/Popen) in Python

I'm trying to run an external program from a Python script.
After searching and reading multiple post here I came to what seemed to be the solution.
First, I used subprocess.call function.
If I build the command this way:
hmmer1=subprocess.call("D:\Python_Scripts\HMMer3\hmmsearch.exe --tblout hmmTestTab.out SDHA.hmm Test.fasta")
The external program D:\Python_Scripts\HMMer3\hmmsearch.exe is run taking hmmTestTab.out as file name for the output and SDHA.hmm and Test.fasta as input files.
Nevertheless, if I try to replace the file names with the variables outfile, hmmprofile and fastafile (I intend to receive those variables as arguments for the Python script and use them to build the external program call),
hmmer2=subprocess.call("D:\Python_Scripts\HMMer3\hmmsearch.exe --tblout outfile hmmprofile fastafile")
Python prints an error about being unable to open the input files.
I also used "Popen" function with analogous results:
This call works
hmmer3=Popen(['D:\Python_Scripts\HMMer3\hmmsearch.exe', '--tblout','hmmTestTab.out', 'SDHA.hmm','Test.fasta'])
and this one doesn't
hmmer4=Popen(['D:\Python_Scripts\HMMer3\hmmsearch.exe', '--tblout','outfile', 'hmmprofile','fastafile'])
As result of this, I presume I need to understand which is process to follow to interpolate the variables into the call, because it seems that the problem is there.
Would any of you help me with this issue?
Thanks in advance

You have:
hmmer4=Popen(['D:\Python_Scripts\HMMer3\hmmsearch.exe', '--tblout','outfile', 'hmmprofile','fastafile'])
But that's not passing the variable outfile. It's passing a string, 'outfile'.
You want:
hmmer4=Popen(['D:\Python_Scripts\HMMer3\hmmsearch.exe', '--tblout', outfile, hmmprofile, fastafile])
And the other answer is correct, though it addresses a different problem; you should double the backslashes, or use r'' raw strings.

Try to change this:
hmmer1=subprocess.call("D:\Python_Scripts\HMMer3\hmmsearch.exe"
to
hmmer1=subprocess.call('D:\\Python_Scripts\\HMMer3\\hmmsearch.exe'
Edit
argv = ' --tblout outfile hmmprofile fastafile' # your arguments
program = [r'"D:\\Python_Scripts\\HMMer3\\hmmsearch.exe"', argv]
subprocess.call(program)

Passing Python variable to Powershell parameter using Python's Popen

I am calling a Powershell script within a Python script using Python's subprocess Popen. The Powershell script requires two input parameters: -FilePath and -S3Key. It uploads a file to AWS S3 server. If I pass in hard coded strings, the script works.
os.Popen([r'C:\\WINDOWS\\system32\\WindowsPowerShell\\v1.0\\powershell.exe','-ExecutionPolicy','RemoteSigned','./Upload.ps1 -FilePath \"C:\TEMP\test.txt\" -S3Key \"mytrialtest/test.txt\"'])
However, if I try to pass in Python string variable, the Powershell script errors out saying it can not find the file specified by the filename variable.
filename = 'C:\TEMP\test.txt'
uploadkey = 'mytrialtest/test.txt'
os.Popen([r'C:\\WINDOWS\\system32\\WindowsPowerShell\\v1.0\\powershell.exe','-ExecutionPolicy','RemoteSigned','./Upload.ps1 -FilePath \"filename\" -S3Key \"uploadkey\"'])
Any help would be greatly appreciated. Thanks.

I know, it's an old question, so this is for those who find this question via google:
The solution mentioned in comment has some risks (string injection) and might not work if there are special characters involved. Better:
import subprocess
filename = r'C:\TEMP\test.txt'
uploadkey = 'mytrialtest/test.txt'
subprocess.Popen(['powershell',"-ExecutionPolicy","RemoteSig‌​ned","-File", './Upload.ps1', '-FilePath:', filename , '-S3Key:', uploadkey])
Notice the : appended to the parameter names - in most cases it will also work without the :, but if the value starts with a dash, it will fail without the :.