Develop Reference

Execute Knime with multiple arguments in Python (subprocess.run)

Hello all,
i'm looking for a way to execute a KNIME workflow in Python in batch mode (without opening the GUI of KNIME, https://www.knime.com/faq#q12)
After hours of trying I am asking you whether you can help me in this case:
When I run the python file, it opens the Knime exe, after some seconds the knime GUI is also opened. Unfortunately, the exe is not excuting the workflow (for testing the workflow should read an csv file and save it in another file destination)
This is actual code in python 3.7:
import subprocess
subprocess.run(["C:/Program Files/KNIME/knime.exe","-consoleLog","-nosplash","-noexit","-nosave","-reset","-application org.knime.product.KNIME_BATCH_APPLICATION","-workflowDir= C:/Users/jssch/knime-workspace/testexecute"]
When i paste the following code in command line the code is working and is executed correctly (it just hands over the arguments and does not open the knime GUI):
C:\Program Files\KNIME\knime.exe" -consoleLog -noexit -nosplash -nosave -reset -application org.knime.product.KNIME_BATCH_APPLICATION -workflowDir="C:\Users\jssch\knime-workspace\testexecute"
Thanks for your help in advance!

I think you made a mistake with the -application part, they should be in different Strings. Also the -workflowDir= C:/... seems to have an extra space too.
The problematic part:
"-application org.knime.product.KNIME_BATCH_APPLICATION"
it should be:
"-application", "org.knime.product.KNIME_BATCH_APPLICATION"
Probably you do not want the -noexit argument either.
All together:
import subprocess
subprocess.run(["C:/Program Files/KNIME/knime.exe", "-consoleLog", "-nosplash", "-nosave", "-reset", "-application", "org.knime.product.KNIME_BATCH_APPLICATION", "-workflowDir=C:/Users/jssch/knime-workspace/testexecute"]
(I usually prefer the paths without spaces, strange characters, I would use a KNIME installation from a different path, though this is fine too.)

Python does not allow additional comment with env python

I have a commenting line problem in python. I created a ages.py with vim and here is my script
#!/usr/bin/env python
ages={"dad":42, "mom":35, "lisa":7}
for item in ages:
print item
When I add a comment above the !/usr/bin, like
# this is a python script
#!/usr/bin/env python
ages={"dad":42, "mom":35, "lisa":7}
for item in ages:
print item
and after I go back to directory, which includes the same script, and run the script with writing to terminal ages.py but I get this error
$ ./ages.py
./ages.py: line 3: mom:35,: command not found
./ages.py: line 5: syntax error near unexpected token `print'
./ages.py: line 5: 'print item'
I know that when I remove the #!/usr/bin/env python line # works perfectly for commenting. However, I would like to add this line to run the script only writing its name to prompt. Otherwise, I have to write python ages.py every single time to run it, which I see as a burden.
1st Q: How can I comment to .py script with still having the '#!/usr/bin/env python' line
P.S: I've already tried """ ''' before and after comment line, it does not work either.
2nd Q: Is there a way to run a .py script in python environment? For instance, I can run .m file in octave simply writing its name to command line. But I could not figure out if there is a same way in python?
P.S: I know the questions are so easy and deserve -1, however, for new python users, the all answers will provide great information, thanks for all of them. It is so subtle solution but without it, I've constantly got the errors

That first line is a very important one. It's called the Hashbang and sometimes known as the shebang. It tells the operating system what interpreter to use to execute the script. When the shebang is used, it has to be the first line. Other variations include
#!/bin/sh
#!/usr/bin/perl/
#!/usr/bin/python
These are for system default sh, perl and python. Any other comments in your code has to be after this line.

the line containing #!/usr/bin/env python must be first, you can add your comments below;
yes, you can do it like this: exec(open('yourscript.py').read()) - however, results might sometimes differ from your expectations.

python: How does subprocess.check_output create it's calls?

I'm trying to read the duration of video files using mediainfo. This shell command works
mediainfo --Inform="Video;%Duration/String3%" file
and produces an output like
00:00:33.600
But when I try to run it in python with this line
subprocess.check_output(['mediainfo', '--Inform="Video;%Duration/String3%"', file])
the whole --Inform thing is ignored and I get the full mediainfo output instead.
Is there a way to see the command constructed by subprocess to see what's wrong?
Or can anybody just tell what's wrong?

Try:
subprocess.check_output(['mediainfo', '--Inform=Video;%Duration/String3%', file])
The " in your python string are likely passed on to mediainfo, which can't parse them and will ignore the option.
These kind of problems are often caused by shell commands requiring/swallowing various special characters. Quotes such as " are often removed by bash due to shell magic. In contrast, python does not require them for magic, and will thus replicate them the way you used them. Why would you use them if you wouldn't need them? (Well, d'uh, because bash makes you believe you need them).
For example, in bash I can do
$ dd of="foobar"
and it will write to a file named foobar, swallowing the quotes.
In python, if I do
subprocess.check_output(["dd", 'of="barfoo"', 'if=foobar'])
it will write to a file named "barfoo", keeping the quotes.

is there a built-in python analog to unix 'wc' for sniffing a file?

Everyone's done this--from the shell, you need some details about a text file (more than just ls -l gives you), in particular, that file's line count, so:
# > wc -l iris.txt
149 iris.txt
i know that i can access shell utilities from python, but i am looking for a python built-in, if there is one.
The crux of my question is getting this information without opening the file (hence my reference to the unix utility *wc -*l)
(is 'sniffing' the correct term for this--i.e., peeking at a file w/o opening it?')

You can always scan through it quickly, right?
lc = sum(1 for l in open('iris.txt'))

No, I would not call this "sniffing". Sniffing typically refers to looking at data as it passes through, like Ethernet packet capture.
You cannot get the number of lines from a file without opening it. This is because the number of lines in the file is actually the number of newline characters ("\n" on linux) in the file, which you have to read after open()ing it.

Passing a multi-line string as an argument to a script in Windows

I have a simple python script like so:
import sys
lines = sys.argv[1]
for line in lines.splitlines():
print line
I want to call it from the command line (or a .bat file) but the first argument may (and probably will) be a string with multiple lines in it. How does one do this?
Of course, this works:
import sys
lines = """This is a string
It has multiple lines
there are three total"""
for line in lines.splitlines():
print line
But I need to be able to process an argument line-by-line.
EDIT: This is probably more of a Windows command-line problem than a Python problem.
EDIT 2: Thanks for all of the good suggestions. It doesn't look like it's possible. I can't use another shell because I'm actually trying to invoke the script from another program which seems to use the Windows command-line behind the scenes.

I know this thread is pretty old, but I came across it while trying to solve a similar problem, and others might as well, so let me show you how I solved it.
This works at least on Windows XP Pro, with Zack's code in a file called
"C:\Scratch\test.py":
C:\Scratch>test.py "This is a string"^
More?
More? "It has multiple lines"^
More?
More? "There are three total"
This is a string
It has multiple lines
There are three total
C:\Scratch>
This is a little more readable than Romulo's solution above.

Just enclose the argument in quotes:
$ python args.py "This is a string
> It has multiple lines
> there are three total"
This is a string
It has multiple lines
there are three total

The following might work:
C:\> python something.py "This is a string^
More?
More? It has multiple lines^
More?
More? There are three total"

This is the only thing which worked for me:
C:\> python a.py This" "is" "a" "string^
More?
More? It" "has" "multiple" "lines^
More?
More? There" "are" "three" "total
For me Johannes' solution invokes the python interpreter at the end of the first line, so I don't have the chance to pass additional lines.
But you said you are calling the python script from another process, not from the command line. Then why don't you use dbr' solution? This worked for me as a Ruby script:
puts `python a.py "This is a string\nIt has multiple lines\nThere are three total"`
And in what language are you writing the program which calls the python script? The issue you have is with argument passing, not with the windows shell, not with Python...
Finally, as mattkemp said, I also suggest you use the standard input to read your multi-line argument, avoiding command line magic.

Not sure about the Windows command-line, but would the following work?
> python myscript.py "This is a string\nIt has multiple lines\there are three total"
..or..
> python myscript.py "This is a string\
It has [...]\
there are [...]"
If not, I would suggest installing Cygwin and using a sane shell!

Have you tried setting you multiline text as a variable and then passing the expansion of that into your script. For example:
set Text="This is a string
It has multiple lines
there are three total"
python args.py %Text%
Alternatively, instead of reading an argument you could read from standard in.
import sys
for line in iter(sys.stdin.readline, ''):
print line
On Linux you would pipe the multiline text to the standard input of args.py.
$ <command-that-produces-text> | python args.py