Develop Reference

Linux bash executables behave differently depending on whether I am activating from the command line or with os.system('')

I've been attempting to execute a certain CLI from within python and store the output for later use within the same script. I suspect this question has a simple answer, but if one wishes to go through the entire pipeline, here is the tool in question.
wget http://rna.urmc.rochester.edu/Releases/current/RNAstructureForLinux.tgz
tar xvf as usual, go inside the resulting directory and execute 'make all', the executables I use in the bash script are within the 'exe' directory.
I attempted to execute the commands with os.system(), but with little luck. The CLI I am using; however, seems to be running. The function which I have set to execute the os.system() commands contains the following block.
txt = open('home/spectre/tools/RNAstructure/exe/RNAStructure_nucleic_acid.txt',"w")
txt.write('AAGGCTGTCCAGGCGCAATGTGGTGGCTGCTTCTCTGGGGAGTCCTCCAGGCTTGCCCAACCCGGGGCTCCGTCCTCTTGGCCCAAGAGCTACCCCAGCAGCTGACATCCCCCGGGTACCCAGAGCCGTATGGCAAAGGCCAAGAGAGCAGCACGGACATCAAGGCTCCAGAGGGCTTTGCTGTGAGGCTCGTCTTCCAGGACTTCGACCTGGAGCCGTCCCAGGACTGTGCAGGGGACTCTGTCACAGTGAGCTGGGGATGGGGGGGGTCCCGCCAGGACTGTGGCCAGGGAGATTCCCGGGGTTGTGGGAAGTGGCGGTGCCCTGAATCCCCCATCTGGAGGAGGGATGAAT')
os.system(' cd ~/tools/RNAstructure/exe ; ./python_RNA_structure.sh')
nucleotides, structure, MFE =
RNAStructure_from_file('home/spectre/tools/RNAstructure/exe/RNAStructure_bracket_output.txt')
The executable *.sh file contains this.
#!/bin/bash
cd ~/tools/RNAstructure/exe
./Fold RNAStructure_nucleic_acid.txt RNAStructure_nucleic_acid_output.txt
./ct2dot RNAStructure_nucleic_acid_output.txt -1 RNAStructure_bracket_output.txt
If I execute the bash script from the command line the output should look a little like this
Initializing nucleic acids...
Using auto-detected DATAPATH: "../data_tables" (set DATAPATH to avoid this warning).
done.
98% \[==================================================\] \\ done.
Writing output ct file...done.
Single strand folding complete.
Converting CT file...
Using auto-detected DATAPATH: "../data_tables" (set DATAPATH to avoid this warning).
CT file conversion complete.
If I execute the bash script form the python file.
Initializing nucleic acids...
Using auto-detected DATAPATH: "../data_tables" (set DATAPATH to avoid this warning).
Error reading sequence. The file did not contain any nucleotides.
Single strand folding complete with errors.
Converting CT file...
Using auto-detected DATAPATH: "../data_tables" (set DATAPATH to avoid this warning).
CT file conversion complete.
It looks an awful lot like my CLI can find the files it needs inside the terminal, but not outside of it. I haven't experimented with any parameters like trying absolute paths, but I understood by using os.system() I could execute a bash script, but it is not clear to me why this is changing how that script behaves.
What I've done to resolve the problem:
reopening the file seems to resolve the problem, but I am still working out why.

The problem seems to resolve when I reopen the file within the python script like so:
txt = open('home/spectre/tools/RNAstructure/exe/RNAStructure_nucleic_acid.txt',"w")
txt.write('AAGGCTGTCCAGGCGCAATGTGGTGGCTGCTTCTCTGGGGAGTCCTCCAGGCTTGCCCAACCCGGGGCTCCGTCCTCTTGGCCCAAGAGCTACCCCAGCAGCTGACATCCCCCGGGTACCCAGAGCCGTATGGCAAAGGCCAAGAGAGCAGCACGGACATCAAGGCTCCAGAGGGCTTTGCTGTGAGGCTCGTCTTCCAGGACTTCGACCTGGAGCCGTCCCAGGACTGTGCAGGGGACTCTGTCACAGTGAGCTGGGGATGGGGGGGGTCCCGCCAGGACTGTGGCCAGGGAGATTCCCGGGGTTGTGGGAAGTGGCGGTGCCCTGAATCCCCCATCTGGAGGAGGGATGAAT')
txt = open('home/spectre/tools/RNAstructure/exe/RNAStructure_nucleic_acid.txt')
os.system(' cd ~/tools/RNAstructure/exe ; ./python_RNA_structure.sh')
nucleotides, structure, MFE =
RNAStructure_from_file('home/spectre/tools/RNAstructure/exe/RNAStructure_bracket_output.txt')
I am not sure why this resolves the problem, I found this solution serendipitously. I'll update the answer when I figure out why, unless someone wants to beat me to it. It's magic to me for now.
It seems that after opening the file, RNAStructure_nucleic_acid.txt, and assigning it to the txt variable for writing, I need to reopen it after writing is complete. Otherwise the file is blank when I try printing it's output within the program, but after the program finishes executing, the file contains the correct text.

Python - script outputs wrong .xml file

I have a python script that makes a .xml file. I am running this script on my server. And it runs fine and at the end of the script I put the generated file on my server like this.
output_path = "/path_to_my_loication/"
ofname = "name_of_file.xml"
output_file = output_path + ofname
open(output_file, "w").write(str(BeautifulSoup(get_xml(menu_url_list[0][1]))))
and is just part of making the file, pretty sure its irrelevant to this. So run the script on my server and it outputs my file, and I can see it, but the url is not correct instead of it being.
myserver/path_to_my_loication/name_of_file.xml
it is
myserver/path_to_my_loication/%0d%0dname_of_file.xml
This is being added %0d%0d before the file name I am not sure how to fix this, and when I print my file name just before I write the file, the characters are not there? How can I get rid of this/ why does this happen.
When I just run the script on my laptop the file name does not contain theses characters. Is it a server side issues that I should contact my serve provider about?
Thanks for the help.
Okay AIG cause when I printed for my server I saw this, I though enter was just some weird thing
How can I get rid of the enters?
Here is full code
https://gist.github.com/spennyf/de3349252695cecd4e6c

Opening .exe from .txt error

I'm currently creating a script that will simply open a program in the SAME directory as the script. I want to have a text file named "target.txt", and basically the script will read what's in "target.txt" and open a file based on its contents.
For example.. The text file will read "program.exe" inside, and the script will read that and open program.exe. The reason I'm doing this is to easily change the program the script opens without having to actually change whats inside.
The current script Im using for this is:
import subprocess
def openclient():
with open("target.txt", "rb") as f:
subprocess.call(f.read())
print '''Your file is opening'''
Its giving me an error saying it cannot find target.txt, even though I have it in the same directory. I have tried taking away the .txt, still nothing. This code actually worked before, however; it stopped working for some strange reason. I'm using PythonWin compiler instead of IDLE, I don't know if this is the reason.

There are two possible issues:
target.txt probably ends with a newline, which messes up subprocess.call()
If target.txt is not in the current directory, you can access the directory containing the currently executing Python file by parsing the magic variable __file__.
However, __file__ is set at script load time, and if the current directory is changed between loading the script and calling openclient(), the value of __file__ may be relative to the old current directory. So you have to save __file__ as an absolute path when the script is first read in, then use it later to access files in the same directory as the script.
This code works for me, with target.txt containing the string date to run the Unix date command:
#!/usr/bin/env python2.7
import os
import subprocess
def openclient(orig__file__=os.path.abspath(__file__)):
target = os.path.join(os.path.dirname(orig__file__), 'target.txt')
with open(target, "rb") as f:
subprocess.call(f.read().strip())
print '''Your file is opening'''
if __name__ == '__main__':
os.chdir('foo')
openclient()

Python code not writing to file unless run in interpreter

I have written a few lines of code in Python to see if I can make it read a text file, make a list out of it where the lines are lists themselves, and then turn everything back into a string and write it as output on a different file. This may sound silly, but the idea is to shuffle the items once they are listed, and I need to make sure I can do the reading and writing correctly first. This is the code:
import csv,StringIO
datalist = open('tmp/lista.txt', 'r')
leyendo = datalist.read()
separando = csv.reader(StringIO.StringIO(leyendo), delimiter = '\t')
macrolist = list(separando)
almosthere = ('\t'.join(i) for i in macrolist)
justonemore = list(almosthere)
arewedoneyet = '\n'.join(justonemore)
with open('tmp/randolista.txt', 'w') as newdoc:
newdoc.write(arewedoneyet)
newdoc.close()
datalist.close()
This seems to work just fine when I run it line by line on the interpreter, but when I save it as a separate Python script and run it (myscript.py) nothing happens. The output file is not even created. After having a look at similar issues raised here, I have introduced the 'with' parameter (before I opened the output file through output = open()), I have tried flushing as well as closing the file... Nothing seems to work. The standalone script does not seem to do much, but the code can't be too wrong if it works on the interpreter, right?
Thanks in advance!
P.S.: I'm new to Python and fairly new to programming, so I apologise if this is due to a shallow understanding of a basic issue.

Where are the input file and where do you want to save the output file. For this kind of scripts i think that it's better use absolute paths
Use:
open('/tmp/lista.txt', 'r')
instead of:
open('tmp/lista.txt', 'r')
I think that the error can be related to this

It may have something to do with where you start your interpreter.
Try use a absolute path /tmp/randolista.txt instead of relative path tmp/randolista.txt to isolate the problem.

Running Python script using WshShell.Run in VBS does not generate output file

I've got a problem with some VB scripting - it doesn't seem like it should be terribly difficult to solve, but even after trudging through many a page of Google I have yet to find a solution.
[The problem]
Here is my python file (test.py), simplified to just show the problem:
f = open("testing.txt", 'w')
f.write("oh hai\n")
f.close()
Of course, when run directly from the command line, this generates the file as you'd expect.
However, when run in a simple .vbs script (WARNING: My vbs skills are lacking. This is probably why I am having a problem. So far I haven't had many issues, apart from hating life from using XP to code when I'm used to using vim)
Set WshShell = WScript.CreateObject("WScript.Shell")
cmd = "C:\Python27\python test.py"
WshShell.Run cmd
no output file is generated! At all! It's infuriating, as when I input that exact command ("C:\Python27\python test.py") into the run program from the start menu, it works!
[System info]
At work, so they're on Windows XP. Everything else is pretty standard, or so I'm lead to believe.
EDIT: Changed "C:\Python27\testing.py" to just "testing.py". This was left over from when I was trying to solve it, and thought maybe it was putting the files somewhere outside of the destination folder.

First, your Python script looks suspicious, I doubt the backslashes work in a simple string. At least, in my test, it didn't work, I just replaced them with forward slashes.
Next, you can see the error message by prepending cmd with cmd /k, the run window remains on screen. You can see the .py file isn't found, because it isn't in the current directory. You have to specify an absolute path to this file, perhaps by getting the current path from the script.
[EDIT] I finally got a working code (my VBS is a bit rusty...)
Dim wshShell, fso, loc, cmd
Set fso = CreateObject("Scripting.FileSystemObject")
loc = fso.GetAbsolutePathName(".")
WScript.Echo loc
'~ cmd = "%ComSpec% /k C:\Languages\Python\python.exe " + loc + "\test.py"
cmd = "C:\Languages\Python\python.exe " + loc + "\test.py"
WScript.Echo cmd
Set wshShell = CreateObject("WScript.Shell")
wshShell.Run cmd
You can also check the arguments if a path is provided:
if WScript.Arguments.Count = 0 then
loc = fso.GetAbsolutePathName(".")
else
loc = WScript.Arguments(0)
end if
Such script is better run with cscript instead of default wscript.

Try
f = open("C:\\Python27\\testing.txt", 'w')
instead of your first line.