Develop Reference

Is there a way to have a python script replace a new file with old

I am a newbie python developer and I have a question.
I have a python script check if updates are available on an external webpage but I got stuck while trying to get it to override old file with new one. What the code does below is get new.py and open it as a txt file. It then searches for __VERSION__. My code for searching for new version is below. How would I replace current.py with new.py?
__VERSION__ = '1.0'
#Get new version
new_version = ''
with open('new.py') as newfile:
for line in newfile:
line = line.strip()
if '__VERSION__' in line:
_, new_version = line.split('=', maxsplit=1)
new_version = new_version.strip()
break
Thanks #user:6530979
Can anyone help me understand what to do after this?
EDIT: The goal is for current.py to override its self with new.py’s code
Any help will be appreciated!!

Overall, these solutions are hacky at best-- no guarantees.
You SHOULD look into package management/version control, because the systems already developed to do this are likely to be much more reliable.
BUT, you might be able to use os.rename. I'm not sure if this is a good solution, or how safe it is, but it worked in my quick trial.
''' current.py (V1) '''
import os
os.rename("current.py", "old.py")
os.rename("new.py", "current.py")
In this case, current.py (V1) is the file run by the user. After execution, you are left with old.py and current.py (V2).
Two things to note:
You won't have line-by-line control via this method.
You might want to make this program call another (like V1 calling V2 and then exiting), using strategies such as these.

# what you want to replace '__VERSION__' with
replace = 'what ever you want'
new_lines = []
with open('new.py',"r") as newfile:
lines = newfile.readlines()
for line in lines:
new_lines.append(line.replace("__VERSION__", replace))
with open('new.py',"w") as newfile2:
full_file_text = "".join(new_lines)
newfile2.write(full_file_text)
is this what you wanted? to replace some text in a specific line?

How to open variable file in specific folder in python?

I'm new to python and working on a way to open a file named acne.txt which is inside my define folder in the same python34 folder as my code.
The code I've written for the same is:
NN_is = [word for word,pos in tagged_sent if pos == 'NN']
print(NN_is[0])
searchfile = open(r"define/NN_is[0].txt", "r")
file_contents = searchfile.readlines()
searchfile.close()
The variable NN_is[0] when printed yields acne. Can someone help me in solving this problem.
Thank you in advance.

Use os.path.join() to concatenate the directory and file name, and use str.format() to construct the string for the file name:
import os.path
path = os.path.join('define', '{}.txt'.format(NN_is[0]))
with open(path) as searchfile:
file_contents = searchfile.readlines()
...
I also recommend that you open the file within a with statement. This ensures that the file will always be closed, even if there is an error. It's also handy because you don't need to explicitly close the file - it will be closed when the with statement goes out of scope.

IO operation failure

This is the code:
def edit(aFile):
s = ''
filename = getMediaPath() + aFile
inputfile = open(filename, 'r')
read = inputfile.readlines()
inputfile.close()
for lines in read:
lines = lines.lower()
lines = lines.replace("it's", "this is")
lines = lines.capitalize()
s = s + str(lines)
newfile = getMediaPath() + 'happyEdited.txt'
x = open(newfile, 'w')
x.write(s)
x.close()
The error I get is on the "inputfile = " line. It says:
"I/O operation failed.
I tried to read a file, and couldn't. Are you sure that file exists? If it does exist, did you specify the correct directory/folder?"**
I've tried entering aFile as a string with the media path. I've tried setting aFile equal to it's media path but nothing works. When I take the parameter out and replace aFile in the code with the name of the .txt file the code works.
Thank y'all!

A few suggestions:
You could include a checking routine for debugging, e.g.,
import os
print os.path.exists(filename)
print os.path.isfile(filename)
And also, I would recommend to use
with open(filename,'r') as inputfile:
# do your stuff
instead of
inputfile = open(filename, 'r')
# do your stuff
inputfile.close()
Because with makes sure that the file stream will be definitely closed if a problem occurs in the # do your stuff section, otherwise you have to use excepts to ensure it, which is just a little bit more effort. with is just a more convenient way.
And I think what you need to get your case to work could be:
newfile = getMediaPath() + '/happyEdited.txt'

I am just adding kwatford`s comment as answer in here. What you need to change is
filename = os.path.join(getMediaPath(),aFile)
newfile = os.path.join(getMediaPath() , 'happyEdited.txt')

The main problem here is probably that you are using simple strings that represent relative file paths. If you were to provide a full traceback, then I could give you a better answer.
Now, this will give you problems a lot of the times, and so it is best practice to always use absolute paths.
Now, what is an absolute path, you say? Well, its the path that goes all the way from your drive to your actual file destination. For example: C:/Foo/Bar/Cheese/happy.py. A relative file path is a path relative to your current directory. For example you were in your command line and you were # C:/Foo/Bar/Cheese/happy.py, and if there was another file in the same directory, say more.py, then you could reference it as ./more.py, but this can lead to several problems as you are facing right now.
So, what is the solution? Like I said, use absolute paths, now how do you do this? Well you use a module called os.
So, something like this:
import os
file_path = os.path.abspath(os.path.join(os.path.dirname(__file__), "name_of_file_in_current_dir")).replace(os.pardir, "/")
Now let me tell you what this means, os.path.abspath gives you an absolute path. os.path.join allows you to join paths in a flexible ways, you can join folders. os.path.dirname gives you the absolute path a specified file, in this case __file__. __file__ is a special variable. Finally, internally an OS can use different ways to separate directories, some use //, some use \, some use \\. Now, this is what works best /, since it works on all systems. We use os.pardir because it will work on all systems, Windows, Linux and Unix, which makes your code portable! :D
Also, a good recommendation would be using the with statement. Like so:
with open(file_path) as file:
This is the same as putting a try/catch block around it, but in once simple line. It also opens and closes the file stream for you.

Python Overwriting files after parsing

I'm new to Python, and I need to do a parsing exercise. I got a file, and I need to parse it (just the headers), but after the process, i need to keep the file the same format, the same extension, and at the same place in disk, but only with the differences of new headers..
I tried this code...
for line in open ('/home/name/db/str/dir/numbers/str.phy'):
if line.startswith('ENS'):
linepars = re.sub ('ENS([A-Z]+)0+([0-9]{6})','\\1\\2',line)
print linepars
..and it does the job, but I don't know how to "overwrite" the file with the new parsing.

The easiest way, but not the most efficient (by far, and especially for long files) would be to rewrite the complete file.
You could do this by opening a second file handle and rewriting each line, except in the case of the header, you'd write the parsed header. For example,
fr = open('/home/name/db/str/dir/numbers/str.phy')
fw = open('/home/name/db/str/dir/numbers/str.phy.parsed', 'w') # Name this whatever makes sense
for line in fr:
if line.startswith('ENS'):
linepars = re.sub ('ENS([A-Z]+)0+([0-9]{6})','\\1\\2',line)
fw.write(linepars)
else:
fw.write(line)
fw.close()
fr.close()
EDIT: Note that this does not use readlines(), so its more memory efficient. It also does not store every output line, but only one at a time, writing it to file immediately.
Just as a cool trick, you could use the with statement on the input file to avoid having to close it (Python 2.5+):
fw = open('/home/name/db/str/dir/numbers/str.phy.parsed', 'w') # Name this whatever makes sense
with open('/home/name/db/str/dir/numbers/str.phy') as fr:
for line in fr:
if line.startswith('ENS'):
linepars = re.sub ('ENS([A-Z]+)0+([0-9]{6})','\\1\\2',line)
fw.write(linepars)
else:
fw.write(line)
fw.close()
P.S. Welcome :-)

As others are saying here, you want to open a file and use that file object's .write() method.
The best approach would be to open an additional file for writing:
import os
current_cfg = open(...)
parsed_cfg = open(..., 'w')
for line in current_cfg:
new_line = parse(line)
print new_line
parsed.cfg.write(new_line + '\n')
current_cfg.close()
parsed_cfg.close()
os.rename(....) # Rename old file to backup name
os.rename(....) # Rename new file into place
Additionally I'd suggest looking at the tempfile module and use one of its methods for either naming your new file or opening/creating it. Personally I'd favor putting the new file in the same directory as the existing file to ensure that os.rename will work atomically (the configuration file named will be guaranteed to either point at the old file or the new file; in no case would it point at a partially written/copied file).

The following code DOES the job.
I mean it DOES overwrite the file ON ONESELF; that's what the OP asked for. That's possible because the transformations are only removing characters, so the file's pointer fo that writes is always BEHIND the file's pointer fi that reads.
import re
regx = re.compile('\AENS([A-Z]+)0+([0-9]{6})')
with open('bomo.phy','rb+') as fi, open('bomo.phy','rb+') as fo:
fo.writelines(regx.sub('\\1\\2',line) for line in fi)
I think that the writing isn't performed by the operating system one line at a time but through a buffer. So several lines are read before a pool of transformed lines are written. That's what I think.

newlines = []
for line in open ('/home/name/db/str/dir/numbers/str.phy').readlines():
if line.startswith('ENS'):
linepars = re.sub ('ENS([A-Z]+)0+([0-9]{6})','\\1\\2',line)
newlines.append( linepars )
open ('/home/name/db/str/dir/numbers/str.phy', 'w').write('\n'.join(newlines))

(sidenote: Of course if you are working with large files, you should be aware that the level of optimization required may depend on your situation. Python by nature is very non-lazily-evaluated. The following solution is not a good choice if you are parsing large files, such as database dumps or logs, but a few tweaks such as nesting the with clauses and using lazy generators or a line-by-line algorithm can allow O(1)-memory behavior.)
targetFile = '/home/name/db/str/dir/numbers/str.phy'
def replaceIfHeader(line):
if line.startswith('ENS'):
return re.sub('ENS([A-Z]+)0+([0-9]{6})','\\1\\2',line)
else:
return line
with open(targetFile, 'r') as f:
newText = '\n'.join(replaceIfHeader(line) for line in f)
try:
# make backup of targetFile
with open(targetFile, 'w') as f:
f.write(newText)
except:
# error encountered, do something to inform user where backup of targetFile is
edit: thanks to Jeff for suggestion

Replace string in a specific line using python

I'm writing a python script to replace strings from a each text file in a directory with a specific extension (.seq). The strings replaced should only be from the second line of each file, and the output is a new subdirectory (call it clean) with the same file names as the original files, but with a *.clean suffix. The output file contains exactly the same text as the original, but with the strings replaced. I need to replace all these strings: 'K','Y','W','M','R','S' with 'N'.
This is what I've come up with after googling. It's very messy (2nd week of programming), and it stops at copying the files into the clean directory without replacing anything. I'd really appreciate any help.
Thanks before!
import os, shutil
os.mkdir('clean')
for file in os.listdir(os.getcwd()):
if file.find('.seq') != -1:
shutil.copy(file, 'clean')
os.chdir('clean')
for subdir, dirs, files in os.walk(os.getcwd()):
for file in files:
f = open(file, 'r')
for line in f.read():
if line.__contains__('>'): #indicator for the first line. the first line always starts with '>'. It's a FASTA file, if you've worked with dna/protein before.
pass
else:
line.replace('M', 'N')
line.replace('K', 'N')
line.replace('Y', 'N')
line.replace('W', 'N')
line.replace('R', 'N')
line.replace('S', 'N')

some notes:
string.replace and re.sub are not in-place so you should be assigning the return value back to your variable.
glob.glob is better for finding files in a directory matching a defined pattern...
maybe you should be checking if the directory already exists before creating it (I just assumed this, this could not be your desired behavior)
the with statement takes care of closing the file in a safe way. if you don't want to use it you have to use try finally.
in your example you where forgetting to put the sufix *.clean ;)
you where not actually writing the files, you could do it like i did in my example or use fileinput module (which until today i did not know)
here's my example:
import re
import os
import glob
source_dir=os.getcwd()
target_dir="clean"
source_files = [fname for fname in glob.glob(os.path.join(source_dir,"*.seq"))]
# check if target directory exists... if not, create it.
if not os.path.exists(target_dir):
os.makedirs(target_dir)
for source_file in source_files:
target_file = os.path.join(target_dir,os.path.basename(source_file)+".clean")
with open(source_file,'r') as sfile:
with open(target_file,'w') as tfile:
lines = sfile.readlines()
# do the replacement in the second line.
# (remember that arrays are zero indexed)
lines[1]=re.sub("K|Y|W|M|R|S",'N',lines[1])
tfile.writelines(lines)
print "DONE"
hope it helps.

You should replace line.replace('M', 'N') with line=line.replace('M', 'N'). replace returns a copy of the original string with the relevant substrings replaced.
An even better way (IMO) is to use re.
import re
line="ABCDEFGHIJKLMNOPQRSTUVWXYZ"
line=re.sub("K|Y|W|M|R|S",'N',line)
print line

Here are some general hints:
Don't use find for checking the file extension (e.g., this would also match "file1.seqdata.xls"). At least use file.endswith('seq'), or, better yet, os.path.splitext(file)[1]
Actually, don't do that altogether. This is what you want:
import glob
seq_files = glob.glob("*.seq")
Don't copy the files, it's much easier to use just one loop:
for filename in seq_files:
in_file = open(filename)
out_file = open(os.path.join("clean", filename), "w")
# now read lines from in_file and write lines to out_file
Don't use line.__contains__('>'). What you mean is
if '>' in line:
(which will call __contains__ internally). But actually, you want to know wether the line starts with a `">", not if there's one somewhere within the line, be it at the beginning or not. So the better way would be this:
if line.startswith(">"):
I'm not familiar with your file type; if the ">" check really is just for determining the first line, there's better ways to do that.
You don't need the if block (you just pass). It's cleaner to write
if not something:
do_things()
other_stuff()
instead of
if something:
pass
else:
do_things()
other_stuff()
Have fun learning Python!

you need to allocate the result of the replacement back to "line" variable
line=line.replace('M', 'N')
you can also use the module fileinput for inplace edit
import os, shutil,fileinput
if not os.path.exists('clean'):
os.mkdir('clean')
for file in os.listdir("."):
if file.endswith(".seq"):
shutil.copy(file, 'clean')
os.chdir('clean')
for subdir, dirs, files in os.walk("."):
for file in files:
f = fileinput.FileInput(file,inplace=0)
for n,line in enumerate(f):
if line.lstrip().startswith('>'):
pass
elif n==1: #replace 2nd line
for repl in ["M","K","Y","W","R","S"]:
line=line.replace(ch, 'N')
print line.rstrip()
f.close()
change inplace=0 to inplace=1 for in place editing of your files.

line.replace is not a mutator, it leaves the original string unchanged and returns a new string with the replacements made. You'll need to change your code to line = line.replace('R', 'N'), etc.
I think you also want to add a break statement at the end of your else clause, so that you don't iterate over the entire file, but stop after having processed line 2.
Lastly, you'll need to actually write the file out containing your changes. So far, you are just reading the file and updating the line in your program variable 'line'. You need to actually create an output file as well, to which you will write the modified lines.