Develop Reference

Reading Specific String from a line in a file

I have file looking like this:
face LODRxERROR
{
source R/com/int/LRxAMEexception.csv
contains R/saqf/LAWODRxERROR.ddf
contains R/bld/LAWODRxERRORtyp.h
contains R/bld/LAWODRxERRORtyp.hpp
requires LAWODRxERR
}
At the moment I'm able to read a specific line and store it. But I need to be more specific. Instead of reading the whole line. I would like to read only the file name no the directory. So, instead of reading R/bld/LAWODRxERRORtyp.hpp I would like to read only LAWODRxERRORtyp.hpp
Here is my python code so far:
with open(file) as scope:
for line in scope:
line = line.strip()
if line.startswith('contains') and line.endswith('.h') or line.endswith('.hpp'):
scopeFileList.append(line.split()[-1])
Thanks in advance

You can use the built-in function os.path.basename() to get only the file-name from a path:
from os.path import basename
with open(file) as scope:
for line in scope:
line = line.strip()
if line.startswith('contains') and line.endswith('.h') or line.endswith('.hpp'):
path = line.split()[-1]
scopeFileList.append(basename(path))

Try this,
with open("file1.txt", "r") as f:
data = [line.replace("\n","").split('/')[-1] for line in f.readlines() if '.' in line]
Output:
print(data)
['LRxAMEexception.csv',
'LAWODRxERROR.ddf',
'LAWODRxERRORtyp.h',
'LAWODRxERRORtyp.hpp']

Try this: You can use the re.search to find the file names from a path
with open('new_file.txt') as file:
for line in file:
line = line.strip()
if line.startswith('source') or line.startswith('contains'):
file_names = re.search('/(.+?)\/((\w+)\.\w+$\Z)', line).group(2)
print(file_names)
O/P:
'LRxAMEexception.csv'
'LAWODRxERROR.ddf'
'LAWODRxERRORtyp.h'
'LAWODRxERRORtyp.hpp'

How to get data from txt file in python log analysis?

I am beginner to python, I am trying to do log analysis, but I do not know how to get the txt file.
This is the code for outputting date, but these dates must be taken from the txt file :
import sys
import re
file = open ('desktop/trail.txt')
for line_string in iter(sys.stdin.readline,''):
line = line_string.rstrip()
date = re.search(r'date=[0-9]+\-[0-9]+\-[0-9]+', line)
date = date.group()
print date

You can use with statement to open a file safely and read each line with a readlines method. readlines returns a list of string.
Below code should work in your case:
import sys
import re
with open('desktop/trail.txt', 'r') as f:
for line in f.readlines():
line = line_string.rstrip()
date = re.search(r'date=[0-9]+\-[0-9]+\-[0-9]+', line)
date = date.group()
print date

you can do something like
for line in file.readlines():
don't forget about file closing! You can do it with file.close()

Importing a text file and manipulating it to create and new file

I'm writing a script I can use at work for bulk lotus note user registration.
Basically I need to manipulate a text file that contains a list of usernames into a file i can import into domino administrator.
For example I have a text file that contains,
Gordan.Freeman
Gordan.Freeman1
Gordan.Freeman2
And I need to get it looking like
Freeman;Gordan;;;12345678;D:\lotus\NotesIDs\Users\;GordanFreeman.id;SERVER01;mail\users\;GordanFreeman.nsf;;;;;;;;;;template.ntf;
Freeman1;Gordan;;;12345678;D:\lotus\NotesIDs\Users\;GordanFreeman1.id;SERVER01;mail\users\;GordanFreeman1.nsf;;;;;;;;;;template.ntf;
Freeman2;Gordan;;;12345678;D:\lotus\NotesIDs\Users\;GordanFreeman2.id;SERVER01;mail\users\;GordanFreeman2.nsf;;;;;;;;;;template.ntf;
I've only gotten as far as reading from the text file into a list, but from what i can tell I need to convert it back into a string before i can write to a new text file.
textloc = input(r" Enter the file path of your list (eg.'C:\names_list.txt) --> ")
textopen = open(textloc, 'r')
nameslistraw = textopen.read().split('\n')
nameslist = [i.split('.') for i in nameslistraw]
I've been fiddling around with this for hours. Any help would be great :)

Here is a working script that does what you appear to want.
file = open('myfile.txt', 'r')
temp = []
for line in file:
item = line.strip('\n').split('.')
temp.append(';'.join(item[::-1])+';'*3+'12345678;D:\lotus\NotesIDs\Users\;'+''.join(item)+'.id;SERVER01;mail\users\;'+''.join(item)+'.nsf;;;;;;;;;;template.ntf;')
file.close()
file = open('myfile.txt', 'w')
file.write('\n'.join(temp))
file.close()
Which transforms the following:
Gordan.Freeman
Gordan.Freeman1
Gordan.Freeman2
Into:
Freeman;Gordan;;;12345678;D:\lotus\NotesIDs\Users\;GordanFreeman.id;SERVER01;mail\users\;GordanFreeman.nsf;;;;;;;;;;template.ntf;
Freeman1;Gordan;;;12345678;D:\lotus\NotesIDs\Users\;GordanFreeman1.id;SERVER01;mail\users\;GordanFreeman1.nsf;;;;;;;;;;template.ntf;
Freeman2;Gordan;;;12345678;D:\lotus\NotesIDs\Users\;GordanFreeman2.id;SERVER01;mail\users\;GordanFreeman2.nsf;;;;;;;;;;template.ntf;

Python- need to append characters to the beginning and end of each line in text file

I should preface that I am a complete Python Newbie.
Im trying to create a script that will loop through a directory and its subdirectories looking for text files. When it encounters a text file it will parse the file and convert it to NITF XML and upload to an FTP directory.
At this point I am still working on reading the text file into variables so that they can be inserted into the XML document in the right places. An example to the text file is as follows.
Headline
Subhead
By A person
Paragraph text.
And here is the code I have so far:
with open("path/to/textFile.txt") as f:
#content = f.readlines()
head,sub,auth = [f.readline().strip() for i in range(3)]
data=f.read()
pth = os.getcwd()
print head,sub,auth,data,pth
My question is: how do I iterate through the body of the text file(data) and wrap each line in HTML P tags? For example;
<P>line of text in file </P> <P>Next line in text file</p>.

Something like
output_format = '<p>{}</p>\n'.format
with open('input') as fin, open('output', 'w') as fout:
fout.writelines( output_format(line.strip()) for line in fin )

This assumes that you want to write the new content back to the original file:
with open('path/to/textFile.txt') as f:
content = f.readlines()
with open('path/to/textFile.txt', 'w') as f:
for line in content:
f.write('<p>' + line.strip() + '</p>\n')

with open('infile') as fin, open('outfile',w) as fout:
for line in fin:
fout.write('<P>{0}</P>\n'.format(line[:-1]) #slice off the newline. Same as `line.rstrip('\n')`.
#Only do this once you're sure the script works :)
shutil.move('outfile','infile') #Need to replace the input file with the output file

in you case, you should probably replace
data=f.read()
with:
data = '\n'.join("<p>%s</p>" % l.strip() for l in f)

use data=f.readlines() here,
and then iterate over data and try something like this:
for line in data:
line="<p>"+line.strip()+"</p>"
#write line+'\n' to a file or do something else

append the and <\p> for each line
ex:
data_new=[]
data=f.readlines()
for lines in data:
data_new.append("<p>%s</p>\n" % data.strip().strip("\n"))

You could use the fileinput module to modify one or more files in-place, with optional backup file creation if desired (see its documentation for details). Here's it being used to process one file.
import fileinput
for line in fileinput.input('testinput.txt', inplace=1):
print '<P>'+line[:-1]+'<\P>'
The 'testinput.txt' argument could also be a sequence of two or more file names instead of just a single one, which could be useful especially if you're using os.walk() to generate the list of files in the directory and its subdirectories to process (as you probably should be doing).

Replace and overwrite instead of appending

I have the following code:
import re
#open the xml file for reading:
file = open('path/test.xml','r+')
#convert to string:
data = file.read()
file.write(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>",r"<xyz>ABC</xyz>\1<xyz>\2</xyz>",data))
file.close()
where I'd like to replace the old content that's in the file with the new content. However, when I execute my code, the file "test.xml" is appended, i.e. I have the old content follwed by the new "replaced" content. What can I do in order to delete the old stuff and only keep the new?

You need seek to the beginning of the file before writing and then use file.truncate() if you want to do inplace replace:
import re
myfile = "path/test.xml"
with open(myfile, "r+") as f:
data = f.read()
f.seek(0)
f.write(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>", r"<xyz>ABC</xyz>\1<xyz>\2</xyz>", data))
f.truncate()
The other way is to read the file then open it again with open(myfile, 'w'):
with open(myfile, "r") as f:
data = f.read()
with open(myfile, "w") as f:
f.write(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>", r"<xyz>ABC</xyz>\1<xyz>\2</xyz>", data))
Neither truncate nor open(..., 'w') will change the inode number of the file (I tested twice, once with Ubuntu 12.04 NFS and once with ext4).
By the way, this is not really related to Python. The interpreter calls the corresponding low level API. The method truncate() works the same in the C programming language: See http://man7.org/linux/man-pages/man2/truncate.2.html

file='path/test.xml'
with open(file, 'w') as filetowrite:
filetowrite.write('new content')
Open the file in 'w' mode, you will be able to replace its current text save the file with new contents.

Using truncate(), the solution could be
import re
#open the xml file for reading:
with open('path/test.xml','r+') as f:
#convert to string:
data = f.read()
f.seek(0)
f.write(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>",r"<xyz>ABC</xyz>\1<xyz>\2</xyz>",data))
f.truncate()

import os#must import this library
if os.path.exists('TwitterDB.csv'):
os.remove('TwitterDB.csv') #this deletes the file
else:
print("The file does not exist")#add this to prevent errors
I had a similar problem, and instead of overwriting my existing file using the different 'modes', I just deleted the file before using it again, so that it would be as if I was appending to a new file on each run of my code.

See from How to Replace String in File works in a simple way and is an answer that works with replace
fin = open("data.txt", "rt")
fout = open("out.txt", "wt")
for line in fin:
fout.write(line.replace('pyton', 'python'))
fin.close()
fout.close()

in my case the following code did the trick
with open("output.json", "w+") as outfile: #using w+ mode to create file if it not exists. and overwrite the existing content
json.dump(result_plot, outfile)

Using python3 pathlib library:
import re
from pathlib import Path
import shutil
shutil.copy2("/tmp/test.xml", "/tmp/test.xml.bak") # create backup
filepath = Path("/tmp/test.xml")
content = filepath.read_text()
filepath.write_text(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>",r"<xyz>ABC</xyz>\1<xyz>\2</xyz>", content))
Similar method using different approach to backups:
from pathlib import Path
filepath = Path("/tmp/test.xml")
filepath.rename(filepath.with_suffix('.bak')) # different approach to backups
content = filepath.read_text()
filepath.write_text(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>",r"<xyz>ABC</xyz>\1<xyz>\2</xyz>", content))