I am trying to alter an existing ASCII data file in a specific way.
The way I would like to go is to find find either one string from an array, which I define beforehand.
If this string is found in the file I would like to change the preceding entry; the string to put in here depends on which of the strings is found in the first place.
I have a file, where the entrys are separated by spaces and I have trailing spaces at the end to fill up 30 columns. The respective strings would not be in the first line and there would never be more than one per line. An example could look like this:
test01out.txt:
a0997 b0998 c0999
a1000 b1001 c1002
a1003 b1004 c1005
a1006 a1000 c1007
a1008 b1009 c1010
b1001 b1011 c1012
a1013 b1014 b1001
a1015 b1016 c1017
The file does not necessarily have to have three columns in a row. It is possible, that a row has only two but can also have four or five columns.
My current attempt was the following:
from numpy import *
findlines = open("test01.txt").read().split("\n")
searcharray = array(["a1000","b1001"])
alterarray = array(["this1","this2"])
tempstring_current = ""
fileout = open("test01out.txt", "w")
for i, line in enumerate(findlines):
tempstring_last = tempstring_current
tempstring_current = line.rstrip().split(" "))
if any(x in tempstring_current for x in searcharray): # check if one of the elements is in the current line -> unfortunately this seems to be true for any line checked...
print(i)
print(tempstring_current)
for j, element in enumerate(tempstring_current):
if any(searcharray == tempstring_current):
currentsearchindex = argmax(searcharray == tempstring_current)
currentalterstring = alterarray[currentsearchindex]
if currentsearchindex == 0:
tempstring_last.split(" ")[-1] = currentalterstring
else:
tempstring_current.split(" ")[currentsearchindex - 1] = currentalterstring
tempstring_current.split(" ")[currentsearchindex-1] = "XPRZeugs_towrite" + repr(currentdesignatedspeed)
tempstring_last = tempstring_last.ljust(30)
try:
fileout.write(str(tempstring_last))
fileout.write("\r")
try:
fileout.close()
searcharray and alterarray would have some more elements, than two.
I have tested the script up to the any condition; unfortunately the any conditions seems to be met always for some reason I do not quite understand:
from numpy import *
findlines = open("test01.txt").read().split("\n")
searcharray = array(["a1000","b1001"])
alterarray = array(["this1","this2"])
tempstring_current = ""
fileout = open("test01out.txt", "w")
for i, line in enumerate(findlines):
tempstring_last = tempstring_current
tempstring_current = line.rstrip().split(" ")
if any(x in tempstring_current for x in searcharray): # check if one of the elements is in the current line -> unfortunately this seems to be true for any line checked...
print(i)
print(tempstring_current)
I get the lines printed for every line in the file, which I did not expect.
Edit/Solution:
I realized I made a mistake in the input testfile:
It should look like this:
a0997 b0998 c0999
a1000 b1001 c1001
a1003 b1004 c1005
a1006 a1000 c1007
a1008 b1009 c1010
c1002 b1011 c1012
a1013 b1014 c1002
a1015 b1016 c1017
The full code doing the job is the following:
from numpy import *
findlines = open("test01.txt").read().split("\n")
searcharray = array(["a1000","c1002"])
alterarray = array(["this1","this2"])
tempstring_current = ""
fileout = open("test01out.txt", "w")
for i, line in enumerate(findlines):
tempstring_last = tempstring_current
tempstring_current = line.rstrip().split(" ")
if any([x in tempstring_current for x in searcharray]): # check if one of the elements is in the current line -> unfortunately this seems to be true for any line checked...
# print(i)
# print(tempstring_current)
# print(searcharray)
# print([x in tempstring_current for x in searcharray])
# print(argmax([x in tempstring_current for x in searcharray]))
currentsearchposindex = argmax([x in tempstring_current for x in searcharray]) # welchen Index hat das entsprechende Element im Searcharray?
currentalterstring = alterarray[currentsearchposindex] # was ist der entsprechende Eintrag im Alterarray
for j, currentXPRelement in enumerate(tempstring_current):
if currentXPRelement == searcharray[currentsearchposindex]:
currentsearchindex_intemparray = j
# print(len(tempstring_current))
# print(searcharray[currentsearchposindex])
# print(tempstring_current == searcharray[currentsearchposindex])
# print(searcharray[currentsearchposindex] == tempstring_current)
# print(argmax(tempstring_current == searcharray[currentsearchposindex]))
# currentsearchindex_intemparray = argmax(tempstring_current == searcharray[currentsearchposindex])
if currentsearchindex_intemparray == 0:
tempstring_last[-1] = currentalterstring
else:
tempstring_current[currentsearchindex_intemparray - 1] = currentalterstring
# tempstring_current[currentsearchindex_intemparray-1] = "XPRZeugs_towrite" + repr(currentalterstring)
tempstring_last = str(" ".join(tempstring_last)).ljust(30)
if not i == 0:
try:
fileout.write(str(tempstring_last))
fileout.write("\r")
finally:
None
try:
fileout.write(" ".join(tempstring_current))
fileout.write("\r")
fileout.close()
finally:
None
To fix your code so at least it can fail to always match, change
if any(x in tempstring_current for x in searcharray):
to
if any([x in tempstring_current for x in searcharray]):
I think the reason is that the 'x in tempstring_current for x in searcharray' expression returns an interator function - any() says 'this value (i.e. the iterator function reference) is not None so it is True', so the result is always True. The changed syntax creates a list from the iterator and then any works as you probably wanted, i.e. it returns true if any element in the list is true.
Related
I want to find 2 elements in the first page of a pdf file.
The elements that start with P and the elements that start with N.
Even if 1 element has been found, the search for the other element still needs to be continued.
If P is found, continue searching for N
If N is found, continue searching for P
When P and N are found, stop searching
I am having troubles with my loops, N is only found if there is no P.
I get that there is a issue with the if-statements, but I can't seem to correct it.
if text is not None:
p_new = "NoPFound"
n_new = "NoNFound"
for line in text.split('\n'):
#get p
if line.startswith('P '):
pieces = line.split()
p_old = pieces[1]
p_new = (p_old[7:11]) + (p_old[0:6])
break
#get n
if line.startswith('N '):
pieces = line.split()
n_old = pieces[1]
n_new = (n_old[0:15]) + "/" + (n_old[18:20])
break
list_parsed_items.append([p_new, n_new])
Use flags. You have a small state machine.
if text is not None:
p_new = None
n_new = None
for line in text.splitlines():
#get p
if not p_new and line.startswith('P '):
pieces = line.split()
p_old = pieces[1]
p_new = (p_old[7:11]) + (p_old[0:6])
#get n
if not n_new and line.startswith('N '):
pieces = line.split()
n_old = pieces[1]
n_new = (n_old[0:15]) + "/" + (n_old[18:20])
if p_new and n_new:
break
list_parsed_items.append([p_new, n_new])
I am trying to solve the "Consensus and Profile" challenge on Rosalind.
The challenge instructions are as follows:
Given: A collection of at most 10 DNA strings of equal length (at most 1 kbp) in FASTA format.
Return: A consensus string and profile matrix for the collection. (If several possible consensus strings exist, then you may return any one of them.)
My code is as follows (I got most of it from another user on this website). My only issue is that some of the DNA strands are broken down into multiple separate lines, so they are being appended to the "allstrings" list as separate strings. I am trying to figure out how to write each consecutive line that does not contain ">" as a single string.
import numpy as np
seq = []
allstrings = []
temp_seq = []
matrix = []
C = []
G = []
T = []
A = []
P = []
consensus = []
position = 1
file = open("C:/Users/knigh/Documents/rosalind_cons (3).txt", "r")
conout = open("C:/Users/knigh/Documents/consensus.txt", "w")
# Right now, this is reading and writing each as an individual line. Thus, it
# is splitting each sequence into multiple small sequences. You need to figure
# out how to read this in FASTA format to prevent this from occurring
desc = file.readlines()
for line in desc:
allstrings.append(line)
for string in range(1, len(allstrings)):
if ">" not in allstrings[string]:
temp_seq.append(allstrings[string])
else:
seq.insert(position, temp_seq[0])
temp_seq = []
position += 1
# This last insertion into the sequence must be performed after the loop to empty
# out the last remaining string from temp_seq
seq.insert(position, temp_seq[0])
for base in seq:
matrix.append([pos for pos in base])
M = np.array(matrix).reshape(len(seq), len(seq[0]))
for base in range(len(seq[0])):
A_count = 0
C_count = 0
G_count = 0
T_count = 0
for pos in M[:, base]:
if pos == "A":
A_count += 1
elif pos == "C":
C_count += 1
elif pos == "G":
G_count += 1
elif pos == "T":
T_count += 1
A.append(A_count)
C.append(C_count)
G.append(G_count)
T.append(T_count)
profile_matrix = {"A": A, "C": C, "G": G, "T": T}
P.append(A)
P.append(C)
P.append(G)
P.append(T)
profile = np.array(P).reshape(4, len(A))
for pos in range(len(A)):
if max(profile[:, pos]) == profile[0, pos]:
consensus.append("A")
elif max(profile[:, pos]) == profile[1, pos]:
consensus.append("C")
elif max(profile[:, pos]) == profile[2, pos]:
consensus.append("G")
elif max(profile[:, pos]) == profile[3, pos]:
consensus.append("T")
conout.write("".join(consensus) + "\n")
for k, v in profile_matrix.items():
conout.write(k + ": " + " ".join(str(x) for x in v) + "\n")
conout.close()
There are a couple of ways that you can iterate a FASTA file as records. You can use a prebuilt library or write your own.
A widely used library for working with sequence data is biopython. This code snippet will create a list of strings.
from Bio import SeqIO
file = "path/to/your/file.fa"
sequences = []
with open(file, "r") as file_handle:
for record in SeqIO.parse(file_handle, "fasta"):
sequences.append(record.seq)
Alternatively, you can write your own FASTA parser. Something like this should work:
def read_fasta(fh):
# Iterate to get first FASTA header
for line in fh:
if line.startswith(">"):
name = line[1:].strip()
break
# This list will hold the sequence lines
fa_lines = []
# Now iterate to find the get multiline fasta
for line in fh:
if line.startswith(">"):
# When in this block we have reached
# the next FASTA record
# yield the previous record's name and
# sequence as tuple that we can unpack
yield name, "".join(fa_lines)
# Reset the sequence lines and save the
# name of the next record
fa_lines = []
name = line[1:].strip()
# skip to next line
continue
fa_lines.append(line.strip())
yield name, "".join(fa_lines)
You can use this function like so:
file = "path/to/your/file.fa"
sequences = []
with open(file, "r") as file_handle:
for name, seq in read_fasta(file_handle):
sequences.append(seq)
I've built a function to cut the extraneous garbage out of text entries. It uses an array slicer. I now need to reconcile the lines that've been removed by my cleanup function so all the lines_lost + lines_kept = total lines. Source code below:
def header_cleanup(entry_chunk):
# Removes duplicate headers due to page-continuations
entry_chunk = entry_chunk.replace("\r\n\r\n","\r\n")
header = lines[1:5]
lines[:] = [x for x in lines if not any(header == x for header in headers)]
lines = headers + lines
return("\n".join(lines))
How could I count the lines that do not show up in lines after the slice/mutation, i.e:
original_length = len(lines)
lines = lines.remove_garbage
garbage = lines.garbage_only_plz
if len(lines) + len(garbage) == original_length:
print("Good!")
else:
print("Bad! ;(")
Final answer ended up looking like this:
def header_cleanup(entry_chunk):
lines = entry_chunk.replace("\r\n\r\n","\r\n")
line_length = len(lines)
headers = lines[1:5]
saved_lines = []
bad_lines = []
saved_lines[:] = [x for x in lines if not any(header == x for header in headers)]
bad_lines[:] = [x for x in lines if any(header == x for header in headers)]
total_lines = len(saved_lines) + len(bad_lines)
if total_lines == line_length:
print("Yay!")
else:
print("Boo.")
print(f"{rando_trace_info}")
sys.exit()
final_lines = headers + saved_lines
return("\n".join(final_lines))
Okokokokok - I know you're thinking: that's redundant, but it's required. Open to edits after solution for anything more pythonic. Thanks for consideration.
Don't reuse the lines variable, use a different variable, so you can get the garbage out of the original lines.
clean_lines = remove_garbage(lines)
garbage = garbage_only(lines)
if len(clean_lines) + len(garbage) == len(lines):
print("Good!")
else:
print("Bad!")
You might want to have a single function that returns both:
clean_lines, garbage = filter_garbage(lines)
def function(file_count,results):#results contain r.name and r.result
args = []
for r in engine_results:
args.append(r.name)
table_line = ""
for x in range (file_count):
table_line += "".join([
"<tr><td>Ex",
str(x + 1),
"</td> <td>",
"</td> <td>".join([args[x] in args]),
"</td> <td></tr>"]);
return table_line
I want to print out for example
"<tr><td>"Ex1"</td><td></td><td>" A B C D E "</td> <td></tr>"
A B C D E being the contetnt of args, this method didn't work nor a previous one i tested column[x] in args where it just gave me the first letter .
I think I understand what you are asking. This is what I did.:
def function(file_count, results):
# Get a string that is each item in result with a space between
arg_string = ' '.join([str(i) for i in results])
# Set up templates
template_1 = '"<tr><td>"'
template_2 = '"</td><td></td><td>"'
template_3 = '"</td> <td></tr>"'
# Create a variable to hold your lines
lines = []
# For every file in the range
for i in range(0, file_count):
# Make the Ex0, Ex1, Ex2, etc. string
ex = 'Ex{}'.format(i)
# Join all the templates and the ex variable and the arg_string
result = ''.join([template_1, ex, template_2, arg_string, template_3])
# Append the result to lines
lines.append(result)
# Return every item in lines joined by a newline
return '\n'.join(lines)
Essentially, I am creating a count by using a dictionary and everytime it sees a "1" in the text file, it adds one to the array.However, I keep getting an error
Letters = ["A","B,"C","D","E","F"]
d= {}
d["A"] = [0]
d["B"] = [0]
d["C"] = [0]
d["D"] = [0]
d["E"] = [0]
file = open('test1.txt','r')
for line in file:
line_array = line.strip("\n").split(",")
for x in range(5):
if line_array[x] == "1":
for y in Letters:
d[y][0] = d[y][0] + 1
BTW, the text file is formatted like this;
1,0,3,0,2
0,2,1,0,3
ETC
EDIT sorry, misworded
You never actually use your dictionary.
Letters= ["A","B","C","D","E"]
d= {key: 0 for key in Letters}
print(Letters)
file = open('test1.txt','r')
for line in file:
line_array = line.strip("\n").split(",")
for x in range(5):
if line_array[x] == "1":
for i, value in enumerate(Letters):
if i == x:
d[value] = d[value] + 1
#print(candidatescores) # No idea where this comes from