I need to append a new line in 9000 json files, so, i want to automate that. And i need to put the new line between the "name" and "description", but every time i try to do it, it give me a weird result.
sample file
Tried to search how to do it but i don't get any good result.
Problem solved.
Basicly, i understood that i can store all lines inside an list, and rewrite the file.
Now, i open the file, store the data, add my text from an string to my list, and rewrite the file with the text.
# The files i need to work with have numbers as names
# it make the process easy
fileIndex = 1
# We will use that number in the string Nlink
Number = 1
while fileIndex < 6 :
# Formated strings cant be used directly on list
NLink = f' "Link" : "https://Link_placeHolder/{Number}.jpg",\n'
# Link stores the data, and we can use this on our list
Link = NLink
# Openning your file for reading to store the lines
with open(f"C:\\Cteste\\Original\\{fileIndex}.json", "r+") as f:
# Create an list with all string information of the file
lines = f.readlines()
# Open a new file for writing
with open(f"C:\\Cteste\\New\\{fileIndex}.json", "w")as f2 :
# Insert the link at especifc position in the list
# in my case index 2
lines.insert(2, Link)
# Write everything on the file, them close the files
for i in lines :
f2.write(i)
# add to index
fileIndex += 1
# add to number
Number += 1
Related
houses = []
#print(houses)
#Creates an empty list for the houses
f = open("houses.txt", "r+")
#Creates a variable called "f" which opens the houses.txt file and imports it r+ = Read and + means you can append on the end
lines = f.readlines()
#The readlines() method returns a list containing each line in the file as a list item. Use the hint parameter to limit the number of lines returned.
#The for loop grabs each line of text from the file, and splits the houses and scores and makes them separate. It runs for as many lines are in the text file
for line in lines:
#The append adds the stripped and split code separately in a list
#.strip .split separates the houses and scores into separate strings
houses.append(line.strip().split())
for i in range(len(houses)):
#Loops for how many houses are in the list
houses[i][1] = int(houses[i][1])
#Turns the second part of the list into an integer
print(houses)
This part of the code imports houses (teams) from the text file which is laid out like this:
StTeresa 0
StKolbe 0
StMary 0
StAnn 0
I created a function to save the points. I would basically like it to take the houses and scores from the list in the program. To do so, it will delete all the contents in the text file, and then I would like it to rewrite it in the same format as the original text file to keep the updated house scores.
def save():
f.truncate(0)
f.write(str(houses))
I tried this but the output is:
This
Can anyone help me to rewrite the text file to include the updated scores and be in the same format as the text file orignally was?
This should accomplish what you want:
def save():
with open("houses.txt", "w") as file:
for index, line_content in enumerate(houses):
houses[index][1] = str(houses[index][1])
file.write("\n".join(map(lambda sub_list: " ".join(sub_list), houses)))
However, as you can see, this is quite hacky.
So I would have two main recommendations for your code:
Use open() within a with statement. Currently your code does not reliably close the file it opens, which can cause your code to do unexpected things (for example when an exception is thrown in the middle of your processing). Therefore, it is recommended practice to use with (further information on this can be found here)
Instead of using lists within a list, you could use tuples:
for line in lines:
# The append adds the stripped and split code separately in a list
# .strip .split separates the houses and scores into separate strings
house_name, house_score = tuple(line.strip().split())
houses.append((house_name, int(house_score)))
This gives you something like this: [("StTeresa", 0), ("StKolbe", 0), ("StMary", 0), ("StAnn", 0)]. Tuples in lists have the advantage that you can unpack them in loops, which makes it easier to handle them compared to a list within a list:
for index, (house_name, house_score) in enumerate(houses):
# do something with the score
houses[index] = (house_name, updated_score)
Another option is to follow what Barmar suggested and use the csv module, which is part of the standard library.
As a bonus: enumerate() provides you the indices of an iterable, so you can easily loop over them. So instead of
for i in range(len(houses)):
#Loops for how many houses are in the list
you can write
for i, house in enumerate(houses):
# Loop over all the houses
which makes the code a little easier to read and write.
So with all my suggestions combined you could have a save() function like this:
def save():
with open("houses.txt", "w") as file:
for house_name, house_score in houses:
file.write(f"{house_name} {house_score}\n")
Hope this helps :)
I want to append data from text file to a list from a given starting point. The stating point string can be anywhere in the file. i want to append the data from that starting point. I tried by using startswith method:
list1=[]
TextFile = "txtfile.txt"
# open the file for data processing
with open(TextFile,'rt',encoding="utf8") as IpFile:
for i,j in enumerate(IpFile):
if(j.startswith("Starting point")):
list1.append(str(j).strip())
i+=1
but it only append the starting point. i want to append the all data from starting point. How to do that ?
Use a boolean variable
list1=[]
TextFile = "txtfile.txt"
doAppend = False
# open the file for data processing
with open(TextFile,'rt',encoding="utf8") as IpFile:
for i,j in enumerate(IpFile):
if(j.startswith("Starting point")):
doAppend = True
if doAppend:
list1.append(str(j).strip())
i+=1
You could do it without a bool as well by breaking the for loop and then reading the rest of the file.
list1 = []
text_file = "txtfile.txt"
# rt is the default so theres no need to specify it
with open(text_file, encoding="utf8") as ip_file:
for line in ip_file:
if line.startswith("Starting point"):
break
# read the rest of the file
remainder = ip_file.read()
# extend the list with the rest of the file split on newlines
list1.extend(remainder.split('\n'))
Im learning python, I´ve been trying to split this txt file into multiple files grouped by a sliced string at the beginning of each line.
currently i have two issues:
1 - The string can have 5 or 6 chars is marked by a space at the end.(as in WSON33 and JHSF3 etc...)
Here is an exemple of the file i would like to split ( first line is a header):
H24/06/202000003TORDISTD
BWSON33 0803805000000000016400000003250C000002980002415324C1 0000000000000000
BJHSF3 0804608800000000003500000000715V000020280000031810C1 0000000000000000
2- I´ve come with a lot of code, but i´m not able to put everything together so this can work:
This code here i adappeted from another post and it kind of works breaking into multiple files, but it requires a sorting of the lines before i start writing files, i aslo need to copy the header in each file and not isolete it one file.
with open('tordist.txt', 'r') as fin:
# group each line in input file by first part of split
for i, (k, g) in enumerate(itertools.groupby(fin, lambda l: l.split()[0]),1):
# create file to write to suffixed with group number - start = 1
with open('{0} tordist.txt'.format(i), 'w') as fout:
# for each line in group write it to file
for line in g:
fout.write(line.strip() + '\n')
So from what I can gather, you have a text file with many lines, where every line begins with a short string of 5 or six characters. It sounds like you want all the lines that begin with the same string to go into the same file, so that after the code is run you have as many new files as there are unique starting strings. Is that accurate?
Like you, I'm fairly new to python, and so I'm sure there are more compact ways to do this. The code below loops through the file a number of times, and makes new files in the same folder as the file where your text and python files are.
# code which separates lines in a file by an identifier,
#and makes new files for each identifier group
filename = input('type filename')
if len(filename) < 1:
filename = "mk_newfiles.txt"
filehandle = open(filename)
#This chunck loops through the file, looking at the beginning of each line,
#and adding it to a list of identifiers if it is not on the list already.
Unique = list()
for line in filehandle:
#like Lalit said, split is a simple way to seperate a longer string
line = line.split()
if line[0] not in Unique:
Unique.append(line[0])
#For each item in the list of identifiers, this code goes through
#the file, and if a line starts with that identifier then it is
#added to a new file.
for item in Unique:
#this 'if' skips the header, which has a '/' in it
if '/' not in item:
# the .seek(0) 'rewinds' the iteration variable, which is apperently needed
#needed if looping through files multiple times
filehandle.seek(0)
#makes new file
newfile = open(str(item) + ".txt","w+")
#inserts header, and goes to next line
newfile.write(Unique[0])
newfile.write('\n')
#goes through old file, and adds relevant lines to new file
for line in filehandle:
split_line = line.split()
if item == split_line[0]:
newfile.write(line)
print(Unique)
I'm creating a program that stores a lot of values in file.txt. I want to be able to delete some value (like 1 value per minute) from the file. (For the storing method I store 1 value per line if it is helpful.)
There is a semi-solution creating a second file but my problem is that my program has to keep storing data in the file.txt, then has to delete some data from the same file.txt.
I also have to stop the program sometimes, and I don't want to lose my data by putting it in a string.
Does anyone have a solution?
name = 'values' # Change to whatever your file is called
directory = '' # Change to your directory (inside a folder etc.:). Leave blank if none
filetype = '.txt' # Change to your file type
data = []
repeat = 0
with open(directory+name+filetype,'r') as file: # Gets data from file
for line in file:
for word in line.split('\n'):
if repeat % 2 == 0: # Doesn't add every other term, which are empty strings
data.append(word)
repeat += 1
del data[0] # Change to delete whatever item of your list
file = open(directory+name+filetype,"w") # Opens and clears the file
for values in data: # Writes to file
file.write(values + '\n')
file.close() # Closes the file
I am beginner in the programming world and a would like some tips on how to solve a challenge.
Right now I have ~10 000 .dat files each with a single line following this structure:
Attribute1=Value&Attribute2=Value&Attribute3=Value...AttibuteN=Value
I have been trying to use python and the CSV library to convert these .dat files into a single .csv file.
So far I was able to write something that would read all files, store the contents of each file in a new line and substitute the "&" to "," but since the Attribute1,Attribute2...AttributeN are exactly the same for every file, I would like to make them into column headers and remove them from every other line.
Any tips on how to go about that?
Thank you!
Since you are a beginner, I prepared some code that works, and is at the same time very easy to understand.
I assume that you have all the files in the folder called 'input'. The code beneath should be in a script file next to the folder.
Keep in mind that this code should be used to understand how a problem like this can be solved. Optimisations and sanity checks have been left out intentionally.
You might want to check additionally what happens when a value is missing in some line, what happens when an attribute is missing, what happens with a corrupted input etc.. :)
Good luck!
import os
# this function splits the attribute=value into two lists
# the first list are all the attributes
# the second list are all the values
def getAttributesAndValues(line):
attributes = []
values = []
# first we split the input over the &
AtributeValues = line.split('&')
for attrVal in AtributeValues:
# we split the attribute=value over the '=' sign
# the left part goes to split[0], the value goes to split[1]
split = attrVal.split('=')
attributes.append(split[0])
values.append(split[1])
# return the attributes list and values list
return attributes,values
# test the function using the line beneath so you understand how it works
# line = "Attribute1=Value&Attribute2=Value&Attribute3=Vale&AttibuteN=Value"
# print getAttributesAndValues(line)
# this function writes a single file to an output file
def writeToCsv(inFile='', wfile="outFile.csv", delim=","):
f_in = open(inFile, 'r') # only reading the file
f_out = open(wfile, 'ab+') # file is opened for reading and appending
# read the whole file line by line
lines = f_in.readlines()
# loop throug evert line in the file and write its values
for line in lines:
# let's check if the file is empty and write the headers then
first_char = f_out.read(1)
header, values = getAttributesAndValues(line)
# we write the header only if the file is empty
if not first_char:
for attribute in header:
f_out.write(attribute+delim)
f_out.write("\n")
# we write the values
for value in values:
f_out.write(value+delim)
f_out.write("\n")
# Read all the files in the path (without dir pointer)
allInputFiles = os.listdir('input/')
allInputFiles = allInputFiles[1:]
# loop through all the files and write values to the csv file
for singleFile in allInputFiles:
writeToCsv('input/'+singleFile)
but since the Attribute1,Attribute2...AttributeN are exactly the same
for every file, I would like to make them into column headers and
remove them from every other line.
input = 'Attribute1=Value1&Attribute2=Value2&Attribute3=Value3'
once for the the first file:
','.join(k for (k,v) in map(lambda s: s.split('='), input.split('&')))
for each file's content:
','.join(v for (k,v) in map(lambda s: s.split('='), input.split('&')))
Maybe you need to trim the strings additionally; don't know how clean your input is.
Put the dat files in a folder called myDats. Put this script next to the myDats folder along with a file called temp.txt. You will also need your output.csv. [That is, you will have output.csv, myDats, and mergeDats.py in the same folder]
mergeDats.py
import csv
import os
g = open("temp.txt","w")
for file in os.listdir('myDats'):
f = open("myDats/"+file,"r")
tempData = f.readlines()[0]
tempData = tempData.replace("&","\n")
g.write(tempData)
f.close()
g.close()
h = open("text.txt","r")
arr = h.read().split("\n")
dict = {}
for x in arr:
temp2 = x.split("=")
dict[temp2[0]] = temp2[1]
with open('output.csv','w' """use 'wb' in python 2.x""" ) as output:
w = csv.DictWriter(output,my_dict.keys())
w.writeheader()
w.writerow(my_dict)