I'm creating a program that stores a lot of values in file.txt. I want to be able to delete some value (like 1 value per minute) from the file. (For the storing method I store 1 value per line if it is helpful.)
There is a semi-solution creating a second file but my problem is that my program has to keep storing data in the file.txt, then has to delete some data from the same file.txt.
I also have to stop the program sometimes, and I don't want to lose my data by putting it in a string.
Does anyone have a solution?
name = 'values' # Change to whatever your file is called
directory = '' # Change to your directory (inside a folder etc.:). Leave blank if none
filetype = '.txt' # Change to your file type
data = []
repeat = 0
with open(directory+name+filetype,'r') as file: # Gets data from file
for line in file:
for word in line.split('\n'):
if repeat % 2 == 0: # Doesn't add every other term, which are empty strings
data.append(word)
repeat += 1
del data[0] # Change to delete whatever item of your list
file = open(directory+name+filetype,"w") # Opens and clears the file
for values in data: # Writes to file
file.write(values + '\n')
file.close() # Closes the file
Related
I need to append a new line in 9000 json files, so, i want to automate that. And i need to put the new line between the "name" and "description", but every time i try to do it, it give me a weird result.
sample file
Tried to search how to do it but i don't get any good result.
Problem solved.
Basicly, i understood that i can store all lines inside an list, and rewrite the file.
Now, i open the file, store the data, add my text from an string to my list, and rewrite the file with the text.
# The files i need to work with have numbers as names
# it make the process easy
fileIndex = 1
# We will use that number in the string Nlink
Number = 1
while fileIndex < 6 :
# Formated strings cant be used directly on list
NLink = f' "Link" : "https://Link_placeHolder/{Number}.jpg",\n'
# Link stores the data, and we can use this on our list
Link = NLink
# Openning your file for reading to store the lines
with open(f"C:\\Cteste\\Original\\{fileIndex}.json", "r+") as f:
# Create an list with all string information of the file
lines = f.readlines()
# Open a new file for writing
with open(f"C:\\Cteste\\New\\{fileIndex}.json", "w")as f2 :
# Insert the link at especifc position in the list
# in my case index 2
lines.insert(2, Link)
# Write everything on the file, them close the files
for i in lines :
f2.write(i)
# add to index
fileIndex += 1
# add to number
Number += 1
I have a folder of 295 text files with each one containing a couple rows of data that I need to condense. I want to open one file, grab two lines from that file, and combine those two lines into another text file I've created, then close the data file and repeat for the next.
I currently have a for loop that does mostly that but the problem I run into is that the for loop copies the same text of the first instance 295 times. How do I get it so that it moves onto the next list item in the filelist? This is my first program in Python so I'm pretty new.
My code:
import os
filelist = os.listdir('/colors')
colorlines = []
for x in filelist:
with open('/colors/'+x, 'rt') as colorfile: #opens single txt file for reading text as colorfile
for colorline in colorfile: #Creates list from each line of the txt file and puts it into colorline
colorlines.append(colorline.rstrip('\n')) #removes the paragraph space in list
tup = (colorlines[1], colorlines[3]) #combines second and fourth line into one line into a tuple
str = ''.join(tup) #joins the tuple into a string with no space between the two
print(str)
newtext = open("colorcode_rework.txt","a") #opens output file for the reworked data
newtext.write(str+'\n') #pastes the string and inserts a new line
newtext.close()
colorfile.close()
You need to reset the color line list for each file. AS you are calling a specific item in the list (1 and 3) you are always calling the same even though more items have been added.
To reset the colorline list for each file:
for x in filelist:
colorlines = []
I have txt file like this:
tw004:Galaxy S5:Samsung:Mobilni telefon:5
tw002:Galaxy S6:Samsung:Mobilni telefon:1
tw001:Huawei P8:Huawei:Mobilni telefon:4
tw003:Huawei P9:Huawei:Mobilni telefon:3
(where tw001 to tw004 is code of some devices and last part of a line is amount 5,1,4,3)
Now I'm trying to add amount to devices with specified code:
def add_devices():
device_file = open ('uredjaji.txt','r').readlines()
code = input("Enter device code: ")
for i in device_file:
device = i.strip("\n").split(":")
if code == device[0]:
device_file = open('uredjaji.txt', 'a')
amount = input("How many devices you are adding: ")
device[4] = int(device[4])
device[4] += int(amount)
device_file.writelines(str(device[4]))
device_file.close()
add_devices()
My problem is that sum of specified device is just add to the end of txt file.
How to fix that?
(For example if I enter tw004 and add 3 sum 8 is just aded to tw003:Huawei P9:Huawei:Mobilni telefon:38)
First of all, don't open multiple file handles to the same file - that's a disaster waiting to happen.
Second, and more to the point, you need to remove the previous number before you add a new one - the way you're doing it is essentially just appending the data to the end of the file. You'll have to do a bit of seeking and truncating in order to achieve what you want, something like:
def add_devices():
# no need to keep our files open while users provide their input
code = input("Enter device code: ")
amount = int(input("How many devices you are adding: "))
# you might want to validate the amount before converting to integer, tho
with open("uredjaji.txt", "r+") as f:
current_position = 0 # keep track of our current position in the file
line = f.readline() # we need to do it manually for .tell() to work
while line:
# no need to parse the whole line to check for the code
if line[:len(code) + 1] == code + ":": # code found
remaining_content = f.read() # read the rest of the file first
f.seek(current_position) # seek back to the current line position
f.truncate() # delete the rest of the file, including the current line
line = line.rstrip() # clear out the whitespace at the end
amount_index = line.rfind(":") + 1 # find the amount index position
current_amount = int(line[amount_index:]) # get our amount
# slice out the old amount, replace with the new:
line = line[:amount_index] + str(current_amount + amount) + "\n"
f.write(line) # write it back to the file
f.write(remaining_content) # write the remaining content
return # done!
current_position = f.tell() # cache our current position
line = f.readline() # get the next line
print("Invalid device code: {}".format(code))
add_devices()
Since you are wanting to update the same file, you will have to separate the code into distinct parts, since you shouldn't read and write to a file at the same time. It could be ordered like:
open the file for reading ('r')and read in the devices (you'll end up with a list of devices, or dictionary, or whatever data structure you want to use), and close the file
process the data - this is where you can increase the number of devices and such
open the file for writing ('w'), write the lines, and close the file
You basically have all the code logic already, just need to untangle it so that you can do the 3 suggested steps separately. :)
Edit: extra note - since you split the lines on ':' when reading the file, you will need to do the reverse and ':'.join(device) when writing it back. ;)
I am beginner in the programming world and a would like some tips on how to solve a challenge.
Right now I have ~10 000 .dat files each with a single line following this structure:
Attribute1=Value&Attribute2=Value&Attribute3=Value...AttibuteN=Value
I have been trying to use python and the CSV library to convert these .dat files into a single .csv file.
So far I was able to write something that would read all files, store the contents of each file in a new line and substitute the "&" to "," but since the Attribute1,Attribute2...AttributeN are exactly the same for every file, I would like to make them into column headers and remove them from every other line.
Any tips on how to go about that?
Thank you!
Since you are a beginner, I prepared some code that works, and is at the same time very easy to understand.
I assume that you have all the files in the folder called 'input'. The code beneath should be in a script file next to the folder.
Keep in mind that this code should be used to understand how a problem like this can be solved. Optimisations and sanity checks have been left out intentionally.
You might want to check additionally what happens when a value is missing in some line, what happens when an attribute is missing, what happens with a corrupted input etc.. :)
Good luck!
import os
# this function splits the attribute=value into two lists
# the first list are all the attributes
# the second list are all the values
def getAttributesAndValues(line):
attributes = []
values = []
# first we split the input over the &
AtributeValues = line.split('&')
for attrVal in AtributeValues:
# we split the attribute=value over the '=' sign
# the left part goes to split[0], the value goes to split[1]
split = attrVal.split('=')
attributes.append(split[0])
values.append(split[1])
# return the attributes list and values list
return attributes,values
# test the function using the line beneath so you understand how it works
# line = "Attribute1=Value&Attribute2=Value&Attribute3=Vale&AttibuteN=Value"
# print getAttributesAndValues(line)
# this function writes a single file to an output file
def writeToCsv(inFile='', wfile="outFile.csv", delim=","):
f_in = open(inFile, 'r') # only reading the file
f_out = open(wfile, 'ab+') # file is opened for reading and appending
# read the whole file line by line
lines = f_in.readlines()
# loop throug evert line in the file and write its values
for line in lines:
# let's check if the file is empty and write the headers then
first_char = f_out.read(1)
header, values = getAttributesAndValues(line)
# we write the header only if the file is empty
if not first_char:
for attribute in header:
f_out.write(attribute+delim)
f_out.write("\n")
# we write the values
for value in values:
f_out.write(value+delim)
f_out.write("\n")
# Read all the files in the path (without dir pointer)
allInputFiles = os.listdir('input/')
allInputFiles = allInputFiles[1:]
# loop through all the files and write values to the csv file
for singleFile in allInputFiles:
writeToCsv('input/'+singleFile)
but since the Attribute1,Attribute2...AttributeN are exactly the same
for every file, I would like to make them into column headers and
remove them from every other line.
input = 'Attribute1=Value1&Attribute2=Value2&Attribute3=Value3'
once for the the first file:
','.join(k for (k,v) in map(lambda s: s.split('='), input.split('&')))
for each file's content:
','.join(v for (k,v) in map(lambda s: s.split('='), input.split('&')))
Maybe you need to trim the strings additionally; don't know how clean your input is.
Put the dat files in a folder called myDats. Put this script next to the myDats folder along with a file called temp.txt. You will also need your output.csv. [That is, you will have output.csv, myDats, and mergeDats.py in the same folder]
mergeDats.py
import csv
import os
g = open("temp.txt","w")
for file in os.listdir('myDats'):
f = open("myDats/"+file,"r")
tempData = f.readlines()[0]
tempData = tempData.replace("&","\n")
g.write(tempData)
f.close()
g.close()
h = open("text.txt","r")
arr = h.read().split("\n")
dict = {}
for x in arr:
temp2 = x.split("=")
dict[temp2[0]] = temp2[1]
with open('output.csv','w' """use 'wb' in python 2.x""" ) as output:
w = csv.DictWriter(output,my_dict.keys())
w.writeheader()
w.writerow(my_dict)
I am trying to write a script that will update a folder of text files based on a lookup table. The lookup table is filename, oldpath, new path. The script looks in each text file for the filename, if there it updates the oldpath in the same line with the new path. The code is:
# Import
from array import *
import glob
# Specify the lookup table, to keep it simple drop it in with the workspaces
Lookup = "./Lookup.csv"
# Specify the
Workspaces = glob.glob('./*.wor')
# Open the Lookup table
for line in open(Lookup).readlines():
# Create the list to store the lookup parameters of the lookup line
LookupList = []
# Split the lookup csv at the comma
for i in line.split(","):
#print i
LookupList.append(i)
# Use the list parameters to populate variables (could use list parameters but
# easier for now to assign to variable)
FileName = LookupList[0]
OldPath = LookupList[1]
NewPath = LookupList[2]
# We now have variables to use in the replace statement
# Use the Workspaces Glob to loop through the workspaces
for wor in Workspaces:
# Try to open the the first workspace (text file)
f = open(wor, 'r+')
# Loop through the open file
for line in f.readlines():
# For each line check whether the current list value (FileName) is in the line
if '"' + OldPath + '"' in line:
print line
# Update the line, replacing the old path with the new path.
line.replace(OldPath, NewPath);
# Close the workspace file
f.close()
It all seems to work as it should, the print statement 5 lines from the end has found the correct lines that contain the search strings from the lookup, but the file is not updated.
I have read as much as I can find on file opening modes and updating files but there is no obvious solution. I guess the issue is reading/writing to the same file. The route I have chosen is open the lookup and embed the files to be changed loop. The alternative is open the file and then loop through the lookup.
Happy to write the updated file out to another name/folder, the issue there is that if you loop through the files to update, update the row based on the lookup, when you get to the next row of the lookup it will overwrite the previous lookup change.
Any ideas gratefully received. Aplogies if the description seems convoluted, happy to clarify any areas where the aim is not obvious.
Thanks
Paul
f.readines()
returns a list of strings, and you are iterating over these strings. So, when you make an edit on that string using
line.replace(...)
you are not changing the text file. Rather, you are changing the string that you have read in.
Your approach should be to write each line to a temp list, then write that temp list to a file, like:
f = open(wor, 'r+')
new_lines = []
for line in f.readlines():
if '"' + OldPath + '"' in line :
line.replace(OldPath, NewPath);
new_lines.append(line)
f.close()
file("path/to/your/new/or/temp/file","w")
file.write("\n".join(new_lines))
file.close()
There are issues with the way that you are appending the LookupList, as you repeatedly assign it to [], which makes it empty again, so only the last iteration is kept. But that aside, this code for writing should do what you are intending it to:
# We now have variables to use in the replace statement
# Use the Workspaces Glob to loop through the workspaces
for wor in Workspaces:
# Handles opening and closing of input and output files
with open(wor, 'r'),open("new_" + wor,'w') as infile,outfile:
# Loop through the input file
for line in infile.readlines():
# For each line check whether the current list value (FileName) is in the line
if '"' + OldPath + '"' in line:
print line
# Update the line, replacing the old path with the new path.
line.replace(OldPath, NewPath);
outfile.write(line)
This is the code I have used which has worked for testing. It contains a large slice of Khol's suggestion so I have credited him with the answer. I am quite new to Python so the code may be a 5 yard beauty but it produces the results we are looking for.
# Import
from array import *
import glob
# Specify the lookup table, to keep it simple drop it in with the workspaces
Lookup = "./Lookup.csv"
# Specify the list of workspaces (text files to update)
Workspaces = glob.glob('./*.wor')
# Open the Lookup table
for line in open(Lookup).readlines():
# Create the list to store the lookup parameters of the lookup line
LookupList = []
# Split the lookup csv at the comma
for i in line.split(","):
# Add items to the lookup list
LookupList.append(i)
# Assign the list value to a variable (could skip assigning to variable),
# strip CR from value or it will add it to the output string
FileName = LookupList[0].strip()
NewPath = LookupList[1].strip()
# Loop through the workspaces
for wor in Workspaces:
# Open the workspace
f = open(wor, 'r+')
# Create list
WorList = []
# Read through the workspace and use it to populate the list
for line in f.readlines():
WorList.append(line)
f.close()
# Loop through the workspace list looking for the FileName string (from the lookup list)
for position, item in enumerate(WorList):
if " " + FileName + " " in item:
# If the filename is found then create a string with the new file name in the old row structure
Newval = "Open Table " + '"'+ NewPath + '" ' + "As " + FileName + " Interactive\n"
# Update the value in the list based on the list position
WorList[position] = Newval;
# Open the output file (this is the same as the original input file)
file=open(wor,"w")
# Work through the list and write it to the file
for s in WorList:
file.write(s)
file.close()