I have a python code which opens a text file read and do some processes. The input text file has to be updated each day with a new one. Currently I use following code to open the file.
f = open('sample20130616.txt','r')
But in this method some one has to go inside the code and modify the file name(Since the file names are not unique).
What is the proper method to handle this?
Pass the filename as parameter when you call your application:
python myapp.py file.txt
Then in your code:
import sys
if __name__ == '__main__':
filename = sys.argv[1]
f = open(filename)
You can easily generate the current date and hence your filename with the datetime module:
import datetime
today = datetime.date.today()
filename = 'sample{0}{1:02d}{2:02d}.txt'.format(today.year, today.month, today.day)
f = open(filename,'r')
Use the built-in time function!
from time import gmtime, strftime
fname='sample'+strftime("%Y%m%d", gmtime())+'.txt'
f = open(fname,'r')
Related
So, I'll explain briefly my idea, then, what I've tried and errors that I've got so far.
I want to make a Python script that will:
Search for files in a directory, example: /home/mystuff/logs
If he found it, he will execute a command like print('Errors found'), and then stop.
If not, he will keep it executing on and on.
But other logs will be there, so, my intention is to make Python read logs in /home/mystuff/logs filtering by the current date/time only.. since I want it to be executed every 2 minutes.
Here is my code:
import time
import os
from time import sleep
infile = r"/home/mystuff/logs`date +%Y-%m-%d`*"
keep_phrases = ["Error",
"Lost Connection"]
while True:
with open(infile) as f:
f = f.readlines()
if phrase in f:
cmd = ['#print something']
erro = 1
else:
sleep(1)
I've searched for few regex cases for current date, but nothing related to files that will keep changing names according by the date/time.. do you have any ideas?
You can't use shell features like command substitutions in file names. To the OS, and to Python, a file name is just a string. But you can easily create a string which contains the current date and time.
from datetime import datetime
infile = r"/home/mystuff/logs%s" % datetime.now().strftime('%Y-%m-%d')
(The raw string doesn't do anything useful, because the string doesn't contain any backslashes. But it's harmless, so I left it in.)
You also can't open a wildcard; but you can expand it to a list of actual file names with glob.glob(), and loop over the result.
from glob import glob
for file in glob(infile + '*'):
with open(file, 'r') as f:
# ...
If you are using a while True: loop you need to calculate today's date inside the loop; otherwise you will be perpetually checking for files from the time when the script was started.
In summary, your changed script could look something like this. I have changed the infile variable name here because it isn't actually a file or a file name, and fixed a few other errors in your code.
# Unused imports
# import time
# import os
from datetime import datetime
from glob import glob
from time import sleep
keep_phrases = ["Error",
"Lost Connection"]
while True:
pattern = "/home/mystuff/logs%s*" % datetime.now().strftime('%Y-%m-%d')
for file in glob(pattern):
with open(file) as f:
for line in f:
if any(phrase in line for phrase in keep_phrases):
cmd = ['#print something']
erro = 1
break
sleep(120)
with open(r"path/sample.txt")as file:
some operations
print('exiting')
when i open the file is it possible to open the filename as below
sample2018-10-25-18-25-36669_devicename_uuid
How to create filenames in python with UTCdatetime & Hostname and guid, for example i need the below mentioned format of file
I am already opening a file to perform some string operations and store it in the same file. It could be very great if I can create a filename while starting the open operation or can I create a file and do all the operations and rename the file in the above mentioned format. How to proceed with this further. I am very new to python
Sure, you can generate a filename in python dynamically.
There is a simple code example that would help you generate file name as you describe.
import os
import socket
from datetime import datetime
from uuid import uuid4
dt = datetime.utcnow().strftime("%Y-%m-%d-%H-%M-%s")
path = 'path'
hostname = socket.gethostname()
filename = f"samle{dt}-{hostname}-{uuid4()}"
with open(os.path.join(path, filename), 'w') as f:
f.write('some_content')
If you want to get a unique hardware ID with Python please check this
link
I am scraping info to a text file and am trying to write the date at the top. I have the method to grab the date but have no clue how I can use the write function to place at top. Below is a stripped down version of what I am working on.
import re
import urllib2
import json
from datetime import datetime
import time
now = datetime.now()
InputDate = now.strftime("%Y-%m-%d")
Today = now.strftime("%B %d")
header = ("Today").split()
newfile = open("File.txt", "w")
### Irrelevant Info Here ###
string = title"\n"+info+"\n"
#newfile.write(header)
newfile.write(string)
print title+" written to file"
newfile.close()
You can't insert something at the beginning of a file. You need to write a new file, starting with the line you want to insert, then finish with the contents of the old file. Unlike appending to the end, writing to the start of the file is really, really inefficient
The key to this problem is to use a NamedTemporaryFile. After you finish constructing it, you then rename it on top of the old file.
Code:
def insert_timestamp_in_file(filename):
with open(filename) as src, tempfile.NamedTemporaryFile(
'w', dir=os.path.dirname(filename), delete=False) as dst:
# Save the new first line
dst.write(dt.datetime.now().strftime("%Y-%m-%d\n"))
# Copy the rest of the file
shutil.copyfileobj(src, dst)
# remove old version
os.unlink(filename)
# rename new version
os.rename(dst.name, filename)
Test Code:
import datetime as dt
import tempfile
import shutil
insert_timestamp_in_file("file1")
file1
I am scraping info to a text file and am trying to write the date at
the top. I have the method to grab the date but have no clue how I can
use the write function to place at top. Been trying for 2 days and all.
Results:
2018-02-15
I am scraping info to a text file and am trying to write the date at
the top. I have the method to grab the date but have no clue how I can
use the write function to place at top. Been trying for 2 days and all.
To write the date to the 'top' of the file you would want to put:
newfile.write(InputDate)
newfile.write(Today)
after where you open the file and before anything else.
Just to give you idea
Try this:-
import re
import urllib2
import json
from datetime import datetime
import time
now = datetime.now()
InputDate = now.strftime("%Y-%m-%d")
Today = now.strftime("%B %d")
#start writing from here
newfile = open("File.txt", "a")
newfile.write(InputDate+"\n")
newfile.write("hello Buddy")
newfile.close()
Simple One will be, if you will not call it as a str then it will throw an error TypeError: write() argument must be str, not list
I have rfreshed teh code to be more precise and effective use..
import re
from datetime import datetime
import time
now = datetime.now()
InputDate = now.strftime("%B"+" "+"%Y-%m-%d")
newfile = open("File.txt", "a")
string = "Hi trying to add a datetime at the top of the file"+"\n"
newfile.write(str(InputDate+"\n"))
newfile.write(string)
newfile.close()
Result will be:
February 152018-02-15
Hi trying to add a datetime at the top of the file
I am getting a TypeError: object of type file' has no len()
I have traced down the issue to the path established upon execution.
What am I missing to correct this error found within the "savePath" deceleration or usage within the "temp = os.path.join(savePath, files)"?
def printTime(time):
savePath = "C:\Users\Nicholas\Documents"
files = open("LogInLog.txt", "a")
temp = os.path.join(savePath, files)
files.write("A LogIn occured.")
files.write(time)
print files.read
files.close
main()
The whole program is below for reference:
from time import strftime
import os.path
def main():
getTime()
def getTime():
time = strftime("%Y-%m-%d %I:%M:%S")
printTime(time)
def printTime(time):
savePath = "C:\Users\Nicholas\Documents"
files = open("LogInLog.txt", "a")
temp = os.path.join(savePath, files)
files.write("A LogIn occured.")
files.write(time)
print files.read
files.close
main()
Here's a working version:
from time import strftime
import os.path
def main():
getTime()
def getTime():
time = strftime("%Y-%m-%d %I:%M:%S")
printTime(time)
def printTime(time):
savePath = "C:\Users\Nicholas\Documents"
logFile = "LogInLog.txt"
files = open(os.path.join(savePath, logFile), "a+")
openPosition = files.tell()
files.write("A LogIn occured.")
files.write(time)
files.seek(openPosition)
print(files.read())
files.close()
if __name__ == '__main__':
main()
There were a few problems with the code snippet posted in the question:
Two import statements were concatenated together. Each should be on a separate line.
The os.path.join function doesn't work on an open filehandle.
The read() and close() methods were missing parens.
If the intent is to read what is written in append mode, it's necessary to get the current file position via tell() and seek() to that position after writing to the file.
While it's legal to call main() without any conditional check, it's usually best to make sure the module is being called as a script as opposed to being imported.
Using python 2.7..
I am using below to send all print output to a file called output.log. How can i have this send to a different file each time it runs...In bash we could declare a variable called date or something and have that part of the file name...so how can i achieve the same with python ??
So my question is..
every time i run the below script, my file should have a naming convention of output_date/time.log
Also how can i delete file that are older than X days that have a file naming convention of output_*.log
import sys
f = open('output.log', 'w')
sys.stdout = f
print "test"
f.close()
with some personal preference of formatting this is generally what I do.
import time
moment=time.strftime("%Y-%b-%d__%H_%M_%S",time.localtime())
f = open('output'+moment+'.log', 'w')
as far as automated deleting, do you want it deleted on run of the test?
os.remove(fileStringName)
works, you just have to do the arithmetic and string conversion. I would use os.listdir(pathToDirWithOutputLogs) iterate through the file names and do the math on them and call os.remove on the old ones.
To get date/time:
from time import gmtime, strftime
outputFileName = "output_#.log"
outputFileName = outputFileName.replace("#", strftime("%Y-%m-%d_%H:%M:%S", gmtime()))
For numerical incrementing:
outputFileName = "output #.log"
outputVersion = 1
while os.path.isfile(outputFileName.replace("#", str(outputVersion))):
outputVersion += 1
outputFileName = outputFileName.replace("#", str(outputVersion))
To delete files older than a certain date, you can iterate through all the files in the directory with ``, and delete them with os.remove(). You can compare the file names after parsing them.
lastTime = "2015-08-03_19:04:41"
for fn in filter(os.path.isfile, os.listdir()):
strtime = fn[fn.find("_"):fn.rfind(".")]
if strtime < lastTime:
os.remove(fn)