Using python 2.7..
I am using below to send all print output to a file called output.log. How can i have this send to a different file each time it runs...In bash we could declare a variable called date or something and have that part of the file name...so how can i achieve the same with python ??
So my question is..
every time i run the below script, my file should have a naming convention of output_date/time.log
Also how can i delete file that are older than X days that have a file naming convention of output_*.log
import sys
f = open('output.log', 'w')
sys.stdout = f
print "test"
f.close()
with some personal preference of formatting this is generally what I do.
import time
moment=time.strftime("%Y-%b-%d__%H_%M_%S",time.localtime())
f = open('output'+moment+'.log', 'w')
as far as automated deleting, do you want it deleted on run of the test?
os.remove(fileStringName)
works, you just have to do the arithmetic and string conversion. I would use os.listdir(pathToDirWithOutputLogs) iterate through the file names and do the math on them and call os.remove on the old ones.
To get date/time:
from time import gmtime, strftime
outputFileName = "output_#.log"
outputFileName = outputFileName.replace("#", strftime("%Y-%m-%d_%H:%M:%S", gmtime()))
For numerical incrementing:
outputFileName = "output #.log"
outputVersion = 1
while os.path.isfile(outputFileName.replace("#", str(outputVersion))):
outputVersion += 1
outputFileName = outputFileName.replace("#", str(outputVersion))
To delete files older than a certain date, you can iterate through all the files in the directory with ``, and delete them with os.remove(). You can compare the file names after parsing them.
lastTime = "2015-08-03_19:04:41"
for fn in filter(os.path.isfile, os.listdir()):
strtime = fn[fn.find("_"):fn.rfind(".")]
if strtime < lastTime:
os.remove(fn)
Related
I'm trying to have my Python code write everything it does to a log, with a timestamp. But it doesn't seem to work.
this is my current code:
filePath= Path('.')
time=datetime.datetime.now()
bot_log = ["","Set up the file path thingy"]
with open ('bot.log', 'a') as f:
f.write('\n'.join(bot_log)%
datetime.datetime.now().strftime("%d-%b-%Y (%H:%M:%S.%f)"))
print(bot_log[0])
but when I run it it says:
Traceback (most recent call last):
File "c:\Users\Name\Yuna-Discord-Bot\Yuna Discord Bot.py", line 15, in <module>
f.write('\n'.join(bot_log)%
TypeError: not all arguments converted during string formatting
I have tried multiple things to fix it, and this is the latest one. is there something I'm doing wrong or missing? I also want the time to be in front of the log message, but I don't think it would do that (if it worked).
You need to put "%s" somewhere in the input string before string formatting. Here's more detailed explanation.
Try this:
filePath= Path('.')
time=datetime.datetime.now()
bot_log = "%s Set up the file path thingy\n"
with open ('bot.log', 'a') as f:
f.write(bot_log % datetime.datetime.now().strftime("%d-%b-%Y (%H:%M:%S.%f)"))
print(bot_log)
It looks like you want to write three strings to your file as separate lines. I've rearranged your code to create a single list to pass to writelines, which expects an iterable:
filePath= Path('.')
time=datetime.datetime.now()
bot_log = ["","Set up the file path thingy"]
with open ('bot.log', 'a') as f:
bot_log.append(datetime.datetime.now().strftime("%d-%b-%Y (%H:%M:%S.%f)"))
f.writelines('\n'.join(bot_log))
print(bot_log[0])
EDIT: From the comments the desire is to prepend the timestamp to the message and keep it on the same line. I've used f-strings as I prefer the clarity they provide:
import datetime
from pathlib import Path
filePath = Path('.')
with open('bot.log', 'a') as f:
time = datetime.datetime.now()
msg = "Set up the file path thingy"
f.write(f"""{time.strftime("%d-%b-%Y (%H:%M:%S.%f)")} {msg}\n""")
You could also look at the logging module which does a lot of this for you.
So, I'll explain briefly my idea, then, what I've tried and errors that I've got so far.
I want to make a Python script that will:
Search for files in a directory, example: /home/mystuff/logs
If he found it, he will execute a command like print('Errors found'), and then stop.
If not, he will keep it executing on and on.
But other logs will be there, so, my intention is to make Python read logs in /home/mystuff/logs filtering by the current date/time only.. since I want it to be executed every 2 minutes.
Here is my code:
import time
import os
from time import sleep
infile = r"/home/mystuff/logs`date +%Y-%m-%d`*"
keep_phrases = ["Error",
"Lost Connection"]
while True:
with open(infile) as f:
f = f.readlines()
if phrase in f:
cmd = ['#print something']
erro = 1
else:
sleep(1)
I've searched for few regex cases for current date, but nothing related to files that will keep changing names according by the date/time.. do you have any ideas?
You can't use shell features like command substitutions in file names. To the OS, and to Python, a file name is just a string. But you can easily create a string which contains the current date and time.
from datetime import datetime
infile = r"/home/mystuff/logs%s" % datetime.now().strftime('%Y-%m-%d')
(The raw string doesn't do anything useful, because the string doesn't contain any backslashes. But it's harmless, so I left it in.)
You also can't open a wildcard; but you can expand it to a list of actual file names with glob.glob(), and loop over the result.
from glob import glob
for file in glob(infile + '*'):
with open(file, 'r') as f:
# ...
If you are using a while True: loop you need to calculate today's date inside the loop; otherwise you will be perpetually checking for files from the time when the script was started.
In summary, your changed script could look something like this. I have changed the infile variable name here because it isn't actually a file or a file name, and fixed a few other errors in your code.
# Unused imports
# import time
# import os
from datetime import datetime
from glob import glob
from time import sleep
keep_phrases = ["Error",
"Lost Connection"]
while True:
pattern = "/home/mystuff/logs%s*" % datetime.now().strftime('%Y-%m-%d')
for file in glob(pattern):
with open(file) as f:
for line in f:
if any(phrase in line for phrase in keep_phrases):
cmd = ['#print something']
erro = 1
break
sleep(120)
I am scraping info to a text file and am trying to write the date at the top. I have the method to grab the date but have no clue how I can use the write function to place at top. Below is a stripped down version of what I am working on.
import re
import urllib2
import json
from datetime import datetime
import time
now = datetime.now()
InputDate = now.strftime("%Y-%m-%d")
Today = now.strftime("%B %d")
header = ("Today").split()
newfile = open("File.txt", "w")
### Irrelevant Info Here ###
string = title"\n"+info+"\n"
#newfile.write(header)
newfile.write(string)
print title+" written to file"
newfile.close()
You can't insert something at the beginning of a file. You need to write a new file, starting with the line you want to insert, then finish with the contents of the old file. Unlike appending to the end, writing to the start of the file is really, really inefficient
The key to this problem is to use a NamedTemporaryFile. After you finish constructing it, you then rename it on top of the old file.
Code:
def insert_timestamp_in_file(filename):
with open(filename) as src, tempfile.NamedTemporaryFile(
'w', dir=os.path.dirname(filename), delete=False) as dst:
# Save the new first line
dst.write(dt.datetime.now().strftime("%Y-%m-%d\n"))
# Copy the rest of the file
shutil.copyfileobj(src, dst)
# remove old version
os.unlink(filename)
# rename new version
os.rename(dst.name, filename)
Test Code:
import datetime as dt
import tempfile
import shutil
insert_timestamp_in_file("file1")
file1
I am scraping info to a text file and am trying to write the date at
the top. I have the method to grab the date but have no clue how I can
use the write function to place at top. Been trying for 2 days and all.
Results:
2018-02-15
I am scraping info to a text file and am trying to write the date at
the top. I have the method to grab the date but have no clue how I can
use the write function to place at top. Been trying for 2 days and all.
To write the date to the 'top' of the file you would want to put:
newfile.write(InputDate)
newfile.write(Today)
after where you open the file and before anything else.
Just to give you idea
Try this:-
import re
import urllib2
import json
from datetime import datetime
import time
now = datetime.now()
InputDate = now.strftime("%Y-%m-%d")
Today = now.strftime("%B %d")
#start writing from here
newfile = open("File.txt", "a")
newfile.write(InputDate+"\n")
newfile.write("hello Buddy")
newfile.close()
Simple One will be, if you will not call it as a str then it will throw an error TypeError: write() argument must be str, not list
I have rfreshed teh code to be more precise and effective use..
import re
from datetime import datetime
import time
now = datetime.now()
InputDate = now.strftime("%B"+" "+"%Y-%m-%d")
newfile = open("File.txt", "a")
string = "Hi trying to add a datetime at the top of the file"+"\n"
newfile.write(str(InputDate+"\n"))
newfile.write(string)
newfile.close()
Result will be:
February 152018-02-15
Hi trying to add a datetime at the top of the file
I've tried a few different methods of doing this but each one has errors or will only function in specific ways.
randomData = ("Some Random stuff")
with open("outputFile.txt", "a") as file:
file.write(randomData)
exit()
What i'm trying to to is write to the "outputFile.txt" file and then next output to a different file such as "outputFileTwo.txt".
If you need different filename at every start then you have to save in other file (ie. config.txt) information about current filename. It can be number which you will use in filename (file1.txt, file2.txt, etc.).
And at start read number from config.txt, increase this number, use in filename, and write number again in config.txt.
Or you can use modul datetime to use current date and time in filename.
https://docs.python.org/3.5/library/datetime.html
There is also module to generate temporary (random and unique) filename.
https://docs.python.org/3.5/library/tempfile.html
If you're writing the same data to multiple files, you can do something like this:
data = "some data"
files = ['file1.txt', 'file2.txt', 'file3.txt']
for file in files:
with open(file, "a") as f:
f.write(data)
Based on your comment concerning a new file name for each run:
import time
randomData = ("Some Random stuff")
t,s = str(time.time()).split('.')
filename = t+".txt"
print ("writing to", filename)
with open(filename, "a") as file:
file.write(randomData)
I have a python code which opens a text file read and do some processes. The input text file has to be updated each day with a new one. Currently I use following code to open the file.
f = open('sample20130616.txt','r')
But in this method some one has to go inside the code and modify the file name(Since the file names are not unique).
What is the proper method to handle this?
Pass the filename as parameter when you call your application:
python myapp.py file.txt
Then in your code:
import sys
if __name__ == '__main__':
filename = sys.argv[1]
f = open(filename)
You can easily generate the current date and hence your filename with the datetime module:
import datetime
today = datetime.date.today()
filename = 'sample{0}{1:02d}{2:02d}.txt'.format(today.year, today.month, today.day)
f = open(filename,'r')
Use the built-in time function!
from time import gmtime, strftime
fname='sample'+strftime("%Y%m%d", gmtime())+'.txt'
f = open(fname,'r')