Develop Reference

Renaming log files with Python using wildcard and datetime

I was searching today for options to manipulate some log files, after executing actions on them, and I found that Python has the os.rename resource, after importing the os module, but I have some regex doubts..
Tried to fit a wildcard "*****" on my file names, but Python seems not to understand it.
My file names are:
Application_2021-08-06_hostname_[PID].log
Currently I'm asking Python to read these application files, and search for defined words/phrases such as "User has logged in", "User disconnected" and etc. And he does well. I'm using datetime module so Python will always read the current files only.
But what I'm trying to do now, is to change the name of the file, after Python read it and execute something. So when he find "Today's sessions are done", he will change the name of the file to:
Application_06-08-2021_hostname_[PID].log
Because it will be easier for manipulating later..
Considering that [PID] will always change, this is the part that I wanted to set the wildcard, because it can be 56, 142, 3356, 74567 or anything.
Using the os.rename module, I've got some errors. What do you suggest?
Code:
import os
import time
from datetime import datetime
path = '/users/application/logs'
file_name = 'Application_%s_hostname_'% datetime.now().strftime('%Y-%m-%d')
new_file_name = 'Application_%s_hostname_'% datetime.now().strftime('%d-%m-%Y')
os.rename(file_name, new_file_name)
The error is:
OSError: [Errno 2] No such file or directory

You can use glob which allows for wildcards:
import glob, os
from datetime import datetime
current_date = datetime.now()
path = '/users/application/logs'
# note the use of * as wild card
filename ='Application_%s_hostname_*'% current_date.strftime('%Y-%m-%d')
full_path = os.path.join(path, filename)
replace_file = glob.glob(full_path)[0] # will return a list so take first element
# or loop to get all files
new_name = replace_file.replace( current_date.strftime('%Y-%m-%d'), current_date.strftime('%d-%m-%Y') )
os.rename(replace_file, new_name)

Finding a file in the same directory as the python script (txt file)

file = open(r"C:\Users\MyUsername\Desktop\PythonCode\configure.txt")
Right now this is what im using. However if people download the program on their computer the file wont link because its a specific path. How would i be able to link the file if its in the same folder as the script.

You can use __file__. Technically not every module has this attribute, but if you're not loading your module from a file, loading a text file from the same folder becomes a moot point anyway.
from os.path import abspath, dirname, join
with open(join(dirname(abspath(__file__)), 'configure.txt')):
...
While this will do what you're asking for, it's not necessarily the best way to store configuration.

Use os module to get your current filepath.
import os
this_dir, this_filename = os.path.split(__file__)
myfile = os.path.join(this_dir, 'configure.txt')
file = open(myfile)

# import the OS library
import os
# create the absolute location with correct OS path separator to file
config_file = os.getcwd() + os.path.sep + "configure.txt"
# Open file
file = open(config_file)
This method will make sure the correct path separators are used.

Renaming file extension using pathlib (python 3)

I am using windows 10 and winpython. I have a file with a .dwt extension (it is a text file). I want to change the extension of this file to .txt.
My code does not throw any errors, but it does not change the extension.
from pathlib import Path
filename = Path("E:\\seaborn_plot\\x.dwt")
print(filename)
filename_replace_ext = filename.with_suffix('.txt')
print(filename_replace_ext)
Expected results are printed out (as shown below) in winpython's ipython window output:
E:\seaborn_plot\x.dwt
E:\seaborn_plot\x.txt
But when I look for a file with a renamed extension, the extension has not been changed, only the original file exists. I suspect windows file permissions.

You have to actually rename the file not just print out the new name.
Use Path.rename()
from pathlib import Path
my_file = Path("E:\\seaborn_plot\\x.dwt")
my_file.rename(my_file.with_suffix('.txt'))
Note: To replace the target if it exists use Path.replace()
Use os.rename()
import os
my_file = 'E:\\seaborn_plot\\x.dwt'
new_ext = '.txt'
# Gets my_file minus the extension
name_without_ext = os.path.splitext(my_file)[0]
os.rename(my_file, name_without_ext + new_ext)
Ref:
os.path.splitext(path)
PurePath.with_suffix(suffix)

From the docs:
Path.rename(target)
Rename this file or directory to the given target. On Unix, if target exists and is a file, it will be replaced silently if the user has permission. target can be either a string or another path object.
pathlib — Object-oriented filesystem paths on docs.python.org
You could use it like this:
from pathlib import Path
filename = Path("E:\\seaborn_plot\\x.dwt")
filename_replace_ext = filename.with_suffix(".txt")
filename.rename(filename_replace_ext)

python import a module with source code in temporary file

import tempfile
tmp = tempfile.NamedTemporaryFile(delete=True)
try:
# do stuff with temp
tmp.write(b'def fun():\n\tprint("hello world!")\n')
if __name__ == '__main__':
func = __import__(tmp.name)
func.fun()
finally:
tmp.close() # deletes the file
So I want to create a temporary file, add some source code to it and then import the module and call the function, but I always run into this error:
ModuleNotFoundError: No module named '/var/folders/3w/yyp887lx4018h9s5sr0bwhkw0000gn/T/tmp84bk0bic'
It doesn't seem to find the module of the temporary file. How do I solve this?

There are a few problems with your code:
Your filename does not end with .py, but Python modules are expected to. You can fix this by setting suffix='.py' in NamedTemporaryFile().
__import__() is not the right way to load a module from a full path. See here: How to import a module given the full path?
You do not flush after writing and before importing, so even if Python does find the file, it may well be empty. Add tmp.flush() after writing to fix this.

Importing can only be done from certain directories which are part of the PYTHON_PATH. You can extend that. Then you will have to use __import__() with a module name (not a path in the file system). You will have to deal with the suffix for the temp file.
I implemented a simple version using the local directory for the temp module file and a version using a proper tempfile:
#!/usr/bin/env python3
import sys
import os
import tempfile
SCRIPT = '''\
def fun():
print("hello world!")
'''
# simple version using the local directory:
with open('bla.py', 'w') as tmp_module_file:
tmp_module_file.write(SCRIPT)
import bla
bla.fun()
# version using the tempfile module:
tmpfile = tempfile.NamedTemporaryFile(suffix='.py', delete=True)
try:
tmpfile.write(SCRIPT.encode('utf8'))
tmpfile.flush()
tmpmodule_path, tmpmodule_file_name = os.path.split(tmpfile.name)
tmpmodule_name = tmpmodule_file_name[:-3] # strip off the '.py'
sys.path.append(tmpmodule_path)
tmpmodule = __import__(tmpmodule_name)
finally:
tmpfile.close()
tmpmodule.fun()

How to include Variable in File Path (Python)

I am currently writing a small networkchat in Python3. I want to include a function to save the users history. Now my user class contains a variable for name and I want to save the history-file in a folder which has the name of the user as its name.
So for example it roughly looks like that:
import os
import os.path
class User:
name = "exampleName"
PATH = './exampleName/History.txt'
def SaveHistory(self, message):
isFileThere = os.path.exists(PATH)
print(isFileThere)
So it is alwasy returning "false" until I create a folder called "exampleName".
Can anybody tell me how to get this working?
Thanks alot!

if you use relative paths for file or directory names python will look for them (or create them) in your current working directory (the $PWD variable in bash).
if you want to have them relative to the current python file, you can use (python 3.4)
from pathlib import Path
HERE = Path(__file__).parent.resolve()
PATH = HERE / 'exampleName/History.txt'
if PATH.exists():
print('exists!')
or (python 2.7)
import os.path
HERE = os.path.abspath(os.path.dirname(__file__))
PATH = os.path.join(HERE, 'exampleName/History.txt')
if os.path.exists(PATH):
print('exists!')
if your History.txt file lives in the exampleName directory below your python script.