I am writting a script to backup a list of paths, inspired by https://github.com/Johanndutoit/Zip-to-FTP-Backup/blob/master/backup_to_ftp.py
So I have an ini file
[folders]
/home/david/docs
/home/david/images
/home/david/videos
[ftp]
username=etc
password=pwd
The code to read it is:
config = configparser.ConfigParser(allow_no_value=True)
config.optionxform = lambda option: option # preserve case for letters
config.read('backupcfg.ini')
filelistings = [] # All Files that will be added to the Archive
# Add Files From Locations
for folder in config.options("folders"):
filelistings.append(str(folder.strip("'")))
The problem is I can't find a way to read it as raw when I'm running it on windows, with folders like
[folders]
Z:\Desktop\winpython
I can't scape the backslash. I've ended up including it as:
filelistings = [r'Z:\docs\winpython', r'Z:\images\family']
Is there any way to write the paths in the ini? Can't find a way to read config.options("folders") to return raw.
Thank you for your help!
I have tried to add the path straight to the tar:
config = configparser.ConfigParser(allow_no_value=True)
config.optionxform = lambda option: option # preserve case for letters
config.read('backupcfg.ini')
now = datetime.datetime.now()
zipname = 'backup.' + now.strftime("%Y.%m.%d") + '.tgz'
with tarfile.open(zipname, "w:gz") as tar:
for name in config.options("folders"):
print ("Adding "+name)
tar.add(str(name))
Which reports can't find filename: 'Z'
Is there any way to access that information as it is, with ?
I got the idea for this answer from this answer. The problem isn't with the slashes. It's with the colon (:). By default, Configparser uses = and : as delimiters. But you can specify other delimiters to use instead.
backupcfg.ini
[folders]
/home/david/docs
Z:\Desktop\winpython
python code
import configparser
config = configparser.ConfigParser(allow_no_value=True, delimiters=(','))
config.optionxform = lambda option: option # preserve case for letters
config.read('backupcfg.ini')
for folder in config.options("folders"):
print(folder)
output:
/home/david/docs
Z:\Desktop\winpython
Related
i am writing an python script. I was having some problems to open the file. The error was always that system just can not find the file.
Because of that i tried get the active path... Replace backslash ... and so on....
Is there any improvements to work with the file in the same folder?
The Code
import os
# The name of the txt file that is in the same folder.
myFile = 'noticia.txt'
# Getting the active script
diretorio = os.path.dirname(os.path.abspath(__file__))
# Replace BackSlash and concatenate myFile
correctPath = diretorio.replace("\\", "/") + "/" + myFile
# Open file
fileToRead = open(correctPath, "r")
# Store text in a variable
myText = fileToRead.read()
# Print
print(myText)
Note:
The script is in the same folder of the txt file.
Is there any improvements to work with the file in the same folder?
First off, please see PEP 8 for standard conventions on variable names.
correctPath = diretorio.replace("\\", "/") + "/" + myFile
While forward slashes are preferred when you specify a new path in your code, there is no need to replace the backslashes in a path that Windows gives you. Python and/or Windows will translate behind the scenes as necessary.
However, it would be better to use os.path.join to combine the path components (something like correct_path = os.path.join(diretorio, my_file)).
fileToRead = open(correctPath, "r")
# Store text in a variable
myText = fileToRead.read()
It is better to use a with block to manage the file, which ensures that it is closed properly, like so:
with open(correct_path, 'r') as my_file:
my_text = my_file.read()
I have around 50 files that have their name and then the date they were created at 3 times. How can I remove that part from the file name in python (You can show an example with other data it doesn't really matter)
I tried something like that:
file = 'directory/imagehellohellohello.png'
keyword = 'hello'
if (file.count(keyword) >= 3):
//functionality (here I want to remove the hello's from the file path)
This can be done quite simply using pathlib:
from pathlib import Path
path = Path("directory/imagehellohellohello.png")
target = path.with_name(path.name.replace("hello", ''))
path.rename(target)
And this indeed renames the file to "directory/image.png".
From Python version 3.8 the rename method also returns the new files' path as a Path object. (So it is possible to do:
target = path.rename(path.with_name(path.name.replace("hello", '')))
Methods/attributes used: Path.rename, Path.with_name, Path.name, str.replace
file = 'directory/imagehellohellohello.png'
keyword = 'hello'
if keyword*3 in file:
newname = file.replace(keyword*3, '')
os.rename(file, newname)
I am trying to rename a set of files in a directory using python. The files are currently labelled with a Pool number, AR number and S number (e.g. Pool1_AR001_S13__fw_paired.fastq.gz.) Each file refers to a specific plant sequence name. I would like to rename these files by removing the 'Pool_AR_S' and replacing it with the sequence name e.g. 'Lbienne_dor5_GS1', while leaving the suffix (e.g. fw_paired.fastq.gz, rv_unpaired.fastq.gz), I am trying to read the files into a dictionary, but I am stuck as to what to do next. I have a .txt file containing the necessary information in the following format:
Pool1_AR010_S17 - Lbienne_lla10_GS2
Pool1_AR011_S18 - Lbienne_lla10_GS3
Pool1_AR020_S19 - Lcampanulatum_borau4_T_GS1
The code I have so far is:
from optparse import OptionParser
import csv
import os
parser = OptionParser()
parser.add_option("-w", "--wanted", dest="w")
parser.add_option("-t","--trimmed", dest="t")
parser.add_option("-d", "--directory", dest="working_dir", default="./")
(options, args) = parser.parse_args()
wanted_file = options.w
trimmomatic_output = options.t
#Read the wanted file and create a dictionary of index vs species identity
with open(wanted_file, 'rb') as species_sequence:
species_list = list(csv.DictReader(species_sequence, delimiter='-'))
print species_list
#Rename the Trimmomatic Output files according to the dictionary
for trimmed_sequence in os.listdir(trimmomatic_output):
os.rename(os.path.join(trimmomatic_output, trimmed_sequence),
os.path.join(trimmomatic_output, trimmed_sequence.replace(species_list[0], species_list[1]))
Please can you help me to replace half of the . I'm very new to python and to stack overflow, so I am sorry if this question has been asked before or if I have asked this in the wrong place.
First job is to get rid of all those modules. They may be nice, but for a job like yours they are very unlikely to make things easier.
Create a .py file in the directory where those .gz files reside.
import os
files = os.listdir() #files is of list type
#'txt_file' is the path of your .txt file containing those conversions
dic=parse_txt(txt_file) #omitted the body of parse_txt() func.Should return a dictionary by parsing that .txt file
for f in files:
pre,suf=f.split('__') #"Pool1_AR001_S13__(1)fw_paired.fastq.gz"
#(1)=assuming prefix and suffix are divided by double underscore
pre = dic[pre]
os.rename(f,pre+'__'+suf)
If you need help with parse_txt() function, let me know.
Here is a solution that I tested with Python 2. Its fine if you use your own logic instead of the get_mappings function. Refer comments in code for explanation.
import os
def get_mappings():
mappings_dict = {}
with(open('wanted_file.txt', 'r')) as f:
for line in f:
# if you have Pool1_AR010_S17 - Lbienne_lla10_GS2
# it becomes a list i.e ['Pool1_AR010_S17 ', ' Lbienne_lla10_GS2']
#note that there may be spaces before/after the names as shown above
text = line.split('-')
#trim is used to remove spaces in the names
mappings_dict[text[0].strip()] = text[1].strip()
return mappings_dict
#PROGRAM EXECUTION STARTS FROM HERE
#assuming all files are in the current directory
# if not replace the dot(.) with the path of the directory where you have the files
files = os.listdir('.')
wanted_names_dict = get_mappings()
for filename in files:
try:
#prefix='Pool1_AR010_S17', suffix='fw_paired.fastq.gz'
prefix, suffix = filename.split('__')
new_filename = wanted_names_dict[prefix] + '__' + suffix
os.rename(filename, new_filename)
print 'renamed', filename, 'to', new_filename
except:
print 'No new name defined for file:' + filename
I have some code in python that checks that a file path access a file in a subdirectory.
It is for a web server to access files in the static folder.
I use the following code snippet :
path = 'static/' + path
try:
if '/../' in path:
raise RuntimeError('/../ in static file path')
f = open(path)
except (RuntimeError, IOError):
app.abort(404)
return
If the path is clean it would be enough.
Could there be ways to write a path accessing parent directories that would not be detected by this simple test ?
I would suggest using os.path.relpath, it takes a path and works out the most concise relative path from the given directory. That way you only need to test if the path starts with ".."
eg.
path = ...
relativePath = os.path.relpath(path)
if relativePath.startswith(".."):
raise RuntimeError("path escapes static directory")
completePath = "static/" + relativePath
You could also use os.readlink to replace symbolic links with a real path if sym links are something you have to worry about.
Flask has a few helper functions that I think you can copy over to your code without problems. The recommended syntax is:
filename = secure_filename(dirty_filename)
path = os.path.join(upload_folder, filename)
Werkzeug implements secure_filename and uses this code to clean filenames:
_filename_ascii_strip_re = re.compile(r'[^A-Za-z0-9_.-]')
_windows_device_files = ('CON', 'AUX', 'COM1', 'COM2', 'COM3', 'COM4', 'LPT1',
'LPT2', 'LPT3', 'PRN', 'NUL')
def secure_filename(filename):
r"""Pass it a filename and it will return a secure version of it. This
filename can then safely be stored on a regular file system and passed
to :func:`os.path.join`. The filename returned is an ASCII only string
for maximum portability.
On windows system the function also makes sure that the file is not
named after one of the special device files.
>>> secure_filename("My cool movie.mov")
'My_cool_movie.mov'
>>> secure_filename("../../../etc/passwd")
'etc_passwd'
>>> secure_filename(u'i contain cool \xfcml\xe4uts.txt')
'i_contain_cool_umlauts.txt'
The function might return an empty filename. It's your responsibility
to ensure that the filename is unique and that you generate random
filename if the function returned an empty one.
.. versionadded:: 0.5
:param filename: the filename to secure
"""
if isinstance(filename, unicode):
from unicodedata import normalize
filename = normalize('NFKD', filename).encode('ascii', 'ignore')
for sep in os.path.sep, os.path.altsep:
if sep:
filename = filename.replace(sep, ' ')
filename = str(_filename_ascii_strip_re.sub('', '_'.join(
filename.split()))).strip('._')
# on nt a couple of special files are present in each folder. We
# have to ensure that the target file is not such a filename. In
# this case we prepend an underline
if os.name == 'nt' and filename and \
filename.split('.')[0].upper() in _windows_device_files:
filename = '_' + filename
return filename
..//
This essentially is the same thing, however since you are straight away matching strings for /../
the, one I added would go undetected and would get the parent directory.
According to all the sources I've read, the open method creates a file or overwrites one with an existing name. However I am trying to use it and i get an error:
File not found - newlist.txt (Access is denied)
I/O operation failed.
I tried to read a file, and couldn't. Are you sure that file exists? If it does exist, did you specify the correct directory/folder?
def getIngredients(path, basename):
ingredient = []
filename = path + '\\' + basename
file = open(filename, "r")
for item in file:
if item.find("name") > -1:
startindex = item.find("name") + 5
endindex = item.find("<//name>") - 7
ingredients = item[startindex:endindex]
ingredient.append(ingredients)
del ingredient[0]
del ingredient[4]
for item in ingredient:
printNow(item)
file2 = open('newlist.txt', 'w+')
for item in ingredient:
file2.write("%s \n" % item)
As you can see i'm trying to write the list i've made into a file, but its not creating it like it should. I've tried all the different modes for the open function and they all give me the same error.
It looks like you do not have write access to the current working directory. You can get the Python working directory with import os; print os.getcwd().
You should then check whether you have write access in this directory. This can be done in Python with
import os
cwd = os.getcwd()
print "Write access granted to current directory", cwd, '>', os.access(cwd, os.W_OK)
If you get False (no write access), then you must put your newfile.txt file somewhere else (maybe at path + '/newfile.txt'?).
Are you certain the directory that you're trying to create the folder in exists?
If it does NOT... Then the OS won't be able to create the file.
This looks like a permissions problem.
either the directory does not exist or your user doesn't have the permissions to write into this directory .
I guess the possible problems may be:
1) You are passing the path and basename as parameters. If you are passing the parameters as strings, then you may get this problem:
For example:
def getIngredients(path, basename):
ingredient = []
filename = path + '\\' + basename
getIngredients("D","newlist.txt")
If you passing the parameters the above way, this means you are doing this
filename = "D" + "\\" + "newlist.txt"
2) You did not include a colon(:) after the path + in the filename.
3) Maybe, the file does not exist.