I'm sure this is really simple but I can't figure out how to make a parsed result into its own file then move it to my desktop using python. Here is my code so far. I just want to save the result "names" as its own file then move it to my desktop but I can't find the answer anywhere. Is this an uncommon practice?
from gedcom.element.individual import IndividualElement
from gedcom.parser import Parser
import os
import shutil
import pickle
# Path to your `.ged` file
file_path = '/Users/Justin/Desktop/Lienhard_Line.ged'
# Initialize the parser
gedcom_parser = Parser()
# Parse your file
gedcom_parser.parse_file(file_path, False)
root_child_elements = gedcom_parser.get_root_child_elements()
# Iterate through all root child elements
for element in root_child_elements:
# Is the `element` an actual `IndividualElement`? (Allows usage of extra functions such as `surname_match` and `get_name`.)
if isinstance(element, IndividualElement):
# Get all individuals whose surname matches "Doe"
if element.surname_match('Lienhard'):
# Unpack the name tuple
(first, last) = element.get_name()
names = (first + " " + last)
pickle.dumps(names)
Saving a file to one location and then moving it is to another not how it's usually done, no. Just save to the final location.
from pathlib import Path
pic = Path.home() / 'Desktop' / 'names.pickle'
with pic.open('w') as picklefile:
pickle.dump(names, picklefile)
The pathlib library makes working with file names somewhat easier than the old os.path API, though both are in common use.
Writing and then renaming has some uses; if you need to absolutely make sure your data is saved somewhere, saving it to a temporary file until it can be renamed to its final name is a fairly common technique. But in this case, saving to a different location first seems like it would only introduce brittleness.
The above assumes that you have a directory named Desktop in your home directory, as is commonly the default on beginner-oriented systems. To be perfectly robust, the code should create the directory if it doesn't already exist, or perhaps simply save to your home directory.
Indeed, a much better convention for most purposes is simply to always save in the current directory, and let the user take it from there. Hardcoding paths in another directory just adds confusion and uncertainty.
with open('names.pickle', 'w') as picklefile:
pickle.dump(names, picklefile)
This code creates a file called names.pickle in the invoking user's current working directory. The operating system provides the concept of a current working directory for precisely this purpose. Perhaps see also Difference between ./ and ~/ if you need more details about how this works.
Related
I tried creating a code where the file, when you run creates a replica of itself and deletes the original file.
Here is my code:
import shutil
import os
loc=os.getcwd()
shutil.move("./aa/test.py", loc, copy_function=shutil.copy2)
But the issue with this is that:
this code is only 1 time usable and to use it again, I need to change the name of the file or delete the newly created file and then run it again.
Also, If I run it inside a folder, It will always create the new file outside the folder (in a dir up from the exceuting program).
How Do I fix this?
Some Notes:
The copy should be made at the exact place where the original file was.
The folder was empty, just having this file. The file doesn't needs to be in a folder but I just used it as a test instance.
Yes, I understand that if I delete the original file it should stop working. I actually have a figure in my mind of how It should work:
First, a new file with the exact same content in it will be made > in the same path as the original file (with a different name probably).
Then, the original file will be deleted and the 2nd file (which is > the copy of the original file) will be renamed as the exact name and > extension as of the original file which got deleted.
This thing above should repeat every time I run the .py file (containing this code) thus making this code portable and suitable for multiple uses.
Maybe the code to be executed after the file deletion can be stored in memory cache (I guess?).
Easiest way (in pseudo code):
Get name of current script.
Read all contents in memory.
Delete current script.
Write memory contents into new file with the same name.
this code is only 1 time usable and to use it again, I need to change the name of the file or delete the newly created file and then run it again.
That is of course because the file is called differently. You could approach this by having no other files in that folder, or always prefixing the filename in the same way, so that you can find the file although it always is called differently.
Also, If I run it inside a folder, It will always create the new file outside the folder (in a dir up from the exceuting program).
That is because you move it from ./aa to ./. You can take the path of the file and reuse it, apart for the filename, and then it would be in the same folder.
Hey TheKaushikGoswami,
I believe your code does exactly what you told him to and as everybody before me stated, surely only works once. :)
I would like to throw in another idea:
First off I'd personally believe that shutil.move is more of a method for actually moving a file into another directory, as you did by accident.
https://docs.python.org/3/library/shutil.html#shutil.move
So why not firstly parameterize your folder (makes it easier for gui/cmd access) and then just copy to a temporary file and then copying from that temporary file. That way you wont get caught in errors raised if you try to create files already existing and have an easy-to-understand code!
Like so:
import shutil
import os
try:
os.mkdir('./aa/')
except:
print('Folder already exists!')
dest= './aa/'
file= 'test.py'
copypath = dest + 'tmp' + file
srcpath = dest + file
shutil.copy2(srcpath, copypath, follow_symlinks=True)
os.remove(srcpath)
shutil.copy2(copypath, srcpath, follow_symlinks=True)
os.remove(copypath)
But may I ask what your use-case is for that since it really doesn't change anything for me other than creating an exact same file?
I am trying to rename folders in bulk based on the folders/files contained within them (then moving the image files at the end of each path into they're respective model/color folder directories).
Each folder/file has a similar naming convention of MODEL_COLOR.
The code below works, though seems to only be working correctly on the first folder, in other words, the folders are being renamed correctly but the last leg of code seems to be taking the folder which contains the images and moves it to the corresponding path, instead of specifically moving the images to the corresponding path and dropping the folder they're originally in.
On the first folder the loop iterates, it actually moves the images to the correct Model > Color directory, though on all folders after that it seems to be moving the folder containing the images into the correct Model > Color directory, instead of just moving the images alone into the corresponding directory.
After looking at the forums I've seen similar issues where when changing the directory or deleting certain instances, the loop can't iterate correctly due to the initial set changing during the looping process (i.e. deleting or renaming part of the path while iterating). I'm pretty sure it's a simple fix but I can't seem to find the solution that'll work best.
Standard FolderNames:
CL4003IN_45F
CL4003IN_56F
CL40157U_01D
CL40157U_52H
import glob, os, shutil
folder = 'C:\\testing'
# create new folder directory based on Model/Color [is working, but moves file_path into base directory]
# arr = []
for file_path in glob.glob(os.path.join(folder, '*_*')):
new_dir = file_path.rpartition('_')[0]
new_subdir = file_path.rpartition('_')[2]
try:
os.mkdir(os.path.join(new_dir, new_subdir))
except WindowsError:
# Handle the case where the target dir already exist.
pass
shutil.move(file_path, os.path.join(new_dir, new_subdir))
# arr.append(file_path)
Completing the iteration of glob before the loop by storing it in a list helped avoid some unwanted errors.
#...
for file_path in list(glob.glob(os.path.join(folder, '*_*')
...#
But by modifying my code and removing the following from the loop:
try:
os.mkdir(os.path.join(new_dir, new_subdir))
except WindowsError:
pass
Allowed the code to iterate through all the folders in the directory without transferring the folder before the file into the new_dir > new_subdir directory.
The new code that works across a multitude of folders within a directory is:
import glob, os, shutil
folder = 'C:\\testing'
# create new folder directory based on Model > Color
for file_path in list(glob.glob(os.path.join(folder, '*_*'), recursive=True)):
new_dir = file_path.rpartition('_')[0]
new_subdir = file_path.rpartition('_')[2]
shutil.move(file_path, os.path.join(new_dir, new_subdir))
This may not be the most efficient code (and may not work across all instances, that remains to be determined!), though definitely works as intended for now.
Special thanks to those that helped with their suggestions.
The reason why you don't see the actual error is that you catch too many errors in your except: statement.
You intend to catch FileExistsError, so you should also just look for this one. Otherwise you would have noticed that the code in fact throws a FileNotFoundError.
The reason for that is that os.mkdir does not automatically create parent directories. It only creates directories one layer deep, but your code requires two layers of new directories.
For that, you would have to use os.makedirs(...) instead. Conveniently, os.makedirs also accepts an exist_ok flag, which gets rid of your entire try:-except: construct.
A further annotation is that you have a lot of duplicate calculations in your code:
file_path.rpartition('_') gets calculated twice
os.path.join(new_dir, new_subdir) gets calculated twice
I'd suggest storing those in meaningful variables. This speeds up your code, makes it more readable and more maintainable.
Here's a reworked version of your code:
import glob
import os
import shutil
folder = 'C:\\testing'
for file_path in glob.glob(os.path.join(folder, '*_*')):
(new_dir, _, new_subdir) = file_path.rpartition('_')
target_path = os.path.join(new_dir, new_subdir)
os.makedirs(target_path, exist_ok=True)
shutil.move(file_path, target_path)
Further improvements/fixes
There's still a bunch wrong in your code:
You don't check if the thing found by glob is a file
Splitting by _ does not stop at the directory divider. This means that something like C:\\my_files\\bla will get split into C:\\my and files\\bla.
I think you didn't care about either of those, because you thought 'the user would not use the script like this'. But this is actually a case that will happen:
You have a file C:\\my_files\\CL4003IN_45F, which will get moved to C:\\my_files\\CL4003IN\\45F\\CL4003IN_45F, as expected.
You run the script again. The script will find C:\\my_files\\CL4003IN. It doesn't check if it's a folder, so it will process it anyway. It will then split it into C:\\my and files\\CL4003IN.
The entire folder C:\\my_files\\CL4003IN will get moved to C:\\my\\files\\CL4003IN. Therefore the original file CL4003IN_45F ends up in C:\\my\\files\\CL4003IN\\CL4003IN\\45F\\CL4003IN_45F
The solution is:
only use rpartition on the filename, not the entire path
check if it actually is a file, or a directory
Most of these tasks get solved easier with pathlib. I took the liberty of rewriting your code and fixing those issues:
from pathlib import Path
folder = Path('C:\\testing')
for file_path in folder.iterdir():
file_name = file_path.name
# Ignore directories
if not file_path.is_file():
continue
# Split the filename by '_'
(target_dir, _, target_subdir) = file_name.rpartition('_')
# If no '_' found, ignore
if not target_dir:
continue
# Compute the target path and create it if necessary
target_path = folder / target_dir / target_subdir
target_path.mkdir(parents=True, exist_ok=True)
# Move the file to its new position
target_file_path = target_path / file_name
file_path.rename(target_file_path)
One last remark: folder.iterdir() actually does return an iterator. But that shouldn't be a problem in this case, as we explicitely check if the path is an existing file and not a directory or something that already got deleted. But if you want to be 100% safe write list(folder.iterdir()).
I'm trying to create a program that duplicates itself to another location and creates a batch file on the desktop. I can make it duplicate itself and I can create the batch file but I need some help with the paths.
I can find the path that my program is currently in. Both the direct path and the path to the directory. My problem lies in the fact that I want to place the file in (let's just say for simplicity) 'C:\Users\Me\Documents'. How would I edit the path? I want to be able to place this on a generic windows computer so I can't hard code the path in because each user will be different. This goes the same for placing the batch file and setting it for the right directory to run the python script in documents.
I have tried both
import os
print os.path.dirname(os.path.abspath(__file__))
and
import os
print os.path.abspath(__file__)
but am clueless as to how to edit the path. When I try googling for it and searching this wonderful site, all I get is stuff about configuring the Python path on windows and other stuff that I can't quite understand at my current level of Python.
Now I turn to you, can you help? Any input would be appreciated, if you could explain how it worked that would be even better!
<>
Due to some questions about my code (and a specific one to post it) Here it is
from sys import argv # Imports module
import os
script, create = argv # Gets script name and desired amount of copies
data = open(script) # Creates a variable to store the script
indata = copy.read() # Creates the data to be copied from the script
batData = """
echo off
%s
""" % # This is not finished, creating that batch file
createT = int(create) + 1
for i in range(1, createT): # Runs a set amount of times
copyName = "%s.py" % str(i) # Creates the name for the file
copy = open(copyName, 'w+') # Opens/creates the file for editing
copy.write(indata) # Writies the indata to the file opened
copy.close # Closes that file
batName = "%s.bat" % str(i)
bat = open(batName, 'w+')
It is not finished but hopefully you get the gist. The argv at the beginning is so I can change the amount of copies made, that will be deleted later as I evolve the code but for now I like it there.
I have currently tried the following to find the path:
import os
print os.path.abspath(__file__)
print os.path.dirname(os.path.abspath(__file__))
print os.path.dirname(__file__)
test = os.path.dirname(__file__)
a, b, c, d, e, f, g, h, i = test.split("\\")
print c
What I want to happen (or think I want to happen) is for the path to be found, then split into pieces (either each directory or break off everything after the username). Then I want to append the document folder tag to the end. For the batch instead of the document tag it will be for the desktop.
among a few others that people have posted. I hope this helps!
Your code snippet returns a string. Just take that string and edit to make the path you want. I'm on a mac so I can't test with an actual windows directory but I'll try to make it look Windows-ish. For instance, lets say this code:
directory_path = os.path.dirname(os.path.abspath(__file__))
print(directory_path)
gives you:
C:\Users\username\AppData
You can use the split function to break the path into pieces (docs).
stuff = path_string.split('\')
print(stuff)
Code output:
['C:', 'Users', 'username', 'AppData']
You can use the pieces create the path you want and then use it to write the file. So, if you want the username folder just loop until you find it. Some example code is below (just an example to get you started - read up on Python if you need help understanding the code).
username = ""
for i in range(0, len(stuff)):
if stuff[i] == "Users":
username = stuff[i + 1]
Not sure if that answers your question but hope it helps.
Am I correct that you are trying to figure out how to make a file path to some location on the user's directory without knowing who the user is going to be that is executing the program?
You may be able to take advantage of environment variables for this. For instance, I can get a file path to the C://Users/username/ directory of whoever is executing the code with:
my_path = os.path.join("C:\\", "Users", os.getenv("USERNAME"))
Where os.getenv("USERNAME") returns the value of the USERNAME environment variable (should be the name of the user that is currently logged on).
You can list all of the available environment variables and their values in Python with:
for key, val in os.environ.items():
print("{} \t {}\n".format(key, val)) # or however you want to prettify it
You may get lucky and find an environment variable that gives you most of the path to begin with.
In general, os.path.join(), os.path.relpath(), and os.path.abspath() combined with some environment variables might be able to help you. Check out the os documentation and os.path documentation.
Many times when modifying a path, I am looking to add/remove folders. Here is my simple method for adding a path, e.g. if I want to move the path of an object into a folder added_path='/train/.
Since my paths are usually uniform, I check the last split characters in the first file location. Usually, my experience is that windows have \\ at the end while Mac and Linux have `/', which makes this work across operating systems. (note: if the paths are not uniform, you obviously place the if-else in the for-loop.)
if '\\' in data[0].img_file:
split_char = '\\'
else:
split_char = '/'
for img in data:
img_path = img.img_file.split(split_char)
label_path = img.label_file.split(split_char)
img.img_file = '/'.join(img_path[:-1]) + added_path + img_path[-1]
img.label_file = '/'.join(label_path[:-1]) + added_path + label_path[-1]
So, this for loop uses all the folders up until the file name, where we insert the extra folder, and then, at last, add the file name.
Example input path: 'data/combined/18.png'
Example output path: 'data/combined/train/18.png'
Using IronPython 2.6 (I'm new), I'm trying to write a program that opens a file, saves it at a series of locations, and then opens/manipulates/re-saves those. It will be run by an upper-level program on a loop, and this entire procedure is designed to catch/preserve corrupted saves so my company can figure out why this glitch of corruption occasionally happens.
I've currently worked out the Open/Save to locations parts of the script and now I need to build a function that opens, checks for corruption, and (if corrupted) moves the file into a subfolder (with an iterative renaming applied, for copies) or (if okay), modifies the file and saves a duplicate, where the process is repeated on the duplicate, sans duplication.
I tell this all for context to the root problem. In my situation, what is the most pythonic, consistent, and windows/unix friendly way to move a file (corrupted) into a subfolder while also renaming it based on the number of pre-existing copies of the file that exist within said subfolder?
In other words:
In a folder structure built as:
C:\Folder\test.txt
C:\Folder\Subfolder
C:\Folder\Subfolder\test.txt
C:\Folder\Subfolder\test01.txt
C:\Folder\Subfolder\test02.txt
C:\Folder\Subfolder\test03.txt
How to I move test.txt such that:
C:\Folder\Subfolder
C:\Folder\Subfolder\test.txt
C:\Folder\Subfolder\test01.txt
C:\Folder\Subfolder\test02.txt
C:\Folder\Subfolder\test03.txt
C:\Folder\Subfolder\test04.txt
In an automated way, so that I can loop my program overnight and have it stack up the corrupted text files I need to save? Note: They're not text files in practice, just example.
assuming you are going to use the convention of incrementally suffinxing numbers to the files:
import os.path
import shutil
def store_copy( file_to_copy, destination):
filename, extension = os.path.splitext( os.path.basename(file_to_copy)
existing_files = [i for in in os.listdir(destination) if i.startswith(filename)]
new_file_name = "%s%02d%s" % (filename, len(existing_files), extension)
shutil.copy2(file_to_copy, os.path.join(destination, new_file_name)
There's a fail case if you have subdirectories or files in destination whose names overlap with the source file, ie, if your file is named 'example.txt' and the destination containst 'example_A.txt' as well as 'example.txt' and 'example01.txt' If that's a possibility you'd have to change the test in the existing files = line to something more sophisticated
I am making a python program, and I want to check if it is the users first time running the program (firstTime == True). After its ran however, I want to permanently change firstTime to False. (There are other variables that I want to take input for that will stay if it is the first run, but that should be solved the same way).
Is there a better way then just reading from a file that contains the data? If not, how can I find where the file is being ran from (so the data will be in the same dir)?
If you want to persist data, it will "eventually" be to disk files (though there might be intermediate steps, e.g. via a network or database system, eventually if the data is to be persistent it will be somewhere in disk files).
To "find out where you are",
import os
print os.path.dirname(os.path.abspath(__file__))
There are variants, but this is the basic idea. __file__ in any .py script or module gives the file path in which that file resides (won't work on the interactive command line, of course, since there's no file involved then;-).
The os.path module in Python's standard library has many useful function to manipulate path strings -- here, we're using two: abspath to give an absolute (not relative) version of the file's path, so you don't have to care about what your current working directory is; and dirname to extract just the directory name (actually, the whole directory path;-) and drop the filename proper (you don't care if the module's name is foo.py or bar.py, only in what directory it is;-).
It is enough to just create file in same directory if program is run first time (of course that file can be deleted to do stuff for first run again, but that can be sometimes usefull):
firstrunfile = 'config.dat'
if not os.path.exists(firstrunfile):
## configuration here
open(firstrunfile,'w').close() ## .write(configuration)
print 'First run'
firstTime == True
else:
print 'Not first run'
## read configuration
firstTime == False