I have just started working on web2py. Personally, I find it easier to learn than Django.
My query is that I have to load a file at application startup. Its a pickled hashtable. Where should I store this file so that the system is able to see it
My code is :
import cPickle as pickle
def index():
"""
Load the file into memory and message the number of entries
"""
f = open('tables.pkl','rb')
session.tables = pickle.load(f)
f.close()
terms = len(session.tables.keys())
message = 'The total entries in table = ' + str(terms)
return dict(message=message)
As you can see, I have put the code in index() to load it at startup. At present I am using the absolute path upto the physical location of the 'tables.pkl' file. Where should i put it in my application folder.
Also, I want tables variable to be available to all functions in the controller. Is session.tables the right way to go? It is just a search app so there is no user login.
The table has to be loaded only once for all users accessing the page.
Thank you.
I think the private folder would be a good place for this. You can get the absolute path with:
import os
fp = os.path.join(request.folder,'private','tables.pkl')
I would use cache instead of session if the file is not unique per user.
Related
I'm currently working on a program that uses databases and all that kind of stuff for a friend of mine. Now while doing my basic planning of what I will need to do, I thought to myself how will I get the program to connect to the right MS Access database if I hard code the path to my own personal computer? Because ill be using pyinstaller to package the program for him.
Do I need to build the directory in a dynamic way that uses the path of the current script, and then some string manipulation of that path to get the right file?
i.e.
import os
file_path = os.path.realpath(__file__)
# Using prior knowledge knowing the file name length is 7
file_path = file_path[:-7]
#appending the path string to point to the database
file_path = file_path + "/databases/test.accdb"
Or will the program still someway find its way to that database?
You can use a: os.walk(...) function to locate specified name file or every database file in specified directories. Next user can choose which file is correct
I'm sure this is really simple but I can't figure out how to make a parsed result into its own file then move it to my desktop using python. Here is my code so far. I just want to save the result "names" as its own file then move it to my desktop but I can't find the answer anywhere. Is this an uncommon practice?
from gedcom.element.individual import IndividualElement
from gedcom.parser import Parser
import os
import shutil
import pickle
# Path to your `.ged` file
file_path = '/Users/Justin/Desktop/Lienhard_Line.ged'
# Initialize the parser
gedcom_parser = Parser()
# Parse your file
gedcom_parser.parse_file(file_path, False)
root_child_elements = gedcom_parser.get_root_child_elements()
# Iterate through all root child elements
for element in root_child_elements:
# Is the `element` an actual `IndividualElement`? (Allows usage of extra functions such as `surname_match` and `get_name`.)
if isinstance(element, IndividualElement):
# Get all individuals whose surname matches "Doe"
if element.surname_match('Lienhard'):
# Unpack the name tuple
(first, last) = element.get_name()
names = (first + " " + last)
pickle.dumps(names)
Saving a file to one location and then moving it is to another not how it's usually done, no. Just save to the final location.
from pathlib import Path
pic = Path.home() / 'Desktop' / 'names.pickle'
with pic.open('w') as picklefile:
pickle.dump(names, picklefile)
The pathlib library makes working with file names somewhat easier than the old os.path API, though both are in common use.
Writing and then renaming has some uses; if you need to absolutely make sure your data is saved somewhere, saving it to a temporary file until it can be renamed to its final name is a fairly common technique. But in this case, saving to a different location first seems like it would only introduce brittleness.
The above assumes that you have a directory named Desktop in your home directory, as is commonly the default on beginner-oriented systems. To be perfectly robust, the code should create the directory if it doesn't already exist, or perhaps simply save to your home directory.
Indeed, a much better convention for most purposes is simply to always save in the current directory, and let the user take it from there. Hardcoding paths in another directory just adds confusion and uncertainty.
with open('names.pickle', 'w') as picklefile:
pickle.dump(names, picklefile)
This code creates a file called names.pickle in the invoking user's current working directory. The operating system provides the concept of a current working directory for precisely this purpose. Perhaps see also Difference between ./ and ~/ if you need more details about how this works.
I have a .py script that pulls in data from Google Sheets and outputs it in a yaml format. This is for my Hugo powered website which is served via Netlify. As I understand it, Netlify is capable of running Python too so I thought I could upload the web content and the python file in the same directory. This is required for updating the content dynamically, and I expect the python file to run everytime I trigger a build for the website. However, the python file requires certain credentials to work.
My code currently looks like this:
# Set location to write new files to.
outputpath = Path("D:/content/submission/")
#Read JSON:
json = Path("D:/credentials.json")
These are hardcoded local paths. When I bundle the script in the website directory, what paths should I write in so that when the script runs, these files are read in and outputted correctly?
I would want to output in my content/submission folder and read in from my creds/credentials.json. Should I just put these paths in? Will it know that it has to look within the directory for these folders, or is there something I need to add to the script that tells it to work within the directory it is sitting in?
🧨 First, credentials and secrets are best kept out of files (and esp, esp, source control).
For general file locations however, you can use something like:
pa_same_but_json = Path(__file__).with_suffix(".json")
pa_same_directory = Path(__file__).parent / "nosecrets.json")
To answer your comments:
(Mind you not 100% sure about Window Drives in the following):
parent
is an attribute on Path objects allowing you to "climb up" hierarchy.
Path("c:\temp\foo.json").parent returns same as Path("c:\temp")
Yes, you can do mypath.parent.parent
/ is a path concatenation operator
when applied to Path objects
So
myfile = os.path.join(["c:", "temp", "foo.json"])
and
myfile_as_a_Path = Path("c:") / "temp" / "foo.json"
are the same, except for one being a string, the other a Path instance. Once the first Path has been built (on C:) the rest of the code "knows that it operating on Path instances" and re-purposes the division operator support (probably some magic __div__ method intended for instance math ) to support path concatenation. This happens because most operations on Path instances return another Path, allowing you to do this type of chaining.
It's best not to write way up the hierarchy in a hosted/VM context (you never know directory structure above or if you have permissions), but something based on your script location might be
pa_current = Path(__file__).parent
# could `content/submission` but that's assuming you're always on Posix
# systems. Letting Pathlib do the work is safer, even if Windows probably
# puts up with `/`
pa_write = pa_current / "content" / "submission"
pa_read = pa_current / "credentials.json"
These at this points are Path instances, but really not much different than strings except having smarter methods to manipulate them. They don't know or care if the files exist or not.
P.S.
🧨 A consideration is that, in many web contexts, writing to code directories (like what happens in a content/submission under the python scripts) is a security goof as well.
Maybe pa_write = pa_current.parent.parent / "uploads" / "content" / "submission" would be better.
Specifically when it comes to user uploads and secrets, please refer to best practices for your platform, not just what Python can do. This answer was about pathlib.Path, not Hugo uploads.
I recently came across this question and answer https://stackoverflow.com/a/44926557/12322095 regarding Flask file uploads.
This worked perfectly fine until I uploaded an image with the same name again. It didn't change the image or overlayed it.
My question here is what if a user uploads an image with the same name, is there any way we could show an error message or maybe automatically change the name to something else.
For automatic changing the name, I researched and it can be done via resolve_conflict but I couldn't understand how to implement it.
my code is ditto as the the reference
You need to create some kind of uniuqe ID to append to the filename, before saving the file.
This can be done with something like:
from uuid import uuid4
def make_unique(string):
ident = uuid4().__str__()
return f"{ident}-{string}"
Which adds an unique UUID to the beginning of the string:
>>> make_unique('something.txt')
'f374b80c-b09e-468f-adc6-3d9df175cee7-something.txt'
To use this in the upload code, just run the filename through that function before you save. Be sure to put the filename through the secure_filename function first though:
if file and allowed_file(file.filename):
original_filename = secure_filename(file.filename)
unique_filename = make_unique(original_filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], unique_filename))
Although this works for the purpose of avoiding duplicates, in a larger application you may wish to extend this approach.
If you store the values of original_filename and unique_filename in the database, then this allows you to do the following in the download route:
from flask import send_file
# ...
f = os.path.join(app.config['UPLOAD_FOLDER'], unique_filename)
send_file(f, attachment_filename=original_filename)
This has the advantage that the file is stored on your server with a unique identifier, but the user would never know this, as the file is served back to them with the originally uploaded filename.
In fact you may wish to go further, and simply save the file on your end with the UUID string, rather than appending the filename; instead of using the make_unique function above, change that third line to:
unique_filename = uuid4().__str__()
This will still serve the file with the correct mimetype, as send_file guesses the mimetype based on the provided attachment_filename.
How do I access the tags attribute here in the Windows File Properties panel?
Are there any modules I can use? Most google searches yield properties related to media files, file access times, but not much related to metadata properties like Tags, Description etc.
the exif module was able to access a lot more properties than most of what I've been able to find, but still, it wasn't able to read the 'Tags' property.
The Description -> Tags property is what I want to read and write to a file.
There's an entire module dedicated to exactly what I wanted: IPTCInfo3.
import iptcinfo3, os, sys, random, string
# Random string gennerator
rnd = lambda length=3 : ''.join(random.choices(list(string.ascii_letters), k=length))
# Path to the file, open a IPTCInfo object
path = os.path.join(sys.path[0], 'DSC_7960.jpg')
info = iptcinfo3.IPTCInfo(path)
# Show the keywords
print(info['keywords'])
# Add a keyword and save
info['keywords'] = [rnd()]
info.save()
# Remove the weird ghost file created after saving
os.remove(path + '~')
I'm not particularly sure what the ghost file is or does, it looks to be an exact copy of the original file since the file size remains the same, but regardless, I remove it since it's completely useless to fulfilling the read/write purposes of metadata I need.
There have been some weird behaviours I've noticed while setting the keywords, like some get swallowed up into the file (the file size changed, I know they're there, but Windows doesn't acknowledge this), and only after manually deleting the keywords do they reappear suddenly. Very strange.