AppEngine throws the following error:
IOError: [Errno 13] file not accessible: '/home/username/code/appname/csv/master.csv'
The relevant part of the script looks like this:
project_dir = os.path.dirname(__file__)
csv_data = csv.DictReader(open(project_dir+'master.csv','rU'))
The relevant part of the app.yaml looks like this:
handlers:
- url: /csv
static_dir: csv
I get the same error when deleting the handler.
FYI: I do not get the IO error when putting the csv file in the top directory of my app. I need the handler because javascript on my website is sending a get request to the csv file and this does not work for the top-level directory (why?). I could have the csv in the top directory AND the csv directory at the same time but I think there could be a cleaner solution.
Any ideas?
Update your app.yaml static handler to be application_readable.
https://developers.google.com/appengine/docs/python/config/appconfig
application_readable
Optional. By default, files declared in static file handlers are uploaded as static data and are only served to end users, they cannot be read by an application.
Related
Issue: Unable to save file in directory (/root/Notion/Image) when using Cron schedule
This is what my code is trying to do:
Check email
Download image attachment
Store in a directory - root/Notion/Image
Retrieve file path
The script is working when I run it manually in Google Cloud terminal. The problem is when I try to schedule it on Cron, it's unable to access the folder to save the file locally.
This is the error when the script failed and require permission:
Traceback (most recent call last):
File "Notion/test.py", line 121, in <module>
path = get_attachments(email.message_from_bytes(msg[0][1]))
File "Notion/test.py", line 47, in get_attachments
with open(filePath, 'wb') as f:
PermissionError: [Errno 13] Permission denied: '/root/Notion/Image/3.jpeg'
This is the code to retrieve attachment from email
def get_attachments(msg):
for part in msg.walk():
if part.get_content_maintype()=='multipart':
continue
if part.get('Content-Disposition') is None:
continue
fileName = part.get_filename()
if bool(fileName):
filePath = os.path.join(attachment_dir, fileName)
with open(filePath, 'wb') as f:
f.write(part.get_payload(decode=True))
return str(filePath)
Resolved:
The problem is that I shouldn't use root directory since it requires permission. I've changed it to home directory instead.
attachment_dir = '/home/dev_thomas_yang/folder_name/folder_name'
For people who needs to check their home direction, simply run this script.
from pathlib import Path
home= str(Path.home())
print(home)
Thanks Triplee for the patience to breakdown my issue despite my sloppy ways of presenting it!
The easiest fix hands down is to change the code so it doesn't try to write to /root. Have it write to the invoking user's home directory instead.
Your question doesn't show the relevant parts of the code, but just change attachment_dir so it's not an absolute path. Maybe separately take care of creating the directory if it doesn't already exist.
import pathlib
# ...
attachment_dir = pathlib.Path("cron/whatever/attachments").mkdir(parents=True, exist_ok=True)
# ...
for loop in circumstances:
get_attachments(something)
A better design altogether would be to have get_attachments accept the directory name as a parameter, so you can make this configurable from the code which calls it. Global variables are a nuisance and cause hard-to-debug problems because they hide information which is important for understanding the code, and tricky to change when you try to debug that code and don't know which parts of the code depend on the old value.
This question already has answers here:
Refering to a directory in a Flask app doesn't work unless the path is absolute
(1 answer)
Saving upload in Flask only saves to project root
(1 answer)
Get root path of Flask application
(2 answers)
Closed 4 years ago.
There are 2 functions or views located in the same file version.py, one for upload the other for downloading that file. The issue is that the send_file function keeps changing directories while opening the file.
Code 1:
File Upload function:
binfile = form.binfile.data
filename = secure_filename("arduino.bin")
#without the app folder
binfile.save(os.path.join(
app.config['UPLOAD_FOLDER'],
device_names.farm_id,
device_names.mqtt_id,
filename))
flash('File Uploaded', 'success')
File Download Function:
file_path = os.path.join(
app.config['UPLOAD_FOLDER'],
farm_id,
device_id,
'arduino.bin')
print(file_path)
response = make_response(send_file(
file_path,
mimetype='application/octet-stream',
as_attachment=True
))
response.headers['x-MD5'] = md5(file_path)
print(response.headers)
return response, 200
Here the file gets uploaded to the project root folder.
When I send a request for this file, I get this error:
[Errno 2] No such file or directory:
‘/home/maxwell/Desktop/python/aquaponics-monitor/app/firmware-manager/FARM0/node2/arduino.bin’
When I manually move the file to the 'app' folder I get no errors and a 200 response code while requesting. So I assume that the function is looking for the file inside the 'app' folder. I changed the upload location by adding 'app' to the path while uploading.
Code 2:
File Upload Function:
binfile = form.binfile.data
filename = secure_filename("arduino.bin")
#app folder included
binfile.save(os.path.join(
'app',
app.config['UPLOAD_FOLDER'],
device_names.farm_id,
device_names.mqtt_id,
filename))
flash('File Uploaded', 'success')
The file download function is the same as code 1. For this code I get this error:
[Errno 2] No such file or directory:
‘firmware-manager/FARM0/node2/arduino.bin’
The file download function now looks in the project root folder. I manually moved the file to the root folder this time and I again get a status code 200 and no errors when I send a request for it.
The only code change I made in the entire application is adding 'app' to the path in the File Upload function. Why is this happening?
I'm trying to read a .json file from within my Flask application using:
def renderblog():
with open(url_for("static", filename="blogs.json")) as blog_file:
data = json.load(blog_file)
However I get the error:
FileNotFoundError: [Errno 2] No such file or directory: '/static/blogs.json'
Now I know for a fact that the directory exists within my project structure, but I have no idea why I'm getting this error. Any ideas? Is there a specific way to retrieve .json in Flask?
You generated a URL path, not a path to the local static folder. Use the app.static_folder attribute instead:
def renderblog():
filename = os.path.join(app.static_folder, 'blogs.json')
with open(filename) as blog_file:
data = json.load(blog_file)
I have a static directory that contains some resources (data files), and I can access the files directly: i.e., http://hello.com/static/dir/abc.pdf. However, I got error with the directory as the address: i.e., http://hello.com/static/dir.
Using flask, I can solve this issue by showing the contents of the directory.
#app.route('/static/<path:url>') #protect
def show(url):
content_dir = app.config['CONTENT_DIR']
directory = "%s/%s/box/%s" % (content_dir, url, url2)
result = []
if os.path.isdir(directory):
for file in os.listdir(directory):
content['url'] = '/static/...
result.append(content)
return render_template("box.html",...)
The issue is that with this route processing, the direct file accessing doesn't work any more as http://hello.com/static/dir/abc.pdf always triggers the show() method.
How can I redirect to the resource file (abc.pdf in the example), without being redirected to the show() method?
In your custom static route, check if the path is a file or a directory. If it's a file, serve it, otherwise show the directory index.
import os
from flask import send_file
path = os.path.join(app.config['CONTENT_DIR'], url, 'box', url2)
if os.path.isfile(path):
return send_file(path)
return render_template('box.html', ...)
Of course, since the path is specified by the url sent by the user, you should check that it's safe first.
I'm trying to deploy a project to Google App Engine. The main HTML page I render is stored in the /documents/form.html directory of the project. If I run on my local host it finds the file no problem. When I deploy to GAE it gives the below error:
File "/base/data/home/apps/s~andesonchase/1.372703354720880550/main.py", line 4, in <module>
fileHandle = open("documents/form.html", "r")
IOError: [Errno 2] No such file or directory: 'documents/form.html'
I think I need to include it on my app.yaml but I'm not sure on the correct syntax.
I can list three options for you
A) as suggested by the previous poster is to add to app.yaml as either a static_files entry or by making documents a static_dir which would allow access to the files using raw http requests but completely bypassing your handlers in main.py
B) [probably the most kosha] is to access the file with the jinja2 template library as explained here which doesn't require you to add the files explicitly to app.yaml
C) or you could stick with whatever your doing inside main.py at the moment but modify your open statement as follows
import os.path
f = open(os.path.dirname(__file__) + '/documents/form.html')
as explained in this stackoverlflow answer since open works a little differently with appengine
If you want to serve it as a static file add it like this:
Add it to your app.yaml and replace /form with the url you please
- url: /form
static_files: documents/form.html
upload: documents/form.html
If you need to run a script then it's different.