I'm working on a python project in which I need to create a new JSON file.It's working locally but when I deploy my app to Heroku the file creation doesn't work.
Here's what I have tried:
From settings.py
APP_ROOT = os.path.dirname(os.path.abspath(__file__)) # refers to application_top
APP_FINALIZED = os.path.join(APP_ROOT, 'finalized')
From app.py
HOME = os.path.join(APP_FINALIZED)
print(HOME)
with open(HOME + '/description_' + str(fid) + '.json', 'w', encoding="utf-8")\
as f:
f.write(json.dumps(data, indent=4, ensure_ascii=False))
Updated: can we write this file directly to the S3 bucket, anyway?
it's working fine locally, but when I deploy it on Heroku the file doesn't create, even it doesn't show any error.
I'll add this as answer as well in case someone elese needs help.
Heroku's file system is (as far as I can remember) read-only.
Please check this answer.
Related
I have developed a Django app where I am uploading a file, doing some processing using a project folder name media.
Process:
user uploads a csv file, python code treats the csv data by creating temp folders in Media folder. After processing is complete, these temp folders are deleted and processed data is downloaded through browser.
I am using the below lines of code to make and delete temp file after processing
temp = 'media/temp3'
os.mkdir(temp)
shutil.copyfile('media/' + file_name, temp + '/' + file_name)
shutil.rmtree(temp, ignore_errors=True)
To set the media root, I used the below lines in settings.py which I am sure I am not using in other parts of the code.
MEDIA_ROOT = os.path.join(BASE_DIR, 'media/')
MEDIA_URL = "/media/"
Everything works fine when I run the app on local host. but as soon as i deployed it to heroku, it seems like these folders were not created/not found.
I am looking for:
Either a solution to create, read and delete folders/files in runtime using heroku,
or
a better way to manage files/folders in runtime.
I'm using Flask to make a web application and I want to upload a user input file to Google Storage Cloud. I'm using Heroku to host my web app and I don't know how to save files on Heroku's temporary storage so I'm trying to use tempfile to store the file in a directory and then access the directory to upload the file.
When I try to do that, I get this error: PermissionError: [Errno 13] Permission denied: 'C:\\Users\\[MyName]\\AppData\\Local\\Temp\\tmpbpom7ull'
Here is my code I'm working with, if anyone has any other way to upload a FileStorage object to the Google Storage cloud or a way to access the saved file, that would be very appreciated!
# File is currently a "FileStorage" object from werkzeug, gotten by doing
# file = request.files["filename"]
tempdir = tempfile.mkdtemp()
file.name = filename
file.save(tempdir)
upload_blob(BUCKET_NAME,filename,filename)
Following up on yesterday's Flask: Could not authenticate question the Google Cloud Storage client, you can use werkzeug's FileStorage object as described in the Flask-GoogleStorage usage:
Assuming you a have a file hellofreddie.txt in the working directory:
hellofreddie.txt:
Hello Freddie!
You can then open it, create a FileStorage object and then use the save on Bucket object (files):
from datetime import timedelta
from flask import Flask
from flask_googlestorage import GoogleStorage, Bucket
from werkzeug.datastructures import FileStorage
import os
files = Bucket("files")
storage = GoogleStorage(files)
app = Flask(__name__)
app.config.update(
GOOGLE_STORAGE_LOCAL_DEST = app.instance_path,
GOOGLE_STORAGE_SIGNATURE = {"expiration": timedelta(minutes=5)},
GOOGLE_STORAGE_FILES_BUCKET = os.getenv("BUCKET")
)
storage.init_app(app)
with app.app_context():
with open("hellofreddie.txt","rb") as f:
file = FileStorage(f)
filename = files.save(file)
After the code has run, you will see a UUID-named equivalent created in Cloud Storage.
You can use the storage browser or gsutil:
gsutil ls gs://${BUCKET}
gs://{BUCKET}/361ea9ea-5599-4ff2-84d1-3fe1a802ac08.txt
NOTE I was unable to resolve an issue trying to print either files.url(filename) or files.signed_url(filename). These methods correctly return the Cloud Storage Object but as PurePosixPath('f3745268-5c95-4c61-a892-09c0de556635.txt'). My Python naivete.
I've realized my error, I was trying to use file.save() to a folder and not to an actual file, my code has been updated to
tempdir = tempfile.mkdtemp()
file.name = filename
file.save(tempdir + "/" + filename)
upload_blob(BUCKET_NAME,tempdir + "/" + filename,filename)
Thank you to PermissionError: [Errno 13] Permission denied
This question already has an answer here:
Refering to a directory in a Flask app doesn't work unless the path is absolute
(1 answer)
Closed 5 years ago.
I'm trying to build a website using the Flask framework for Python.
I'm on a Linux Ubuntu server, with Apache2.
On my website, whenever someone enters the URL "/Elv_1.html", I want to open a .txt file, get some values and create a graph using pygal. Here is my code:
#app.route('/river_1.html')
def riv_1():
try:
document = open('temp.txt','r')
temp_list = []
for n in document:
n = n.rstrip('\n')
n = int(n)
temp_list.append(n)
document.close()
graf = pygal.Line(title=u'Tempt last 24h')
graf.x_labels = (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24)
graf.add('Temp', temp_list)
graf = graf.render_data_uri()
return render_template('river_1.html', graf=graf)
except Exception, e:
return str(e)
if __name__ == '__main__':
app.run(debug=True)
The file 'temp.txt' is located in the same directory as the __init__.py file. __init__.py is the Flask app that the code comes from.
When I do this on my computer using localhost to run the server, it works just fine. However, when I upload this to my Linux server and try to enter that specific URL, it shows the following error:
[Error 2] No such file or directory: 'temp.txt'
Any suggestions as to why it doesn't appear to find the file?
Try using the os module when specifying the path to your file. I am asuming you are using a windows pc when runing on localhost?
import os
document_path = os.getcwd()+'temp.txt'
document = open(documnet_path, 'r')
Make sure you are running the server from it's directory. So if you have this structure as shown below, you can't simply open terminal and type server/__init__.py, because you are in your home directory (/home/username/). You need to cd to server and there run ./__init__.py
/home/
username/
server/
__init__.py
temp.txt
Or, if you want to run it from somewhere else, run open the file from os.path.abspath(os.path.dirname(__file__)) + '/temp.txt') (Tested with python 3.5.2)
See python docs for os.path.
im have FLASK app with
www/FlaskApp/FlaskApp/init.py file with funtion
python file wut next contains
#app.route('/')
def hello():
file = open('myfile.txt', 'w+')
os.mknod("newfile.txt")
return render_template('page2.html')
but if im run site,its return error, in file log write
PermissionError: [Errno 13] Permission denied: 'myfile.txt'
im set permision 777 for all www directories
open FileZilla
right click on www dir, and set 777 permision
Why file dont create?
Not sure if this is an optimal solution, and I don't know enough about Flask as to tell you why the relative path isn't working (I would think that it would write the file where ever your python script was) but you could get it to work by using an environment variable to specify where to store your apps data. For instance:
import os
#app.route('/')
def open_file():
filename = os.path.join(os.environ['FLASK_APP_DATA'], 'myfile.txt')
print (filename)
file = open(filename, 'w+')
file.write("This is a test")
file.close()
Then you could have the environment variable set differently on your dev box and your prod box.
I'm trying to deploy a project to Google App Engine. The main HTML page I render is stored in the /documents/form.html directory of the project. If I run on my local host it finds the file no problem. When I deploy to GAE it gives the below error:
File "/base/data/home/apps/s~andesonchase/1.372703354720880550/main.py", line 4, in <module>
fileHandle = open("documents/form.html", "r")
IOError: [Errno 2] No such file or directory: 'documents/form.html'
I think I need to include it on my app.yaml but I'm not sure on the correct syntax.
I can list three options for you
A) as suggested by the previous poster is to add to app.yaml as either a static_files entry or by making documents a static_dir which would allow access to the files using raw http requests but completely bypassing your handlers in main.py
B) [probably the most kosha] is to access the file with the jinja2 template library as explained here which doesn't require you to add the files explicitly to app.yaml
C) or you could stick with whatever your doing inside main.py at the moment but modify your open statement as follows
import os.path
f = open(os.path.dirname(__file__) + '/documents/form.html')
as explained in this stackoverlflow answer since open works a little differently with appengine
If you want to serve it as a static file add it like this:
Add it to your app.yaml and replace /form with the url you please
- url: /form
static_files: documents/form.html
upload: documents/form.html
If you need to run a script then it's different.