Develop Reference

Flask app. FileNotFoundError: [Errno 2] No such file or directory

I am trying to create my first Flask app where I am getting user data, doing a vlookup(merge) to add a few columns to the user, then doing a prediction on the whole dataframe. The problem is I cannot read my CSV data saved in the static folder so as to do the merge. Flask cannot see the csv files. Any help will be highly appreciated:
Here is my code snippet
app = Flask(__name__, template_folder='templates')
def predict():
if request.method == 'POST':
with open("static/region_map.csv", "r") as f1:
region_map = f1.read()
FileNotFoundError: [Errno 2] No such file or directory: 'static/region_map.csv'

The values for paths set during configuration can be requested. These are absolute and should work. The static directory can be queried using the static_folder attribute.
import os
def predict():
if request.method == 'POST':
target = os.path.join(app.static_folder, 'region_map.csv')
with open(target) as fp:
region_map = fp.read()
# ...
The open function opens the file and returns a file object. If you read the content in text mode using read, you will in fact get a string back.
To get a dataframe from a csv file, it is advisable to use pandas.read_csv.
import os
import pandas as pd
def predict():
target = os.path.join(app.static_folder, 'region_map.csv')
df = pd.read_csv(target)
# ...

How to import pickled python object into Django view.py file

This file runs perfectly well on its own:
import pickle
with open('model.obj', 'rb') as f:
model = pickle.load(f)
print(model)
But what I really want to do is to use model, which is a very large stored python dictionary, in my Django views.py file, like so:
from django.shortcuts import render
from modeler.load_model import model
def page(request):
return render(request, 'page.html', {'model':model})
However, when I try this, I get the error below:
File "C:\Users\mmm\PycharmProjects\MyProject\modeler\load_model.py", line 3, in <module>
with open('model.obj', 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'model.obj'
I don't understand why, because if I run the first file directly it works fine - the only extra step here is importing it into views.py (I tried including the entire load_model.py code within views.py at first, but got this error, so then tried it in this separate file and it worked on its own, so obviously the issue is that I don't know how to correctly load/import my python model objects or files within Django.

I found the answer! Apparently, all I had to do was to put the absolute path to 'model.obj' in my open statement, not just the relative path, like this:
with open('C:\\Users\\mmm\\PycharmProjects\\MyProject\\modeler\\model.obj', 'rb') as f:
model = pickle.load(f)

Reading json in pythonanywhere flask app

First I have seen this question. My problem is I have a flask app running on pythonanywhere that reads info from a json file in the same directory on the server, and get the following error:
Internal Server Error:The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application..
I simplified the app down to:
from flask import Flask
import json
app = Flask(__name__)
#app.route('/')
#app.route('/index')
def index():
return 'Index'
#app.route('/courses')
def courses():
with open('courses.json', 'r') as f:
these_courses = json.load(f)
return str(these_courses)
If I go to the index page I see index, as expected, but if I try to go to /courses then I get the error.The whole things runs fine on localhost, then with the same code I get an error on the server, so I know reading from the file works fine. This makes me think it might be a problem unique to json combined with pythonanywhere.
Edit: Perhaps a problem with the path name for courses.json, but it's in the same directory so I feel like it should be fine, just a thought

Turns out it was a pathname problem. I guess on files need to be routed from the root directory.
I ran:
def courses():
my_dir = os.path.dirname(__file__)
json_file_path = os.path.join(my_dir, 'courses.json')
return json_file_path
to find the path, then changed the function to:
def courses():
with open('/home/username/path/to/file/courses.json', 'r') as f:
these_courses = json.load(f)
return str(these_courses)
and now it worked :D
Then to make a better version that doesn't break when you move the project I did it like this:
def courses():
my_dir = os.path.dirname(__file__)
json_file_path = os.path.join(my_dir, 'courses.json')
with open(json_file_path, 'r') as f:
these_courses = json.load(f)
return str(these_courses)

As an alternative:
import pathlib
path = pathlib.Path('courses.json').absolute()
these_courses = json.loads(path.read_text())

How to use pyramid.response.FileIter

I have the following view code that attempts to "stream" a zipfile to the client for download:
import os
import zipfile
import tempfile
from pyramid.response import FileIter
def zipper(request):
_temp_path = request.registry.settings['_temp']
tmpfile = tempfile.NamedTemporaryFile('w', dir=_temp_path, delete=True)
tmpfile_path = tmpfile.name
## creating zipfile and adding files
z = zipfile.ZipFile(tmpfile_path, "w")
z.write('somefile1.txt')
z.write('somefile2.txt')
z.close()
## renaming the zipfile
new_zip_path = _temp_path + '/somefilegroup.zip'
os.rename(tmpfile_path, new_zip_path)
## re-opening the zipfile with new name
z = zipfile.ZipFile(new_zip_path, 'r')
response = FileIter(z.fp)
return response
However, this is the Response I get in the browser:
Could not convert return value of the view callable function newsite.static.zipper into a response object. The value returned was .
I suppose I am not using FileIter correctly.
UPDATE:
Since updating with Michael Merickel's suggestions, the FileIter function is working correctly. However, still lingering is a MIME type error that appears on the client (browser):
Resource interpreted as Document but transferred with MIME type application/zip: "http://newsite.local:6543/zipper?data=%7B%22ids%22%3A%5B6%2C7%5D%7D"
To better illustrate the issue, I have included a tiny .py and .pt file on Github: https://github.com/thapar/zipper-fix

FileIter is not a response object, just like your error message says. It is an iterable that can be used for the response body, that's it. Also the ZipFile can accept a file object, which is more useful here than a file path. Let's try writing into the tmpfile, then rewinding that file pointer back to the start, and using it to write out without doing any fancy renaming.
import os
import zipfile
import tempfile
from pyramid.response import FileIter
def zipper(request):
_temp_path = request.registry.settings['_temp']
fp = tempfile.NamedTemporaryFile('w+b', dir=_temp_path, delete=True)
## creating zipfile and adding files
z = zipfile.ZipFile(fp, "w")
z.write('somefile1.txt')
z.write('somefile2.txt')
z.close()
# rewind fp back to start of the file
fp.seek(0)
response = request.response
response.content_type = 'application/zip'
response.app_iter = FileIter(fp)
return response
I changed the mode on NamedTemporaryFile to 'w+b' as per the docs to allow the file to be written to and read from.

current Pyramid version has 2 convenience classes for this use case- FileResponse, FileIter. The snippet below will serve a static file. I ran this code - the downloaded file is named "download" like the view name. To change the file name and more set the Content-Disposition header or have a look at the arguments of pyramid.response.Response.
from pyramid.response import FileResponse
#view_config(name="download")
def zipper(request):
path = 'path_to_file'
return FileResponse(path, request) #passing request is required
docs:
http://docs.pylonsproject.org/projects/pyramid/en/latest/api/response.html#
hint: extract the Zip logic from the view if possible

How to assign a local file to the FileField in Django?

I was trying to assign a file from my disk to the FileField, but I have this error:
AttributeError: 'str' object has no attribute 'open'
My python code:
pdfImage = FileSaver()
pdfImage.myfile.save('new', open('mytest.pdf').read())
and my models.py
class FileSaver(models.Model):
myfile = models.FileField(upload_to="files/")
class Meta:
managed=False
Thank you in advance for your help

Django uses it's own file type (with a sightly enhanced functionality). Anyway Django's file type works like a decorator, so you can simply wrap it around existing file objects to meet the needs of the Django API.
from django.core.files import File
local_file = open('mytest.pdf')
djangofile = File(local_file)
pdfImage.myfile.save('new', djangofile)
local_file.close()
You can of course decorate the file on the fly by writing the following (one line less):
pdfImage.myfile.save('new', File(local_file))

If you don't want to open the file, you can also move the file to the media folder and directly set myfile.name with the relative path to MEDIA_ROOT :
import os
os.rename('mytest.pdf', '/media/files/mytest.pdf')
pdfImage = FileSaver()
pdfImage.myfile.name = '/files/mytest.pdf'
pdfImage.save()

if you are getting error like:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position...
then you have to open the file in binary mode: open("mytest.pdf", "rb")
full example:
from django.core.files import File
pdfImage = FileSaver()
pdfImage.myfile.save('new.pdf', File(open('mytest.pdf','rb')))

If you want to upload a local file in django, you can do this
from django.core.files import File
file = open(filepath, 'rb')
file_to_upload = File(file, name=f'{your_desired_name}')
file.close()
Now you can pass the file to django rest serializer to upload or whatever your use-case is. If passing this file to serializer, be sure to close it after it has been passed to serializer.
Note: The filepath here can be django's temporary file or any stored file.