Develop Reference

How to decode the file uploaded in postman using python starlette

I have uploaded a .mp4 file in postman and now i need to read the file contents into a variable in python
This is my postman upload
This is my python file where i my request will be catched in this function
now how will be decode the file uploaded in this function
from starlette.requests import Request
from starlette.routing import Route, Mount
from starlette.applications import Starlette
from starlette.responses import PlainTextResponse, FileResponse
class Post:
async def PostValuesToDb(requests: Request):
return PlainTextResponse("true")
routes = [
Route("/post", Post.PostValuesToDb,
name="PostValuesToDb", methods=["POST"]),
]
app = Starlette(
routes=routes
)
PostValuesToDb(requests: Request) captures the postman request.

as per the documentation
for the time being i am choosing to save the contents in a local directory, you can do as you wish from the contents variable
from starlette.requests import Request
from starlette.routing import Route, Mount
from starlette.applications import Starlette
from starlette.responses import PlainTextResponse, FileResponse
class Post:
async def PostValuesToDb(requests: Request):
form = await requests.form()
# print(dir(form)) # check what methods and attrs are available
# print(form.multi_items) # check all form fields submitted
filename = form["image"].filename # my particular form field
contents = await form["image"].read()
with open(f'./data/{filename}', 'wb') as uploaded_file:
uploaded_file.write(contents)
return PlainTextResponse("true")
routes = [
Route("/post", Post.PostValuesToDb,
name="PostValuesToDb", methods=["POST"]),
]
app = Starlette(
routes=routes
)

Having Trouble Making a RESTful API with Flask-RestX: "No operations defined in spec!" and "404"s

In summary, I have been following the flask restx tutorials to make an api, however none of my endpoints appear on the swagger page ("No operations defined in spec!") and I just get 404 whenever I call them
I created my api mainly following this https://flask-restx.readthedocs.io/en/latest/scaling.html
I'm using python 3.8.3 for reference.
A cut down example of what I'm doing is as follows.
My question in short is, what am I missing?
Currently drawing blank on why this doesn't work.
Directory Structure
project/
- __init__.py
- views/
- __init__.py
- test.py
manage.py
requirements.txt
File Contents
requirements.txt
Flask-RESTX==0.2.0
Flask-Script==2.0.6
manage.py
from flask_script import Manager
from project import app
manager = Manager(app)
if __name__ == '__main__':
manager.run()
project/init.py
from flask import Flask
from project.views import api
app = Flask(__name__)
api.init_app(app)
project/views/init.py
from flask_restx import Api, Namespace, fields
api = Api(
title='TEST API',
version='1.0',
description='Testing Flask-RestX API.'
)
# Namespaces
ns_test = Namespace('test', description='a test namespace')
# Models
custom_greeting_model = ns_test.model('Custom', {
'greeting': fields.String(required=True),
})
# Add namespaces
api.add_namespace(ns_test)
project/views/test.py
from flask_restx import Resource
from project.views import ns_test, custom_greeting_model
custom_greetings = list()
#ns_test.route('/')
class Hello(Resource):
#ns_test.doc('say_hello')
def get(self):
return 'hello', 200
#ns_test.route('/custom')
class Custom(Resource):
#ns_test.doc('custom_hello')
#ns_test.expect(custom_greeting_model)
#ns_test.marshal_with(custom_greeting_model)
def post(self, **kwargs):
custom_greetings.append(greeting)
pos = len(custom_greetings) - 1
return [{'id': pos, 'greeting': greeting}], 200
How I'm Testing & What I Expect
So going to the swagger page, I expect the 2 endpoints defined to be there, but I just see the aforementioned error.
Just using Ipython in a shell, I've tried to following calls using requests and just get back 404s.
import json
import requests as r
base_url = 'http://127.0.0.1:5000/'
response = r.get(base_url + 'api/test')
response
response = r.get(base_url + 'api/test/')
response
data = json.dumps({'greeting': 'hi'})
response = r.post(base_url + 'test/custom', data=data)
response
data = json.dumps({'greeting': 'hi'})
response = r.post(base_url + 'test/custom/', data=data)
response

TL;DR
I made a few mistakes in my code and test:
Registering api before declaring the routes.
Making a wierd assumption about how the arguments would be passed to the post method.
Using a model instead of request parser in the expect decorator
Calling the endpoints in my testing with an erroneous api/ prefix.
In Full
I believe it's because I registered the namespace on the api before declaring any routes.
My understanding is when the api is registered on the app, the swagger documentation and routes on the app are setup at that point. Thus any routes defined on the api after this are not recognised. I think this because when I declared the namespace in the views/test.py file (also the model to avoid circular referencing between this file and views/__init__.py), the swagger documentation had the routes defined and my tests worked (after I corrected them).
There were some more mistakes in my app and my tests, which were
Further Mistake 1
In my app, in the views/test.py file, I made a silly assumption that a variable would be made of the expected parameter (that I would just magically have greeting as some non-local variable). Looking at the documentation, I learnt about the RequestParser, and that I needed to declare one like so
from flask_restx import reqparse
# Parser
custom_greeting_parser = reqparse.RequestParser()
custom_greeting_parser.add_argument('greeting', required=True, location='json')
and use this in the expect decorator. I could then retrieve a dictionary of the parameters in my post method. with the below
...
def post(self):
args = custom_greeting_parser.parse_args()
greeting = args['greeting']
...
The **kwargs turned out to be unnecessary.
Further Mistake 2
In my tests, I was calling the endpoint api/test, which was incorrect, it was just test. The corrected test for this endpoint is
Corrected test for test endpoint
import json
import requests as r
base_url = 'http://127.0.0.1:5000/'
response = r.get(base_url + 'test')
print(response)
print(json.loads(response.content.decode()))
Further Mistake 3
The test for the other endpoint, the post, I needed to include a header declaring the content type so that the parser would "see" the parameters, because I had specified the location explictily as json. Corrected test below.
Corrected test for test/custom endpoint
import json
import requests as r
base_url = 'http://127.0.0.1:5000/'
data = json.dumps({'greeting': 'hi'})
headers = {'content-type': 'application/json'}
response = r.post(base_url + 'test/custom', data=data, headers=headers)
print(response)
print(json.loads(response.content.decode()))
Corrected Code
For the files with incorrect code.
views/init.py
from flask_restx import Api
from project.views.test import ns_test
api = Api(
title='TEST API',
version='1.0',
description='Testing Flask-RestX API.'
)
# Add namespaces
api.add_namespace(ns_test)
views/test.py
from flask_restx import Resource, Namespace, fields, reqparse
# Namespace
ns_test = Namespace('test', description='a test namespace')
# Models
custom_greeting_model = ns_test.model('Custom', {
'greeting': fields.String(required=True),
'id': fields.Integer(required=True),
})
# Parser
custom_greeting_parser = reqparse.RequestParser()
custom_greeting_parser.add_argument('greeting', required=True, location='json')
custom_greetings = list()
#ns_test.route('/')
class Hello(Resource):
#ns_test.doc('say_hello')
def get(self):
return 'hello', 200
#ns_test.route('/custom')
class Custom(Resource):
#ns_test.doc('custom_hello')
#ns_test.expect(custom_greeting_parser)
#ns_test.marshal_with(custom_greeting_model)
def post(self):
args = custom_greeting_parser.parse_args()
greeting = args['greeting']
custom_greetings.append(greeting)
pos = len(custom_greetings) - 1
return [{'id': pos, 'greeting': greeting}], 200

In Django how to mock an object method called by views.py during its import?

I am writing System Tests for my Django app, where I test the complete application via HTTP requests and mock its external dependencies' APIs.
In views.py I have something like:
from external_service import ExternalService
externalService = ExternalService
data = externalService.get_data()
#crsf_exempt
def endpoint(request):
do_something()
What I want is to mock (or stub) ExternalService to return a predefined response when its method get_data() is called.
The problem is that when I run python manage.py test, views.py is loaded before my test class. So when I patch the object with a mocked one, the function get_data() was already called.
This solution didn't work either.

First off, don't call your method at import time. That can't be necessary, surely?
If get_data does something like a get request, e.g.
def get_data():
response = requests.get(DATA_URL)
if response.ok:
return response
else:
return None
Then you can mock it;
from unittest.mock import Mock, patch
from nose.tools import assert_is_none, assert_list_equal
from external_service import ExternalService
#patch('external_service.requests.get')
def test_getting_data(mock_get):
data = [{
'content': 'Response data'
}]
mock_get.return_value = Mock(ok=True)
mock_get.return_value.json.return_value = data
response = ExternalService.get_data()
assert_list_equal(response.json(), data)
#patch('external_service.requests.get')
def test_getting_data_error(mock_get):
mock_get.return_value.ok = False
response = ExternalService.get_data()
assert_is_none(response)
For this you'll need pip install nose if you don't already have it.

send, receive, save local image file with requests / django rest framework

I'd like to http-send (e.g. requests.post) image files from servers/clients (with the requests lib) and receive/save these files with a django rest framework app. How do I do this?
Secondly I'd like to know how to extract parts from a QueryDict sent by requests.post in general. And more specific: How do I parse and save the _io-Object from this set of data?
# sending app
file = "path/to/image.jpg"
data = open(file, 'rb')
files = {
'json': (None, crawlingResultConnectionsJson, 'application/json'),
'file': (os.path.basename(file), open(file, 'rb'), 'application/octet-stream')
}
url = "http://127.0.0.1:8000/result/" # receiving/saving django rest framework app
r = requests.post(url, files=files)
I've tried quite a while now. Your help would be much appreciated! Thanks!

I came to a solution that perfectly fits my needs. As I only found contributions for either the sending OR the receiving part, I'll try to put everything together here.
Due to more flexibility my approach is to transfer json and images in seperated requests. The two following apps are completely independent.
The sending side does as follows (app with no server needed):
from django.core.serializers.json import DjangoJSONEncoder
import requests # http://docs.python-requests.org/en/master/
import datetime # in case...
import json
### send image stuff ###
urlImages = "http://127.0.0.1:8000/receive-images/"
file = "C:\\path\\to\\filename.jpg" # "\\" windows machine...
# this will probably run in a loop or so to process a bunch of images
with open(file, 'rb') as f:
filename = "filename.jpg"
files = {'file': (filename, f)}
r = requests.post(urlImages, files=files)
print(r) # some logging here
### send data stuff ###
data = data # python data - lists, dicts, whatever
json = json.dumps(data, cls=DjangoJSONEncoder) # DjangoJSONEncoder handles datetime fields
urlData = "http://127.0.0.1:8000/receive-data/"
headers = {'content-type': 'application/json'}
r = requests.post(urlData, json, headers=headers)
print(r) # some logging here
The receiving side needs to run a server (built-in django server for dev, apache with the WSGInterface in production) and it has this chum installed: http://www.django-rest-framework.org/
Finally we have two views to handle the requests:
from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status
from .api_controller import ApiController
from django.core.files.storage import default_storage
class ReceiveImages(APIView): # make sure to nail down corresponding url-confs
def post(self, request, format=None):
file = request.data.get('file')
filename = str(file)
with default_storage.open('images/' + filename, 'wb+') as destination:
for chunk in file.chunks():
destination.write(chunk)
return Response("ok", status=status.HTTP_200_OK)
class ReceiveData(APIView): # make sure to nail down corresponding url-confs
def post(self, request, format=None):
json = request.data
ApiController().createDataIfNotExists(json)
# As my json is quite complex,
# I've sourced out the DB-interactions to a controller-like class (coming
# from PHP symfony :)) with heavy use of the great
# MyModel.objects.get_or_create() method. Also the strptime() method is used
# to convert the datetime fields. This could also go here...
return Response("ok", status=status.HTTP_200_OK)
using chunk() in respect to https://stackoverflow.com/a/30195605/6522103
Please (!) comment/answer if you don't agree or think that this could be improved. Thanks!!

Django Rest Framework Test case issue: 'HttpResponseNotAllowed' object has no attribute 'data'

My tests file code:
from rest_framework import status
from rest_framework.test import APITestCase
class CategoryTests(APITestCase):
def test_create_create(self):
url = '/category/add/'
data = {"name":"Sports","description":"get live updates here"}
response = self.client.post(url, data, format='json')
self.assertEqual(response.data, data)
Error which I am getting:
Traceback (most recent call last):
File "/Users/test/webapp/apps/core/tests.py", line 16, in test_create_create
self.assertEqual(response.data, data)
AttributeError: 'HttpResponseNotAllowed' object has no attribute 'data'
Infact the tests are not even calling the exact api statements(I checked that using debug statements in api code). Please let me know what may be going wrong or you need any more information on this.

Try using the DRF extended test client:
from rest_framework import status
from rest_framework.test import APITestCase, APIClient
class CategoryTests(APITestCase):
client = APIClient()
def test_create_create(self):
url = '/category/add/'
data = {"name":"Sports","description":"get live updates here"}
response = self.client.post(url, data, format='json')
self.assertEquals(response.data, data)

Issue was with url, I correct it and it worked. So my url was actually
url = '/v1.0/category/add/'