I'm creating a custom middleware to django edit response object to act as a censor. I would like to find a way to do a kind of search and replace, replacing all instances of some word with one that I choose.
I've created my middleware object, added it to my MIDDLEWARE_CLASSES in settings and have it set up to process the response. But so far, I've only found methods to add/edit cookies, set/delete dictionary items, or write to the end of the html:
class CensorWare(object):
def process_response(self, request, response):
"""
Directly edit response object here, searching for and replacing terms
in the html.
"""
return response
Thanks in advance.
You can simply modify the response.content string:
response.content = response.content.replace("BAD", "GOOD")
Perhaps my reply little better. When you try to make response.content.replace("BAD", "GOOD"), you will get error, that you cannot do it with strings, because response.content is byte array. I've added syntactic strings 'gen_duration_time_777' and 'server_time_777' to base template. And this works for me.
import time
from datetime import datetime
class StatsMiddleware(object):
duration = 0
def process_request(self, request):
# Store the start time when the request comes in.
request.start_time = time.time()
def process_response(self, request, response):
# Calculate and output the page generation duration
# Get the start time from the request and calculate how long
# the response took.
self.duration = time.time() - request.start_time
response["x-server-time"] = datetime.now().strftime("%d/%m/%Y %H:%M")
response.content = response.content.replace(b"server_time_777", str.encode(response["x-server-time"]))
response["x-page-generation-duration-ms"] = '{:.3f}'.format(self.duration)
response.content = response.content.replace(b"gen_duration_time_777", str.encode(response["x-page-generation-duration-ms"]))
return response
Related
the django views.py
def quiz_automate(request):
val=automate_data()
# print(val['data'])
get_quiz=Quiz.objects.get(subject='english',exam_type='jamb')
for data in val['data']:
print(data['question'])
d_question=Question(text=data['question'],quiz=get_quiz)
d_question.save()
return HttpResponse('saved')`
the scraped.py function
def automate_data():
url = 'https://questions.aloc.com.ng/api/v2/m?subject=chemistry'
headers = {'Accept': 'application/json','Content-Type': 'application/json'}
r = requests.get(url, headers=headers)
return r.json()
i tried scraping data from a site and it returned multipled values but whenever i use in my django views to store in my postgres database it just store a value instead of multiple values
Your return statement is tabbed inside of the for loop. So when it hits that line, it will exit the function, thus only saving the record in the first loop.
Hope that helps.
I'm trying to change a bit of Python code at the moment that is being used as a Lambda function. Currently, the code in question takes a number of parameters that are URLs, and uses the urllib2 library to get the images at those URLs (for example, a URL could be: https://www.sundanceresort.com/wp-content/uploads/2016/08/nature-walk-1600x1400-c-center.jpg).
I'd like to change this so that the code will handle a POST request that has the image in its body. Looking at some tutorials I thought Flask request might do the trick but I’m confused as to how it would work in this particular case.
At the moment, the relevant code that would be replaced is in three sections:
urls = urls_str.split(",")
results = []
for url in urls:
headers = {"User-Agent": "Mozilla/5.0"}
try:
req = urllib2.Request(url, None, headers)
image = urllib2.urlopen(req).read()
...
def get_param_from_url(event, param_name):
params = event['queryStringParameters']
return params[param_name]
...
def predict(event, context):
try:
param = get_param_from_url(event, 'url')
result = model.predict(param)
The overall code is here.
Any help or suggestions would be greatly appreciated.
I'm looking to export all orders from the WooCommerce API via a python script.
I've followed the
authentication process
and I have been using method to obtain orders described
here. My code looks like the following:
wcapi = API(
url = "url",
consumer_key = consumerkey,
consumer_secret = consumersecret
)
r = wcapi.get('orders')
r = r.json()
r = r['orders']
print(len(r)) # output: 8
This outputs the most recent 8 orders, but I would like to access all of them. There are over 200 orders placed via woocommerce right now. How do I access all of the orders?
Please tell me there is something simple I am missing.
My ultimate goal is to pull these orders automatically, transform them, and then upload to a visualization tool. All input is appreciated.
First: Initialize your API (as you did).
wcapi = API(
url=eshop.url,
consumer_key=eshop.consumer_key,
consumer_secret=eshop.consumer_secret,
wp_api=True,
version="wc/v2",
query_string_auth=True,
verify_ssl = True,
timeout=10
)
Second: Fetch the orders from your request(as you did).
r=wcapi.get("orders")
Third: Fetch the total pages.
total_pages = int(r.headers['X-WP-TotalPages'])
Forth: For every page catch the json and access the data through the API.
for i in range(1,total_pages+1):
r=wcapi.get("orders?&page="+str(i)).json()
...
The relevant parameters found in the corresponding documentation are page and per_page. The per_page parameter defines how many orders should be retrieved at every request. The page parameter defines the current page of the order collection.
For example, the request sent by wcapi.get('orders/per_page=5&page=2') will return orders 5 to 10.
However, as the default of per_page is 10, it is not clear as to why you get only 8 orders.
I encountered the same problem with paginated response for products.
I built on the same approach described by #gtopal, whereby the X-WP-TotalPages header returned by WooCommerce is used to iterate through each page of results.
I knew that I would probably encounter the same issue for other WooCommerce API requests (such as orders), and I didn't want to have to confuse my code by repeatedly performing a loop when I requested a paginated set of results.
To avoid this I used a decorator to abstract the pagination logic, so that get_all_wc_orders can focus just on the request.
I hope the decorator below might be useful to someone else (gist)
from woocommerce import API
WC_MAX_API_RESULT_COUNT = 100
wcapi = API(
url=url,
consumer_key=key,
consumer_secret=secret,
version="wc/v3",
timeout=300,
)
def wcapi_aggregate_paginated_response(func):
"""
Decorator that repeat calls a decorated function to get
all pages of WooCommerce API response.
Combines the response data into a single list.
Decorated function must accept parameters:
- wcapi object
- page number
"""
def wrapper(wcapi, page=0, *args, **kwargs):
items = []
page = 0
num_pages = WC_MAX_API_RESULT_COUNT
while page < num_pages:
page += 1
log.debug(f"{page=}")
response = func(wcapi, page=page, *args, **kwargs)
items.extend(response.json())
num_pages = int(response.headers["X-WP-TotalPages"])
num_products = int(response.headers["X-WP-Total"])
log.debug(f"{num_products=}, {len(items)=}")
return items
return wrapper
#wcapi_aggregate_paginated_response
def get_all_wc_orders(wcapi, page=1):
"""
Query WooCommerce rest api for all products
"""
response = wcapi.get(
"orders",
params={
"per_page": WC_MAX_API_RESULT_COUNT,
"page": page,
},
)
response.raise_for_status()
return response
orders = get_all_wc_orders(wcapi)
Here's a simple Flask-Restful resource:
class ListStuff(Resource):
def get(self):
stuff = SomeFunctionToFetchStuff()
if re.match('/api/', request.path):
return {'stuff': stuff}
return make_response("{}".format(stuff), 200, {'Content-Type': 'text/html'})
api.add_resource(ListStuff, '/list', '/api/list', endpoint='list')
My idea is to allow users to call both /list and /api/list. If they use the first URL, they will get back an HTML representation of the data. If they use the second URL, they will get a JSON representation.
My trouble is when I want to access the URL of this endpoint elsewhere in the program. I can't just use url_for('list'), because that will always return /list , no matter whether the user accessed http://host.example.com/list or http://host.example.com/api/list
So how can I build the URL for /api/list ?
Looks like Hassan was on the right track - I can add a new resource for the same class but give it a different endpoint.
api.add_resource(ListStuff, '/list', endpoint='list')
api.add_resource(ListStuff, '/api/list', endpoint='api-list')
>>> print('URL for "list" is "{}"'.format(url_for('list'))
>>> print('URL for "api-list" is "{}"'.format(url_for('api-list'))
URL for "list" is "/list"
URL for "api-list" is "/api/list"
I'm designing a REST API, and I have an endpoint with a relatively flexible input.
Basically, it would be ideal to have a 48x48 array, but so long as it is an array, we can resize it to the correct size in a relatively intelligent way.
The resize operation isn't very costly, but I feel like the user should know that whatever input is being given is non-ideal, but I want this error message to be noninvasive.
I think this should still have an HTTP code of 200, but I could be persuaded otherwise.
Is there any accepted way of including metadata with a REST response?
I haven't found anything like this, but I feel like it can't be that strange a request.
For reference, using flask, and example code is below:
class Function(MethodView):
def post(self):
post_array = np.array(json.loads(request.form['data']))
if post_array.shape != (48, 48):
post_array = post_array.resize((48,48)) # Add some warning
return process(post_array)
As Jonathon Reinhart points out, including the warning as part of the response would be ideal. It should return 200 status for sure.
from flask import Flask, request, jsonify # jsonify is helpful for ensuring json in response
class Function(MethodView):
def post(self):
# create a response body dict with an empty warnings list
response_body = {
'warnings': [],
'process_result': None
}
# create a guard in case POST request body is not JSON
if request.json:
post_array = np.array(json.loads(request.form['data']))
if post_array.shape != (48, 48):
post_array = post_array.resize((48,48))
# add a warning to the response body
warning = {'data_shape_warning': 'the supplied data was not 48 x 48'}
response_body['warnings'].append(warning)
# compute the process and add the result to the response body
result = process(post_array)
response_body['process_result'] = result
return response_body, 200
# action to take if the POST body is not JSON
else:
warning = {'data_format_warning': 'POST request data was not JSON'}
response_body['warnings'].append(warning)
return jsonify(response_body), 200