Mongodb
db.entity.find()
{
"_id" : ObjectId("5563a4c5567b3104c9ad2951"),
"section" : "section1",
"chapter" : "chapter1",
...
},
{
"_id" : ObjectId("5563a4c5567b3104c9ad2951"),
"section" : "section1",
"chapter" : "chapter2",
...
},....
In my database, the collections entity contain mainly section and chapter. Only chapter values are unique and when we query mongo for a given section it will return several results (one section matches with many chapters).
What I need to do is to get all collections of a given section, it's as simple as that.
settings.py
URL_PREFIX = "api"
API_VERSION = "v1"
ALLOWED_FILTERS = ['*']
schema = {
'section': {
'type': 'string',
},
'chapter': {
'type': 'string',
},
}
entity = {
'item_title': 'entity',
'resource_methods': ['GET'],
'cache_control': '',
'cache_expires': 0,
'url': 'entity/<regex("\w+"):section>/section',
'schema': schema
}
DOMAIN = {
'entity': entity,
}
run.py
from eve import Eve
if __name__ == '__main__':
app.run(debug=True)
What I tried
curl -L -X GET -H 'Content-Type: application/json' http://127.0.0.1:5000/api/v1/entity/23/section
OUTPUT
{
"_status": "ERR",
"_error": {
"message": "The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.",
"code": 404
}
}
What am I doing wrong? How can I get all entities of one section?
Did you try to write the url like this:
http://127.0.0.1:5000/api/v1/entity?where=section=='section32'
please have a look at this python-eve.org/features.html#filtering
Try with:
curl -H 'Accept: application/json' http://127.0.0.1:5000/api/v1/entity/section23/section
You want to use section23 in your url since that's what matches both the regex and the actual value stored on the database.
You might also want to change the url to:
'url': 'entity/<regex("\w+"):section>'
Which would allow for urls like this:
http://127.0.0.1:5000/api/v1/entity/section23
Which is probably more semantic. Hope this helps.
Related
recently my admin asked me if I can create an app to input some data into our Sonicwall.
I found some API, created an account, and ran some "get" methods, which worked just fine. I am trying now to post a new object into our pv4 rules and cannot get over it.
The problem is when I write something like this:
def postIpv4Object(session):
body = {
"address_objects": [{
"ipv4": {
"name": "Test 1",
"zone": "LAN",
"host": {
"ip": "192.168.168.10"
}
}
}
]
}
resp = session.post(fw + '/api/sonicos/address-objects/ipv4', headers=good_headers,params=body, verify=False)
I am still getting this error:
{'status': {'info': [{'code': 'E_INVALID_API_CALL',
'level': 'error',
'message': 'Expected a request body.'}],
'success': False}}
I am reading docs but cannot really figure this out. Does anyone tried it and help me out a little?
After a couple of days of trying, I figured that "body" should be converted into JSON type, and instead of "params I need to use "data".
I try to figure out if I can upload media files, using curl (or anything else really), to python-eve field that is inside a dict type.
The documentation, http://docs.python-eve.org/en/latest/features.html, only have the simple example of
accounts = {
...,
'pic': {'type': 'media'},
...
}
curl -F "pic=#profile.jpg"
But what about this one?
accounts = {
...,
'extras': {
'type': 'dict',
'schema': {
'pic': {'type': 'media'},
...
}
}
}
I've tried with
curl -F "extras.pic=#profile.jpg",
curl -F "extras:pic=#profile.jpg"
etc
I've also experimented with sub resources without luck.
I am about to accept that I'm not allowed to structure my schema as I wish, but if there is a way, I'd be very happy to know.
In scrapyrt's POST documentation we can pass a JSON request like this, but how do you access the meta data like category and item in start_requests?
{
"request": {
"meta": {
"category": "some category",
"item": {
"discovery_item_id": "999"
}
},
, "start_requests": true
},
"spider_name": "target.com_products"
}
Reference: https://scrapyrt.readthedocs.io/en/latest/api.html#id1
There is an unmerged PR in scrapyRT that adds support to pass extra parameters inthe POST request.
1) Patch resources.py file located in scrapyrt folder.
In my case was /usr/local/lib/python3.5/dist-packages/scrapyrt/resources.py
Replace with this code: https://github.com/gdelfresno/scrapyrt/commit/ee3be051ea647358a6bb297632d1ea277a6c02f8
2) Now your spider can access to the new parameters with self.param1
ScrapyRT curl example:
curl -XPOST -d '{
"spider_name":"quotes",
"start_requests": true,
"param1":"ok"}' "http://localhost:9080/crawl.json"
In your spider
def parse(self, response):
print(self.param1)
Regards
I have the following working curl
curl -v -i -H "Content-Type:multipart/form-data" -H "X-Authorization: 12345" -F "file0=#/path/to/image.jpg" -d 'jsonData={ "data": { "field1": 1, "field2": 2 } }' -X POST http://example.com/url/path
I am trying to make the exact same request in python and came up with the following
headers = {
'Content-Type': 'multipart/form-data',
'X-Authorization': '12345',
}
files = {
'file0': ('/path/to/image.jpg',
open('/path/to/image.jpg', 'rb')),
}
file_post = requests.post('http://example.com/url/path',
headers=headers,
files=files,
data={
"jsonData": {
"data": {
"field1": 1,
"field2": 2,
}
}
})
The problem is that I'm getting a different response for curl and for python. Are these requests not equivalent or should I be looking somewhere else?
You can't just pass in a Python dictionary without encoding it to JSON. Your curl post has JSON-encoded data for the jsonData form field, so your Python code needs to provide the same:
import json
headers = {'X-Authorization': '12345'}
files = {
'file0': ('/path/to/image.jpg',
open('/path/to/image.jpg', 'rb')),
}
file_post = requests.post(
'http://example.com/url/path',
headers=headers,
files=files,
data={
"jsonData": json.dumps({
"data": {
"field1": 1,
"field2": 2,
}
})
})
By passing in a dictionary to data, each value that is an iterable, is seen as a sequence of separate values (so {'data': ['foo', 'bar']} is translated to data=foo&data=bar). You passed in a dictionary for the jsonData key, which is an iterable and thus treated a a sequence of values. Ultimately you only posted jsonData=data, nothing else.
By using a (JSON encoded) single value, you tell requests to pass that value directly, as the value for the jsonData key.
Using the files keyword argument triggers the right Content-Type header as well, it doesn't need setting manually.
I can't figure out what's happening with my Preapproval HTTP POST request. I'm just trying to do a basic call to PayPal's Adaptive Payments API, the Preapproval call specifically. And the PayPal error 580001 "Invalid request" is not that helpful in itself.
Request Headers (based on my Sandbox's account credentials, which I changed to xxx):
{
'X-PAYPAL-REQUEST-DATA-FORMAT': 'JSON',
'X-PAYPAL-SECURITY-PASSWORD': 'xxx',
'X-PAYPAL-RESPONSE-DATA-FORMAT': 'JSON',
'X-PAYPAL-SECURITY-SIGNATURE': 'xxx',
'X-PAYPAL-SECURITY-USERID': 'xx',
'X-PAYPAL-APPLICATION-ID': 'APP-80W284485P519543T'
}
My request payload (HTTP POST, body encoded in JSON):
{
"requireInstantFundingSource": "TRUE",
"returnUrl": "http://www.google.com/?paypal=ok",
"maxTotalAmountOfAllPayments": 1002,
"maxNumberOfPaymentsPerPeriod": 1,
"endingDate": "2014-03-14T16:49:36+0000",
"requestEnvelope.errorLanguage": "en_US",
"clientDetails.applicationId": "XXX",
"cancelUrl": "http://www.google.com/paypal=cancel",
"startingDate": "2013-09-15T16:49:36+0000",
"feesPayer": "PRIMARYRECEIVER",
"currencyCode": "SEK"
}
The above POST body is posted to:
https://svcs.sandbox.paypal.com/AdaptivePayments/Preapproval
Response from Paypal ("prettified" for understanding):
{
"responseEnvelope": {
"ack": "Failure",
"timestamp": "2013-09-10T09:56:43.031-07:00",
"build": "6941298",
"correlationId": "26d55e6bfcaa0"
},
"error": [
{
"category": "Application",
"domain": "PLATFORM",
"severity": "Error",
"message": "Invalid request: {0}",
"subdomain": "Application",
"errorId": "580001"
}
]
}
Any feedback is appreciated.
OK fixed. How?
Fix #1
The arguments requestEnvelope.errorLanguage and clientDetails.applicationId need to be "JSONified" into objects on their own, such as:
"requestEnvelope": {
"errorLanguage": "en_US"
},
and
"clientDetails": {
"applicationId": "APP-XXXXXXXXXXXXX"
},
respectively.
Fix #2
Date formats; date format should be of the form 2014-03-15T20:14:38.007+00:00 and not 2014-03-14T20:14:38+0000 as I was passing. Note the milliseconds, and the timezone with the colon in the utc offset.
Next time an Invalid request comes up the parameters I'm passing will be the first thing to look at.