I am using Python Eve, which is awesome, however I ran into a problem and not sure if there is a solution.
I have a 'fields' dict in this schema:
'profiles': {
'fields': {
'type': 'dict',
'default': {}
}
}
I'd like to be able to PATCH update the 'fields' field, but the issue is that a PATCH request will never REMOVE any field inside 'fields', but I cannot use a PUT command or else all my other profile fields (not shown above) will disappear.
I tried using a subresource like this:
'profile-fields': {
'schema': {
'fields': {
'type': 'dict',
'default': {}
}
},
'datasource': {
'source': 'profiles',
'projection': { 'fields': 1 }
}
},
but as the Python Eve documentation states:
Please note that POST and PATCH methods will still allow the whole schema to be manipulated
http://python-eve.org/config.html#multiple-api-endpoints-one-datasource
Anyone know of a way to do this?
For Example:
# Create a record
POST /api/profiles
{
'name': 'Test',
'fields': {
'one': 1,
'two': 2
}
}
# => { _created: 'blah', _id: '123456' }
# then update fields with a PATCH request
PATCH /api/profiles/123456
{
'fields': {
'three': 3,
'four': 4
}
}
# then get the updated record
GET /api/profiles/123456
# RESPONSE
{
'_id': '123456',
'name': 'Test',
'fields': {
'one': 1,
'two': 2,
'three': 3,
'four': 4
}
}
I have just conceded to using a PUT request and sending the entire object back again, which is ok I guess, just thought there might be a way to do this.
Related
OK, I'm sure there's something wrong with this jsonschema, but I just can't seem to wrap my head around the problem.
I'm not going to post the actual code, but a minimal example that reproduces the issue.
Here's what I had before, which worked fine:
person = {
'type': 'object',
'properties': {
'name': {
'type': 'string'
}
'auth_token': {
'type': 'string',
},
'username': {
'type': 'string'
},
'password': {
'type': 'string'
}
},
'oneOf': [
{
'required': ['auth_token']
},
{
'required': ['username', 'password']
}
],
'required': ['name']
}
The idea here was that you always need to provide the name of the person, and then either an auth token or a username and password pair. As I said above, this validation worked fine, since we have parametrized tests that send all posible combinations of invalid JSON and evaluate the resulting error message, and those tests pass.
But then a new requirement came in and I needed to add a second mutually exclusive required pair of fields, which I did in this way:
person = {
'type': 'object',
'properties': {
'name': {
'type': 'string'
}
'auth_token': {
'type': 'string',
},
'username': {
'type': 'string'
},
'password': {
'type': 'string'
},
'project_id': {
'type': 'number'
},
'contract_date_from': {
'type': 'string'
}
'contract_date_to': {
'type': 'string'
}
},
'allOf': [
{
'oneOf': [
{
'required': ['auth_token']
},
{
'required': ['username', 'password']
}
]
},
{
'oneOf': [
{
'required': ['project_id']
},
{
'required': ['contract_date_from', 'contract_date_to']
}
]
}
],
'required': ['name']
}
But now the second validation always fails, whether the json provided is valid or invalid. The error message I get is:
{'name': 'John Doe', 'auth_token': '9d9a324b-26de-4ac3-85eb-05566e4a7204', 'username': None, 'password': None, 'project_id': 2785, 'contract_date_from': None, 'contract_date_to': None} is valid under each of {'required': ['contract_date_from', 'contract_date_to']}, {'required': ['project_id']}
No matter what values I send in those three fields (ie. project id, contract date from and contract date to), it fails with the same error. I've tried leaving all three empty, completing all three, and all permutations in between, but the error stays the same.
I've been reading the documentation for json schema but I can't seem to grasp what's going on with this example. I'm considering trying different approaches for this, but I'd really like to understand why this is not working. Any help is appreciated!
{'name': 'John Doe', 'auth_token': '9d9a324b-26de-4ac3-85eb-05566e4a7204', 'username': None, 'password': None, 'project_id': 2785, 'contract_date_from': None, 'contract_date_to': None} is valid under each of {'required': ['contract_date_from', 'contract_date_to']}, {'required': ['project_id']}
Read the error message more carefully: you requested that project_id be provided, OR contract_date_from and contract_date_to are provided, but you are providing all three of these. Providing a null value in a property is still providing a property. The error message is confusing, but you'd be failing validation anyway because null is not a string. Your evaluator is simply running the allOf->anyOfs first, so that's the error that comes back first. You should still get the type violation errors as well, though (if you don't, that's a bug: evaluators are required to provide ALL errors, not just the first.)
You can make the errors better at the expense of brevity by adding the "type" checks to live next to the "required" keywords. That will ensure the oneOf keywords produce failures rather than successes and maybe make the error messages more obvious.
I am trying to validate JSON, the schema for which specifies a list of dicts with arbitrary string keys, the corresponding values of which are dicts with a strict schema (i.e, the keys of the inner dict are strictly some string, here 'a'). From the Cerberus docs, I think that what I want is the 'keysrules' rule. The example in the docs seems to only show how to use 'keysrules' to validate arbitrary keys, but not their values. I wrote the below code as an example; the best I could do was assume that 'keysrules' would support a 'schema' argument for defining a schema for these values.
keysrules = {
'myDict': {
'type': 'dict',
'keysrules': {
'type': 'string',
'schema': {
'type': 'dict',
'schema': {
'a': {'type': 'string'}
}
}
}
}
}
keysRulesTest = {
'myDict': {
'arbitraryStringKey': {
'a': 'arbitraryStringValue'
},
'anotherArbitraryStringKey': {
'shouldNotValidate': 'arbitraryStringValue'
}
}
}
def test_rules():
v = Validator(keysrules)
if not v.validate(keysRulesTest):
print(v.errors)
assert(0)
This example does validate, and I would like it to not validate on 'shouldNotValidate', because that key should be 'a'. Does the flexibility implied by 'keysrules' (i.e, keys governed by 'keysrules' have no constraint other than {'type': 'string'}) propagate down recursively to all schemas underneath it? Or have I made some different error? How can I achieve my desired outcome?
I didn't want keysrules, I wanted valuesrules:
keysrules = {
'myDict': {
'type': 'dict',
'valuesrules': {
'type': 'dict',
'schema': {
'a': {'type': 'string'}
}
}
}
}
keysRulesTest = {
'myDict': {
'arbitraryStringKey': {
'a': 'arbitraryStringValue'
},
'anotherArbitraryStringKey': {
'shouldNotValidate': 'arbitraryStringValue'
}
}
}
def test_rules():
v = Validator(keysrules)
if not v.validate(keysRulesTest):
print(v.errors)
assert(0)
This produces my desired outcome.
AVOID EVAL
My question has been answered and I ended up using eval, but after some searching on what eval does and can do I ended up not using it and instead used an alternative found here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval#Do_not_ever_use_eval!
In my application i'm building the whole chart options in the backend and returning it as a json response
def get_chart_data(request):
chart = {
'title': {
'text': ''
},
'xAxis': {
'categories': [],
'title': {
'text': ''
},
'type': 'category',
'crosshair': True
},
'yAxis': [{
'allowDecimals': False,
'min': 0,
'title': {
'text': ''
}
}, {
'allowDecimals': False,
'min': 0,
'title': {
'text': ''
},
'opposite': True
}],
'series': [{
'type': 'column',
'yAxis': 1,
'name': '',
'data': []
}, {
'type': 'line',
'name': '',
'data': []
}, {
'type': 'line',
'name': '',
'data': []
}]
}
return JsonResponse(chart)
And then get the data using ajax and use the response for the data
Highcharts.chart('dashboard1', data);
I'm ok with this so far but i've run into problems if I want to use highcharts functions as part of the options, for example setting the color of text using Highcharts.getOptions().colors[0],
'title': {
'text': 'Rainfall',
'style': {
'color': Highcharts.getOptions().colors[0]
}
},
If i don't put quotes to this when building the options in views.py it would be treated as python code and result in an error, however if i add quotes to it, it will be treated as string in javascript which would not work.
Is this possible? or should i just build the options in javascript and just get the data part in the backend and not the whole thing.
You could return the JS code in Django as a string, and then you can run eval() on it, but executing code like that opens the possibility of an XSS attack, especially if the information is user-submittable.
Your best bet otherwise would be to create the styling on the JS end if possible, and manipulate the incoming data.
document.querySelector('a').addEventListener('click', function (e) {
e.preventDefault();
var complexJson = {"parent": {"child": "alert('Here is a nested alert!')"}}
var alertString = "alert('Here is a simple alert!')";
eval(complexJson["parent"]["child"])
eval(alertString)
})
Click me!
I am retrieving a record set from a database.
Then using a for statement I am trying to construct my data to match a 3rd party API.
But I get this error and can't figure it out:
"errorType": "TypeError", "errorMessage": "list indices must be
integers, not str"
"messages['english']['merge_vars']['vars'].append({"
Below is my code:
cursor = connect_to_database()
records = get_records(cursor)
template = dict()
messages = dict()
template['english'] = "SOME_TEMPLATE reminder-to-user-english"
messages['english'] = {
'subject': "Reminder (#*|code|*)",
'from_email': 'mail#mail.com',
'from_name': 'Notifier',
'to': [],
'merge_vars': [],
'track_opens': True,
'track_clicks': True,
'important': True
}
for record in records:
record = dict(record)
if record['lang'] == 'english':
messages['english']['to'].append({
'email': record['email'],
'type': 'to'
})
messages['english']['merge_vars'].append({
'rcpt': record['email']
})
for (key, value) in record.iteritems():
messages['english']['merge_vars']['vars'].append({
'name': key,
'content': value
})
else:
template['other'] = "SOME_TEMPLATE reminder-to-user-other"
close_database_connection()
return messages
The goal is to get something like this below:
messages = {
'subject': "...",
'from_email': "...",
'from_name': "...",
'to': [
{
'email': '...',
'type': 'to',
},
{
'email': '...',
'type': 'to',
}
],
'merge_vars': [
{
'rcpt': '...',
'vars': [
{
'content': '...',
'name': '...'
},
{
'content': '...',
'name': '...'
}
]
},
{
'rcpt': '...',
'vars': [
{
'content': '...',
'name': '...'
},
{
'content': '...',
'name': '...'
}
]
}
]
}
This code seems to indicate that messages['english']['merge_vars'] is a list, since you initialize it as such:
messages['english'] = {
...
'merge_vars': [],
...
}
And call append on it:
messages['english']['merge_vars'].append({
'rcpt': record['email']
})
However later, you treat it as a dictionary when you call:
messages['english']['merge_vars']['vars']
It seems what you want is something more like:
vars = [{'name': key, 'content': value} for key, value in record.iteritems()]
messages['english']['merge_vars'].append({
'rcpt': record['email'],
'vars': vars,
})
Then, the for loop is unnecessary.
What the error is saying is that you are trying to access an array element with the help of string not index (int).
I believe your mistake is in this line:
messages['english']['merge_vars']['vars'].append({..})
You declared merge_vars as array like so:
'merge_vars': []
So, you either make it dict like this:
'merge_vars': {}
Or, use it as array:
messages['english']['merge_vars'].append({..})
Hope it helps
Your issues, as the Error Message is saying, is here: messages['english']['merge_vars']['vars'].append({'name': key,'content': value})
The item messages['english']['merge_vars'] is a list and thus you're trying to access an element when you do something like list[i] and i cannot be a string, as is the case with 'vars'. You probably either need to drop the ['vars'] part or set messages['english']['merge_vars'] to be a dict so that it allows for additional indexing.
So say i have the following document:
test_obj = {
'my_things':{
'id17': {
'blah': 3,
'weird': 'yay',
'thechallenge': ObjectId('5712d06fdb4d0856551300d2')
},
'id32': {
'blah': 62,
'weird': 'hoorah',
'thechallenge': ObjectId('5712d06fdb4d0856551300d4')
}
},
'_id': 12,
'an_extra_field': 'asdf'
}
for this document i have the following schema:
API.config['DOMAIN']['test_obj']['schema'] = {
'id': {'type': 'int'},
'an_extra_field': {'type': 'string'},
'my_things': {
'type': 'dict',
'valueschema': {
'type': 'dict',
'schema': {
'blah': {'type': 'dict'},
'weird': {'type': 'string'},
'thechallenge': {'type': 'objectid'}
}
}
}
}
Now say i make a patch with the following pseudocode:
data = {
'mythings': {
'id17': {
'thechallenge': '5712d06fdb4d0856551300d8'
}
}
}
PATCH(url='/v1/test_objs/12', data=data)
When I make this patch Cerberus raises an error during validation, saying "value '5712d06fdb4d0856551300d8' cannot be converted to a ObjectId". Now this is a valid object id, and i find that if I make a patch to other non-valueschema fields it does not raise this error. It seems like valueschema was not meant to have a value of dict, and adding an extra 'schema' attribute was the only way i could get around cerberus raising a schemaerror/having cerberus actually validate my fields. But eve does not appear to actually be serializing my fields in my dictionary correctly. It should be of type ObjectId when it gets passed to Cerberus.
The way i'm temporarily getting around this is by manipulating my the code in Eve. In common.py (module) in serialize (function) in line 398 i added, where it checks if the field schema is a 'valueschema':
elif field_type == 'dict' and 'schema' in field_schema['valueschema']:
for subdocument in document[field].values():
serialize(subdocument, schema=field_schema['valueschema']['schema'])
Should i not be using type dict for the valueschema? If not how else should i handle this scenario? I would like to not have to maintain my own fork of Eve, so if others do want the ability to have valueschema be of type dict should i submit a pull-request for this change?
This has been fixed with Eve v0.6.4, which has just been released.