Develop Reference

JSON Schema validation in Python

I wonder if it is possible to have a Swagger JSON with all schemas in one file to be validated across all tests.
Assume somewhere in a long .json file a schema is specified like so:
...
"Stop": {
"type": "object",
"properties": {
"id": {
"description": "id identifier",
"type": "string"
},
"lat": {
"$ref": "#/components/schemas/coordinate"
},
"lng": {
"$ref": "#/components/schemas/coordinate"
},
"name": {
"description": "Name of location",
"type": "string"
},
"required": [
"id",
"name",
"lat",
"lng"
]
}
...
So lat and lng schemas are defined in another schema (the same file at the top).
Now I want to get response from backend with array of those stops and would like to validate it against the schema.
How should I approach it?
I am trying to get a partial schema loaded and validate against it but then the $ref won't resolve. Also is there anyway to tell the validator to accept arrays? Schema only tells how a single object looks like. If I manually add
"type": "array",
"items": {
Stop...
with hardcoded coordinates
}
Then it seems to work.
Here are my functions to validate arbitrary response JSON against chosen "chunk" of the full Swagger schema:
def pick_schema(schema):
with open("json_schemas/full_schema.json", "r") as f:
jschema = json.load(f)
return jschema["components"]["schemas"][schema]
def validate_json_with_schema(json_data, json_schema_name):
validate(
json_data,
schema=pick_schema(json_schema_name),
format_checker=jsonschema.FormatChecker(),
)
Maybe another approach is preferred? Separate files with schemas for API responses (it is quite tedious to write, full_schema.json is generated from Swagger itself).

pymsteams. I need to mention a person in message sent by pymsteams. How to do it?

I have installed and connected pymsteams to a channel and able to send messages. I read documentation but didn't find anything related to how to send message with mentioning a person as #me -> mentioning a person in a teams.
that's a beginning of my code:
import pymsteams;
myMessage = pymsteams.connectocard("webhookurl")
.... // here is a logic
myMessage.send()

I believe you can use an AC card. Here is a detailed explanation where I learned it.

There is an issue thread on github requesting exactly this kind of functionality
https://github.com/rveachkc/pymsteams/issues/11
and there is a comment that presents a temporary solution, which is to manually modify the payload attribute of the connectorcard object. This worked for me.
teams_message = pymsteams.connectorcard(webhook)
teams_message.payload = {
"type": "message",
"attachments": [
{
"contentType": "application/vnd.microsoft.card.adaptive",
"content": {
"type": "AdaptiveCard",
"body": [
{
"type": "TextBlock",
"text": "Hello <at>Leonardo</at>"
}
],
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
"version": "1.0",
"msteams": {
"entities": [
{
"type": "mention",
"text": "<at>Leonardo</at>",
"mentioned": {
"id": "leonardo.benitez#contorso.com",
"name": "Leonardo Benitez"
}
}
]
}
}
}]
}
teams_message.send()
The code snippet, thanks to the author for providing it.

PyMongo - "language override unsupported: C++" when creating index

I have the collection posts that contains posts that look something like this
{
"_id": "5ae37fd270f3e72399988198",
"moderator": {
"flagged": false,
"reviewed": true,
"pending": false,
"time": "2018-04-27 20:34:38.099000",
"account": "samhamou"
},
"author": "cryptohazard",
"permlink": "security-enhancements-for-steem-messenger",
"title": "Security enhancements for Steem Messenger",
"repository": {
"owner": {
"login": "kingswisdom"
},
"fork": false,
"html_url": "https:\/\/github.com\/kingswisdom\/SteemMessenger",
"full_name": "kingswisdom\/SteemMessenger",
"name": "SteemMessenger",
"id": 127418766
}
}
I am trying to create an index on the collection in one of my Python files with the following code
posts = DB.posts
posts.drop_indexes()
posts.create_index([
("author", "text"),
("moderator.account", "text"),
("repository.full_name", "text")
])
but this is giving me the following error:
pymongo.errors.OperationFailure: language override unsupported: C++
How can I prevent this from happening? I can create the indexes using code found in the answers of this question:
> db.posts.createIndex({
"moderator.account": "text",
author: "text",
"repository.full_name": "text"
}, {
"language_override": "en"
});
However I want to be able to do it from within my Python script instead of having to do it from the MongoDB shell. I tried finding a way to add the "language_override": "en" option from within my Python script, but this doesn't seem possible when checking the documentation for create_index.

MongoDB Collection with Non-repeated field value

So I'm rather new to MongoDB. Here is an imaginary database with the following format.
{
"_id": "message_id",
"headers": {
"from": <from_email>,
"to": <to_email>,
"timestamp": <timestamp>
},
"message": {
"message": <the message contents>,
"signature": <signature contents>
}
}
Suppose all emails received are inserted into it and sometimes emails are double sent. How can one return a collection of emails from an author without any double sends.
I thought this might do it but it doesn't seem to work as expected:
db.mycoll.find({"headers.from": <authorname>}).distinct("message.message")
Edit:
Please excuse me, It seems I have been making some kind of typo, the above query works, but it only returns messages.messages without the Headers, How would I keep the headers intact as well?

Hard to really determine from your question which part is the "duplicate" or therefore should be unique. It stands to reason though that things such as the message "_id" and "timestamp" are not going to duplicate, so this only really leaves the message content, with the possible additional paranoia of that message being "from" the same person.
Document reshaping is generally best handled by the aggregation framework:
db.collection.aggregate([
{ "$group": {
"_id": { "message": "$message.message", "from": "$headers.from" },
"message_id": { "$first": "$_id" },
"headers": { "$first": "$headers" },
"message": { "$first": "$message" }
}},
{ "$project": {
"_id": "$message_id",
"headers": 1,
"message": 1
}}
])
The $group will filter out any matching message content with the $first operations selecting only the "first" found item for the matching field on the document grouping boundary.
There is an assumption in here that the existing order is by "timestamp" but if not then you might want to apply a $sort as the first pipeline stage before the others:
{ "$sort": { "headers.timestamp": 1 } }
The final $project really just restores the original document form and removes the "grouping key" that was supplied earlier. Just prettier than duplicating information and/or putting things out of place.

You could use distinct() to return an array of distinct messages from a specific author as follows:
db.collection.distinct('message.message', {"headers.from": <authorname>})

What you're looking for is not currently implemented (at least as far as I know). One work around would be this

db.mycoll.aggregate([
{
$match:{"headers.from": <authorname>}
},{
$group:{
_id:"$headers.from",
"message":{$addToSet:"$message.message"}
}
}
])

Building on Neil Lunn's answer above:
I think one can do
db.collection.aggregate([{"$match": {"headers.from": <from email>} } ,
{"$group": { "_id": "$message.message"},
"headers": {"$first": "$headers"},
"signature": {"$first": "$message.signature"},
"message_id": "$_id" }},
{"$project" : { "_id": "$message_id",
"headers": "$headers",
"message": { "message": "$_id", "signature": "$signature" } } }])
Since _id must be unique the consequence is that duplicate messages will not make the list, and then $project will restructure it to the original object structure with correct key names.
I guess I only have one question in this regard - is there a way to force uniqueness without aggregating into _id or is this generally considered the correct way to do it in MongoDB ?

Jsonschema, validating object keys with a custom function

I am using python-jsonschema for json validation. I have an object with localised texts that are specified inside rfc1766 language code keys as followings:
"Description": {
"en": "English Description",
"sv": "Swedish Description",
"fr": "French Description"
},
I've read in the documentation that I could use the 'format' attribute to check a custom format using a function. So,I wrote a method which takes a string as a parameter and returns True if it is an RFC1766 language string.
#_checks_drafts('rfc1766lang')
def rfc1766lang(instance):
"""some logic, return True if rfc1766"""
However I couldn't find any example on how to apply this to do validation on an object key, not a value.
Is this possible?
I have tried something like below but I couldn't succeed
rfc1766_string_schema_v2 = {
'type': 'object',
'format': 'rfc1766lang',
'additionalProperties': False
}
I know that it would be much easier if I had the json string as follows. However, this is not an option for now.
"Description": [{
"lan": "en",
"text": "Description in English"
}, {
"lan": "sv",
"name": "Description in Swedish"
}]

This is a very good and relevant question because this is actually part of the proposed syntax for v5, so the official meta-schema will have to deal with this as well.
JSON Schema cannot specify a "format" for object keys. The only "validation" JSON Schema supports for object keys is patternProperties, which supplies a regular expression.
For language codes, the best you can do is probably something like:
{
"type": "object",
"patternProperties": {
"^[a-zA-Z]+(-[a-zA-Z]+)*$": {...}
},
"additionalProperties": false
}
That would limit the data so that it was only allowed properties matching that pattern - but that's not the full validation you're looking for, I'm afraid.