I've been digging a hole deeper and deeper trying to work out my api script in python. I'm not sure if i'm just doing something wrong, or i'm misunderstanding the way the projection section is to be written.
Here is the information I'll provide.. So we have two ways to get information from our API through JSON or via socketJS. The socket JS code i will provide down the bottom which essentially does the same thing..
What's going wrong is that it doesn't seem to be processing the parameters correctly, all i get is the same values as i would if i have added no filters or projections.. Does anyone have any ideas on what i'm doing wrong? I suspect i'm not using the requests library correctly, however i have looked and i don't seem to find anything for my particular case in the documentation.
Working SocketJS:
{
"address": "service",
"body": {
"action": "getControlers",
"params": {
"filter": {
"deviceClass": {
"$like" : "*mainControllers*"
}
},
"projection": {
"tagValues": {
"IdMap": 1,
"StateDevice": 1
}
},
"limit":1000
}
}
}
equivalent line via API-Rest outside of Python:
https://URLURL/rest/service/controlers?projection={"tagValues":{"StateDevice": 1}}&filter= {"DeviceClass": {"$like" : "*Controlers*"}}
My script is as follows:
import requests
import json
import os
header = {"Authorization": 'access_token *Iputakeyheretomakethingswork*'}
parameters = {"Filter": {"deviceClass": {"$like" : "*Controller*"}},
"Projection": {"tagValues":{"IdStateMap": 1, "stateDevice": 1}}}
response = requests.get("https://urlgoeshere", headers=header, params=parameters)
print(response.status_code)
data = response.json()
with open('data.txt', 'w') as outfile:
json.dump(data, outfile, sort_keys = True, indent = 4,
ensure_ascii = False)
params does not take a nested dictionary structure. Your API is essentially asking for JSON formatted values in the query string, but you are not providing those.
Moreover, your example URL uses lowercased parameter names, your dictionary contains uppercased parameters.
Instead, requests will convert the any container in params to a string by taking each element, before encoding it with URL encoding. For a dictionary, that means only the keys are used; you are essentially producing the following URL:
>>> import requests
>>> parameters = {"Filter": {"deviceClass": {"$like" : "*Controller*"}},
... "Projection": {"tagValues":{"IdStateMap": 1, "stateDevice": 1}}}
>>> prepped = requests.Request('GET', 'http://example.com/', params=parameters).prepare()
>>> prepped.url
'http://example.com/?Filter=deviceClass&Projection=tagValues'
The following would produce the equivalent of your sample URL:
parameters = {
"filter": '{"deviceClass": {"$like": "*Controller*"}}',
"projection": '{"tagValues": {"IdStateMap": 1, "stateDevice": 1}}'
}
Note that I lowercased the keys and the values are just strings. You could use the json.dumps() function to produce those strings from Python dictionaries if need be:
import json
filter = {"deviceClass": {"$like": "*Controller*"}}
projection = {"tagValues": {"IdStateMap": 1, "stateDevice": 1}}
parameters = {
"filter": json.dumps(filter),
"projection": json.dumps(projection),
}
Demo:
>>> parameters = {
... "filter": '{"deviceClass": {"$like" : "*Controller*"}}',
... "projection": '{"tagValues":{"IdStateMap": 1, "stateDevice": 1}}'
... }
>>> prepped = requests.Request('GET', 'http://example.com/', params=parameters).prepare()
>>> prepped.url
'http://example.com/?filter=%7B%22deviceClass%22%3A+%7B%22%24like%22+%3A+%22%2AController%2A%22%7D%7D&projection=%7B%22tagValues%22%3A%7B%22IdStateMap%22%3A+1%2C+%22stateDevice%22%3A+1%7D%7D'
>>> from urllib.parse import urlparse, parse_qsl
>>> parse_qsl(urlparse(prepped.url).query)
[('filter', '{"deviceClass": {"$like" : "*Controller*"}}'), ('projection', '{"tagValues":{"IdStateMap": 1, "stateDevice": 1}}')]
Related
I am trying to get the values from objects in the following JSON response:
[
{
"compositionId": "-Mkl92Mii2UF3xzi1q7L",
"compositionName": null,
"mainComposition": true,
"animation": {
"state": "Out1"
}
},
{
"compositionId": "bbbbbb",
"compositionName": null,
"mainComposition": true,
"animation": {
"state": "Out1"
}
}
]
What I would like to get in a loop is all the compositionIds but I don't get the correct output.
I can dump the complete JSON with the following code:
import requests
import json
url = 'http://192.168.1.33/data'
r = requests.get(url)
data = json.loads(r.content.decode())
json_str = json.dumps(data)
resp = json.loads(json_str)
print (resp)
You can simply use the requests module, in fact it does provide a builtin json decoder, that is the .json() function. Done that, you can simply iterate over your json objects with a simple for.
You could do something similar to this:
import requests
url = 'http://192.168.1.33/data'
r = requests.get(url)
my_json_file = r.json()
for json_object in my_json_file:
# Do something with json_object['compoitionId']
pass
Try something like this:
import requests
import json
url = 'http://192.168.1.33/data'
r = requests.get(url)
data = json.loads(r.content.decode())
print([d['compositionId'] for d in data])
I'm trying to use a REST API apparently constructed with LoopBack and it requires that my filter parameters be encoded in a specific way.
What is this sort of transformation called?
# this is a JSON type encoding
...?filter={"where": {"StartDate": {"gte": "2021-01-01T00:00:00.000Z"}}, "offset": 0, "limit": 100, "order": "id DESC" }
which needs to be encoded as some sort of HTTP query string
...?filter=%7B%22where%22%3A%20%7B%22StartDate%22%3A%20%7B%22gte%22%3A%20%222021-06-01T00%3A00%3A00.000Z%22%7D%7D%2C%20%22offset%22%3A%200%2C%20%22limit%22%3A%20100%2C%20%22order%22%3A%20%22id%20DESC%22%20%7D
Is there a python function to do this?
This is URL encoding.
URLs cannot contain a lot of different special characters, such as spaces (a space would be %20 in URL encoding).
Note that URL encoding is quite easy to recognize once you know it exists, due to the %xx pattern.
The urllib has functions to deal with encoding/decoding this.
To create an URL encoded string use urllib.parse.quote(string). Relevant docs here...
Example
from urllib.parse import quote
jsonstring = '{"where": {"StartDate": {"gte": "2021-01-01T00:00:00.000Z"}}, "offset": 0, "limit": 100, "order": "id DESC" }'
urlencoded = quote(jsonstring)
print(urlencoded)
# output
# %7B%22where%22%3A%20%7B%22StartDate%22%3A%20%7B%22gte%22%3A%20%222021-01-01T00%3A00%3A00.000Z%22%7D%7D%2C%20%22offset%22%3A%200%2C%20%22limit%22%3A%20100%2C%20%22order%22%3A%20%22id%20DESC%22%20%7D
You are looking for URL encoding. If you already have the JSON encoded in a variable as such, then just encode it to the filter parameter:
from urllib.parse import urlencode, quote
base_url = "..."
filter_string = """{"where": {"StartDate": {"gte": "2021-01-01T00:00:00.000Z"}}, "offset": 0, "limit": 100, "order": "id DESC" }"""
query = urlencode({"filter": filter_string}, quote_via=quote)
url = f"{base_url}?{query}"
Now, I expect that the JSON is probably coming from a Python data structure. You can use the dumps function from json to handle that encoding:
from urllib.parse import urlencode, quote
import json
base_url = "..."
data = {
"where": {
"StartDate": {
"gte": "2021-01-01T00:00:00.000Z"
}
},
"offset": 0,
"limit": 100,
"order": "id DESC"
}
filter_string = json.dumps(data)
query = urlencode({"filter": filter_string}, quote_via=quote)
url = f"{base_url}?{query}"
And you have the URL to call to in the variable url.
I have the following JSON data:
x = df.to_json(orient='records')
print(x)
[{"val":"3760","id":"204","quantity":2},{"val":"8221","id":"220","quantity":8}]
I want to add the data to my REST call, but it results in the following string in the payload (note: the single quotes around the square bracket:
'updateVals': '[{"val":"3760","id":"204","quantity":2},{"val":"8221","id":"220","quantity":8}]'}}
The fact that the JASON values are listed as one big string, the REST call results in an HTTP 400 error.
The code is:
url = 'my_url'
payload = {
'client_id': 'my_id',
'api_key': 'my_key',
"data": {
"uuid": "myUUID",
"timeStamp": "2018-09-12T06:17:48+00:00",
"updateVals": x
}
}
How do I plug the JSON into the REST call? I assume I have to split the string, or maybe there is a more straightforward answer?
Still not sure what you need, JSON is really one big string:
x = [{"val":"3760","id":"204","quantity":2},
{"val":"8221","id":"220","quantity":8}]
>>> json.dumps(x)
>>> '[{"val": "3760", "id": "204", "quantity": 2}, {"val": "8221", "id": "220", "quantity": 8}]'
I'm trying to do a basic Bitly shortening URL call. However, I cannot seem to either push the json correctly, or deal with the json response correctly... I omitted some obvious variables for brevity and obfuscated some real values for security purposes.
import requests
import json
bitly_header = {'Authorization':'Bearer
some_long_secret_character_string_here', 'Content-Type':'application/json'}
bitly_data = {
"long_url": ""+long_url+"",
"group_guid": ""+bitly_guid+""
}
short_link_resp =requests.post(bitly_endpoint,data=bitly_data,headers=bitly_header)
short_link_json = short_link_resp.json()
short_link = short_link_json["link"]
Errors is "Key error: 'link'
The json I get from Postman is:
{
"created_at": "1970-01-01T00:00:00+0000",
"id": "bit.ly/2MjdrrG",
"link": "bit.ly/2MjdrrG",
"custom_bitlinks": [],
"long_url": "google.com/",
"archived": false,
"tags": [],
"deeplinks": [],
"references": {
"group": "https://api-ssl.bitly.com/v4/groups/Bi7i8IbM1x9"
}
}
try replace data with json:
short_link_resp =requests.post(bitly_endpoint, json=bitly_data, headers=bitly_header)
see the doc ref.
Here's my function which connects to an API:
def order_summary():
"""Get order summary for a specific order"""
# Oauth2 params
headerKey = api_login()
headers = {'Authorization': headerKey}
# Payload params
payloadOrderSum = {
"domainId": 15,
"domainName": "SGL",
"orderId": 3018361
}
# API response
orderSumResp = requests.post(url + "order/summary", data=payloadOrderSum, headers=headers)
print(orderSumResp.content)
The API expects a JSON array as Payload Params which essentially looks like that:
[
{
"domainId": 0,
"domainName": "string",
"orderId": 0
}
]
The other endpoints I coded for on this API didn't need for the params to be an array so I could just use them as is and send them as a dictionary and it worked.
I've tried a couple things using the JSON library but I can't seem to get it to work. I saw that the JSonEncoder converts lists and tuples to JSON arrays but I couldn't figure it out.
Not sure what other info I could provide but just ask if there are any.
Thanks!
Wrap payloadOrderSum into a list:
payloadOrderSum = {
"domainId": 15,
"domainName": "SGL",
"orderId": 3018361
}
orderSumResp = requests.post(url + "order/summary", json=[payloadOrderSum], headers=headers)
Note that I used json kwarg instead of data (added in version 2.4.2).
dump your dict with json.dumps requests-doc
r = requests.post(url, data=json.dumps(payload))
It could help if you specify what you tried with the JSON library.
However, you might wanna try this if you haven't already done so:
import json
payloadOrderSum = json.dumps(
{
"domainId": 15,
"domainName": "SGL",
"orderId": 3018361
}
)