I am trying to automate some queries to a API using Python. The problem is that the request needs to be created in a special way, and I just cant make it work. This is the part of the string I am having problems creating.
payload = "{\n \"filter\": {\n \"name\":[\"name1\", \"name2\"]\n }"
Where name1 and name2 is variable and is created from a list. The way I tried to do it was just to first create a function to create the
[\"name1\", \"name2\"]
This is the function
def create_string(list_of_names):
#Creates the string of line items we want data from
start = '[\\"%s\\"' % list_of_names[0]
for i in list_of_names[1 : ]:
start += ', \\"%s\\"' %(i)
start += "]"
return start
list_of_names = ['name1', 'name2']
And then just using the %s part to add it into the string.
payload = "{\n \"filter\": {\n \"name\":%s\n }" % create_string(list_of_names)
This doesnt work, and I can think this has something to do with how the \ is used in Python.
The create_string function creates different output depening on if I am printing it or not.
a = create_string(list_of_names)
print(a)
Creates the string I need to pass in using %s
[\"name1\", \"name2\", \"name3\"]
And just a outputs
'[\\"name1\\", \\"name2\\", \\"name3\\"]'
So my problem is then how to pass the print(a) part into the payload string. Does anyone have some sort of solution to this?
Instead of creating your payload by hand, first create a python dictionary and use the json-module to convert it to a string:
payload = {"filter": {"name": list_of_names]}}
payload = json.dumps(payload)
or with your more complex dictionary:
payload = {
"filter": {
"date": "pastThirtyDays",
"lineitem": {
"buyType": "RTB",
"name": list_of_names,
}
},
"metrics": ["cost"],
"dimensions": ["date", "lineItem"],
}
payload = json.dumps(payload)
Related
I've been using the standard Python ElasticSearch client to make single requests in the following format:
es.search(index='my_index', q=query, size=5, search_type='dfs_query_then_fetch')
I now want to make queries in batch for multiple strings q.
I've seen this question explaining how to use the msearch() functionality to do queries in batch. However, msearch requires the full json-formatted request body for each request. I'm not sure which parameters in the query API correspond to just the q parameter from search(), or size, or search_type, which seem to be API shortcuts specific to the single-example search().
How can I use msearch but specify q, size, and search_type?
I read through the API and figured out how to batch simple search queries:
from typing import List
from elasticsearch import Elasticsearch
import json
def msearch(
es: Elasticsearch,
max_hits: int,
query_strings: List[str],
index: str
):
search_arr = []
for q in query_strings:
search_arr.append({'index': index })
search_arr.append(
{
"query": {
"query_string": {
"query": q
}
},
'size': max_hits
})
request = ''
request = ' \n'.join([json.dumps(x) for x in search_arr])
resp = es.msearch(body = request)
return resp
msearch(es, query_strings=['query 1', 'query 2'], max_hits=1, index='my_index')
EDIT: For my use case, I made one more improvement, which was because I didn't want to return the entire document in the result– for my purpose, I just needed the document ID and its score.
So the final search request object part looked like this, including the '_source': False bit:
search_arr.append(
{
# Queries `q` using Lucene syntax.
"query": {
"query_string": {
"query": q
},
},
# Don't return the full profile string, etc. with the result.
# We just want the ID and the score.
'_source': False,
# Only return `max_hits` documents.
'size': max_hits
}
)
I'm working in Python (3.8) and I've successfully called an API gotten it to print the JSON within command line after running the Python file. Now, I want to be able to print a particular list of information (like all of the names from the JSON), and later on save that list as its own set of data, but I'm hitting a block.
Example JSON I'm working with:
{
"data": {
"employees": [
{
"fields": {
"name": "Buddy",
"superheroName": "Syndrome",
"workEmail": "syndrome#example.com",
}
},
{
"fields": {
"name": "Helen Parr",
"superheroName": "Elastigirl",
"workEmail": "elastigirl#example.com",
}
}
]
}
I’ve tried the following so far and I was able to get “data” to print, but anytime I try to print another “layer” and get to say...“employees” or “fields” even, I hit a wall.
url = "my API url"
response = requests.get(url)
if response.status_code != 200:
print('Error with status code {}'.format(response.status_code))
exit()
jsonResponse = response.json()
jsonPretty = json.dumps(jsonResponse, indent=4, sort_keys=True)
jsonDictionary = json.loads(jsonPretty)
keys = jsonDictionary.keys()
for key in jsonDictionary.keys():
print(key)
Ideally, could someone share insight into how I can access the 'name' JSON value and get Python to print it as a list like the following, for example:
Buddy
Helen Parr
JSON files are basically nested dictionaries. jsonDictionary only contains one key and one entry under that key: data and another dictionary with the rest your result respectively.
If you wanted to access the name fields specifically:
employeesDict = jsonDictionary['data']
feildsDictList = employeesDict['employees']
firstFieldsDict = fieldsDictList[0]
secondFieldsDict = fieldsDictList[1]
firstName = firstFieldsDict['name']
secondNAme = secondFieldsDict['name']
You can access it like this (make sure it's already a dictionary):
for i in h['data']['employees']:
print(i['fields']['name'])
This way you can access the names with i['fields']['name']
I have a dictionary in python of this format :
{
"intents": [
{
"name": "ER111-Test1",
"created": "2017-08-18T18:09:36.155Z",
"updated": "2017-08-18T18:09:41.755Z",
"description": null
},
{
"name": "ER2222-Test2",
"created": "2017-08-18T18:05:48.153Z",
"updated": "2017-08-18T18:06:06.004Z",
"description": null
}
],
"pagination": {
"refresh_url": "/v1/workspaces/9978a49e-ea89-4493-b33d-82298d3db20d/intents?version=2017-08-21"
}
}
I am trying to remove the "ER111" and "ER2222" values and reupload. I am running this method from IBM Watson's API.
The code is 99% working, but won't let me upload at the final step.
response = ...
names = [d['name'] for d in response['intents']]
for name in names:
fixed_name = re.sub('ER\d{4,5}-', ' ', name) #this works to remove the prefix
print(name + ' ' + fixed_name) #testing it works, it does
response = conversation.update_intent(workspace_id='1234567',
intent=str(name),
new_intent=str(fixed_name)) #fails here with code 400
At the final line, we fail with "Error: Invalid Request Body, Code: 400". It works if I hardcode the values rather than using variables. I have tried adding quotation marks and a few other things but can't seem to get it to play nice.
Any ideas what might cause this?
So it seems like Watson doesn't like leading spaces in the data you send. For starters, take a look here -
fixed_name = re.sub('ER\d{4,5}-', ' ', name)
# ^
Since name begins with ER..., this pattern is going to be replaced with a leading space. What you should've done is to replace with the empty string, ''.
However, a better way to do this would've been to just split on the first - using str.split. Here's how -
>>> "ER111-Test1".split('-', 1)[-1]
'Test1'
I'm pretty new to Python, so just working my way through understanding the data sets.
I'm having a little trouble producing the JSON output that is required for the API I am working with.
I am using
import json
json.load(data_file)
Working with Python dictionary and then doing
json.dump(dict, json_data)
My data needs to look like the following when it is output.
{
"event":{
"id":10006,
"event_name":"My Event Name",
},
"sub event":[
],
"attendees":[
{
"id":11201,
"first_name":"Jeff",
"last_name":"Smith",
},
{
"id":10002,
"first_name":"Victoria",
"last_name":"Baker",
},
]
}
I have been able to create the arrays in python and dump to json, but I am having difficulty creating the event "object" in the dictionary. I am using the below:
attendees = ['attendees']
attendeesdict = {}
attendeesdict['first_name'] = "Jeff"
attendees.append(attendeesdict.copy())
Can anyone help me add the "event" object properly?
In general, going from JSON to dictionary is almost no work because the two are very similar, if not identical:
attendees = [
{
"first_name": "Jeff"
# Add other fields which you need here
},
{
"first_name": "Victoria"
}
]
In this instance, attendees is a list of dictionaries. For the event:
event = {
"id": 10006,
"event_name": "My Event Name"
}
Putting it all together:
data = {
"event": event,
"sub event": [],
"attendees": attendees
}
Now, you can convert it to a JSON object, ready to send to your API:
json_object = json.dumps(data)
Assuming you have built all the values elsewhere and now you're just putting them together:
result = {'event':event_dict, 'sub event':subevent_list, 'attendees':attendees_list}
If you want just to statically create a nested dict, you can use a single literal. If you paste the JSON above into python code, you would get a valid dict literal.
Construct your dicts and add like below
{
"event":"add your dict"
"sub event":["add your dict"],
"attendees":["add your dict"]
}
I got a string in this form
payload = ["Text 1", "Text 2"]
I want to use Text 2 as an object. How can I return it?
UPDATE
I'm making a function which returns a generic template Facebook API.
The first payload works well, but I want to return a string and a object in the second payload ( result )
button = [
{
"type": "postback",
"title": "Buy item",
"payload": "Buy this item: " + (product['id'])
},
{
"type": "postback",
"title": "Add to wishlist",
"payload": result
}
]
My second payload should look like this:
payload = {
'Order_item',
product['title']
}
Because I got this error [buttons][1][payload] must be a UTF-8 encoded string so I did convert it and it returns a STRING in this form ["Order_item", "Ledverlichting Fiets Blauw"]
Because I want that when a Facebook user clicks on a postback ( Add to wishlist ), the product['title'] value will be save in Django database. And product['title'] is the Text 2 in the question above.
You need to split the string then trim and keep splitting/trimming to get all parts that you want to have in list
s = 'payload = ["Text 1", "Text 2"]'
items = s.strip()[1:-1]
#if 'payload = ' is also part of string you need to split it also by '=' so it would be:
#items = s.split("=").strip()[1:-1]
l = [item.strip()[1:-1] for item in items.split(",")]
then you have list of string that you can iterate, get n-th item and so on
the_item = l[1]
return the_item
assuming that you want a string object, you can get the specific index in the array(first spot = 0, second spot = 1, etc).
you can save the string object contained in the array like this:
payload = ["Text 1", "Text 2"]
s = payload[1]
and than you can return the s object.
you can try:
>>> payload = ["t1", "t2"]
>>> t2 = [1,2]
>>> eval(payload[1])
[1, 2]
I had a string that was in strictly list format and simply passing it through eval() worked.