I am building grafana links in python with urllib like the following:
from urllib.parse import urlencode, urlunsplit
parameters = {
"parameter1":"value1",
"parameter2":"value2"
}
query = urlencode(
query = parameters,
doseq = True
)
link = urlunsplit((
"https",
"my_grafana.com",
"/graph",
query,
""
))
link will be in this case 'https://my_grafana.com/graph?parameter1=value1¶meter2=value2'. I now want to add parameters with no keyword for example "kiosk". The link should look like 'https://my_grafana.com/graph?parameter1=value1¶meter2=value2&kiosk&other_parameter'
As urlencode returns a string with the parameters I could manipulate the string like in the following example before I give it to urlunsplit:
no_keyword_parameters = ["kiosk","other_parameter"]
query = "&".join([query, *no_keyword_parameters])
I wonder if you can put parameters with and without keyword directly with urlencode together. I tried giving "kiosk" as a dictionary entry with None as content ({"kiosk": None}) but it includes the None in the url. Approaches, where I give a list of tuples instead of a dictionary for the parameters, were also unsuccessful.
Thank you for any help.
As mentioned by Ondrej, urlencode builds the query using k + '=' + v.
You could add non value parameters manually:
from urllib.parse import urlencode, urlunsplit, quote_plus
parameters = {"parameter1": "value1", "parameter2": "value2"}
no_value_parameters = ["kiosk", "other_parameter"]
no_value_parameters_quoted = [quote_plus(p) for p in no_value_parameters]
query = urlencode(query=parameters, doseq=True)
link = urlunsplit(("https", "my_grafana.com", "/graph", query, ""))
link = f"{link}&{'&'.join(no_value_parameters_quoted)}"
print(link)
Out:
https://my_grafana.com/graph?parameter1=value1¶meter2=value2&kiosk&other_parameter
What you've done seems sound and you could either do it like that or formalize it a bit more in your own encoding function, but urllib.parse.urlencode does not seem to understand the notion of parameters without value. If you look at the implementation (with doseq you get a variation of the same for the part relevant to your question):
for k, v in query:
...
l.append(k + '=' + v)
I.e. you have to have a key, value pair (to unpack two values) and whatever they are quoted to (that happens in the ellipses) will be a str joined over =. So even using custom qoute_via you cannot really change its function.
That linked implementation is the one provided with CPython, but also the documentation expects: key/value pairs, so that behavior really is as specified / documented:
The resulting string is a series of key=value pairs separated by '&' characters...
Related
I am writing a Python wrapper for an API that supports the query parameters that have values (e.g. param1 below) and query parameters that do not have values value (e.g. param2 below) i.e.
https://example.com/service?param1=value1¶m2
The HTTP requests library famously supports parameters with values and parameters without values. However I need to be able to specify both types of parameters.
You have two options with Requests in terms of the query string: 1) provide key/value pairs by way of a dictionary, or 2) provide a string. If you provide #1, you'll always get a '=' for each key/value pair...not what you want. So you have to use #2, which will let you do whatever you want, since it will just include what you give it as the entire query string. The downside is that you have to construct the query string yourself. There are many ways of doing that, of course. Here's one way:
params = {'param1': 'value1', 'param2': None}
params = '&'.join([k if v is None else f"{k}={v}" for k, v in params.items()])
r = requests.get('https://example.com/service', params=params)
print(r.url)
This way lets you supply a dictionary, just like if you were letting Requests build the query string, but it allows you to specify a value of Null to indicate that you want just the key name, with no '='. Requests will normally not include the parameter at all if its value is None in the dictionary.
The result of this code is exactly what you gave as an example of what you want:
https://example.com/service?param1=value1¶m2
Good Morning,
I need to understand how to insert a variable into this variable (CHANGEME).
payload = "{\n\t"client": {\n\t\t"clientId": "name"\n\t},\n\t"contentFieldOption": {\n\t\t"returnLinkedContents": false,\n\t\t"returnLinkedCategories": false,\n\t\t"returnEmbedCodes": false,\n\t\t"returnThumbnailUrl": false,\n\t\t"returnItags": false,\n\t\t"returnAclInfo": false,\n\t\t"returnImetadata": false,\n\t\t"ignoreITagCombining": false,\n\t\t"returnTotalResults": true\n\t},\n\t"criteria": {\n\t\t"linkedCategoryOp": {\n\t\t\t"linkedCategoryIds": [\n\t\t\t\t" CHANGEME ",\n\t\t\t\t"!_TRASH"\n\t\t\t],\n\t\t\t"cascade": true\n\t\t}\n\t},\n\t"numberOfresults": 50,\n\t"offset": 0,\n\t"orderBy": "creationDate_A"\n}"
It is part of the body to be inserted inside API POST request.
I have tried various alternatives, but to no avail it led me to solve my problem
Don't try to hack this string with regexes; you'll end up with invalid data in no time. Use json.loads() to convert it into a dictionary, find the key CHANGEME, and do whatever you need to do (which you do not really explain).
>>> paydict = json.loads(payload)
>>> print(json.dumps(paydict, indent=4)
{
"criteria": {
"linkedCategoryOp": {
"linkedCategoryIds": [
" CHANGEME ",
"!_TRASH"
...
API objects usually have a consistent structure, so your variable is probably always in the list paydict["criteria"]["linkedCategoryOp"]["linkedCategoryIds"]. Find the index of " CHANGEME " in this list, and take it from there.
You can use re - Python's regular expressions module :
import re
payload = '{\n\t"client": {\n\t\t"clientId": "name"\n\t},\n\t"contentFieldOption": {\n\t\t"returnLinkedContents": false,\n\t\t"returnLinkedCategories": false,\n\t\t"returnEmbedCodes": false,\n\t\t"returnThumbnailUrl": false,\n\t\t"returnItags": false,\n\t\t"returnAclInfo": false,\n\t\t"returnImetadata": false,\n\t\t"ignoreITagCombining": false,\n\t\t"returnTotalResults": true\n\t},\n\t"criteria": {\n\t\t"linkedCategoryOp": {\n\t\t\t"linkedCategoryIds": [\n\t\t\t\t" CHANGEME ",\n\t\t\t\t"!_TRASH"\n\t\t\t],\n\t\t\t"cascade": true\n\t\t}\n\t},\n\t"numberOfresults": 50,\n\t"offset": 0,\n\t"orderBy": "creationDate_A"\n}'
payload = re.sub("\n|\t","",payload).strip() # do some cleanup
payload = re.sub("\s+CHANGEME\s+","NEW VALUE",payload) # Replace the value
print(payload) # CHANGEME is replaced with NEW VALUE
You could use a simple string replace to swap "CHANGEME" with something else.
new_str = 'IMCHANGED'
payload.replace('CHANGEME', new_str)
This solves your stated problem, unless there are extra constraints about what the payload looks like (right now you're assuming it's a string, or how many times the word CHANGEME occurs). Please clarify if that is the case.
/api/stats
?fields=["clkCnt","impCnt"]
&ids=nkw0001,nkw0002,nkw0003,nkw0004
&timeRange={"since":"2019-05-25","until":"2019-06-17"}
I'm currently working on a API called naver_searchad_api
link to github of the api If you want to check it out. but i don't think you need to
the final url should be a baseurl + /api/stats
and on fields and ids and timeRange, the url should be like that
the requests I wrote is like below
r = requests.get(BASE_URL + uri, params={'ids': ['nkw0001','nkw0002','nkw0003','nkw0004'], 'timeRange': {"since": "2019-05-25", "until": "2019-06-17"}}, headers=get_header(method,uri,API_KEY,SECRET_KEY,CUSTOMER_ID))
final_result = r.json()
print(final_result)
as I did below instead
print(r.url)
it returns as below
https://api.naver.com/stats?ids=nkw0001&ids=nkw0002&ids=nkw0002&ids=nkw0002&fields=clkCnt&fields=impCnt&timeRange=since&timeRange=until
the 'ids' is repeated and doesn't have dates that I put.
how would I make my code to fit with the right url?
Query strings are key-value pairs. All keys and all values are strings. Anything that is not trivially convertible to string depends on convention. In other words, there is no standard for these things, so it depends on the expectations of the API.
For example, the API could define that lists of values are to be given as comma-separated strings, or it could say that anything complex should be JSON-encoded.
In fact, that's exactly what the API documentation says:
fields string
Fields to be retrieved (JSON format string).
For example, ["impCnt","clkCnt","salesAmt","crto"]
The same goes for timeRange. The other values can be left alone. Therefore we JSON-encode those two values only.
We can do that inline with a dict comprehension.
import json
import requests
params = {
'fields': ["clkCnt", "impCnt"],
'ids': 'nkw0001,nkw0002,nkw0003,nkw0004',
'timeRange': {"since":"2019-05-25","until":"2019-06-17"},
}
resp = requests.get('https://api.naver.com/api/stats', {
key: json.dumps(value) if key in ['fields', 'timeRange'] else value for key, value in params.items()
})
On top of complying with the API's expectations, all keys and values that go into the query string need to be URL-encoded. Luckily the requests module takes care of that part, so all we need to do is pass a dict to requests.get.
I've got an API request to make that involves passing some variables from user input and a config file to a filter expression contained in a dictionary.
The API uses hashes in its structure to wrap stings by default, although I can specify another string wrapping indicator if need be via a separate request. As is, what I need to do is below, basically.
I can't figure out the syntax to get those strings to populate the values between the wrapper # signs. Lots of questions about this, but none addressing the basic syntax without additional functionality, as far as I can tell.
import config
import requests
var1 = **the result of user input, a string**
var2 = **a value from a config file, also a string**
url = (config.api_url)
payload = {
'key':config.api_key,
'Operation':'GetEntities',
'Entity':'my_entity',
'Attributes':'my_attribute1,my_attribute2',
'Filter':'api_var1<eq>#var1# AND api_var2<eq>#var2#'}
response = requests.post(url,payload)
They key point is here:
'Filter':'api_var1<eq>#var1# AND api_var2<eq>#var2#'
So if var1 = '1234' and var2 = '4321' I need it to be the equivalent of:
'Filter':'api_var1<eq>#1234# AND api_var2<eq>#4321#'
As far as I understand you want something like
'Filter':'api_var1<eq>#{0}# AND api_var2<eq>#{1}#'.format(var1, var2)}
or
'Filter':'api_var1<eq>#%s# AND api_var2<eq>#%s#' % (var1, var2)}
I'm having some trouble in understanding how jsonify works even though I went through the documentation. As you can see below, I'm calling the lookup() function which returns a dictionary, then I'm trying to jsonify it.
#app.route("/articles")
def articles():
a = lookup(33496)
return jsonify([link=a["link"], title = a["title"]]) #invalid syntax error
my helpers.py:
import feedparser
import urllib.parse
def lookup(geo):
"""Looks up articles for geo.""" #this function already parses the 'link' and 'title' form rss feed
# check cache for geo
if geo in lookup.cache:
return lookup.cache[geo]
# get feed from Google
feed = feedparser.parse("http://news.google.com/news?geo={}&output=rss".format(urllib.parse.quote(geo, safe="")))
# if no items in feed, get feed from Onion
if not feed["items"]:
feed = feedparser.parse("http://www.theonion.com/feeds/rss")
# cache results
lookup.cache[geo] = [{"link": item["link"], "title": item["title"]} for item in feed["items"]]
# return results
return lookup.cache[geo]
# initialize cache
lookup.cache = {}
The error that I'm getting is of invalid syntax. Any idea into what I'm doing wrong? Thanks
I think your dict syntax is wrong. You can read about more in official documentation.
The code that I think you are trying for is as follows:
#app.route("/articles")
def articles():
a = lookup(33496)
return jsonify({"link" : a["link"], "title" : a["title"]})
Specifically you should use curly braces instead of brackets ({}) and colon (:) instead of equals sign.
Another option is to let jsonify() to do the conversion (as pointed out in the other answer):
#app.route("/articles")
def articles():
a = lookup(33496)
return jsonify(link = a["link"], title = a["title"])
Nevertheless, I think you would be well advised to use create a dict. It becomes more flexible when you need to create larger JSON objects.
Hope this helps.
You dont need the square brackets, get rid of them.
return jsonify(link=a["link"], title=a["title"])
# ^At this point ^ and this one.
Read about keyword arguments in python.