Develop Reference

python urllib: build urls including parameters with and without keyword

I am building grafana links in python with urllib like the following:
from urllib.parse import urlencode, urlunsplit
parameters = {
"parameter1":"value1",
"parameter2":"value2"
}
query = urlencode(
query = parameters,
doseq = True
)
link = urlunsplit((
"https",
"my_grafana.com",
"/graph",
query,
""
))
link will be in this case 'https://my_grafana.com/graph?parameter1=value1&parameter2=value2'. I now want to add parameters with no keyword for example "kiosk". The link should look like 'https://my_grafana.com/graph?parameter1=value1&parameter2=value2&kiosk&other_parameter'
As urlencode returns a string with the parameters I could manipulate the string like in the following example before I give it to urlunsplit:
no_keyword_parameters = ["kiosk","other_parameter"]
query = "&".join([query, *no_keyword_parameters])
I wonder if you can put parameters with and without keyword directly with urlencode together. I tried giving "kiosk" as a dictionary entry with None as content ({"kiosk": None}) but it includes the None in the url. Approaches, where I give a list of tuples instead of a dictionary for the parameters, were also unsuccessful.
Thank you for any help.

As mentioned by Ondrej, urlencode builds the query using k + '=' + v.
You could add non value parameters manually:
from urllib.parse import urlencode, urlunsplit, quote_plus
parameters = {"parameter1": "value1", "parameter2": "value2"}
no_value_parameters = ["kiosk", "other_parameter"]
no_value_parameters_quoted = [quote_plus(p) for p in no_value_parameters]
query = urlencode(query=parameters, doseq=True)
link = urlunsplit(("https", "my_grafana.com", "/graph", query, ""))
link = f"{link}&{'&'.join(no_value_parameters_quoted)}"
print(link)
Out:
https://my_grafana.com/graph?parameter1=value1&parameter2=value2&kiosk&other_parameter

What you've done seems sound and you could either do it like that or formalize it a bit more in your own encoding function, but urllib.parse.urlencode does not seem to understand the notion of parameters without value. If you look at the implementation (with doseq you get a variation of the same for the part relevant to your question):
for k, v in query:
...
l.append(k + '=' + v)
I.e. you have to have a key, value pair (to unpack two values) and whatever they are quoted to (that happens in the ellipses) will be a str joined over =. So even using custom qoute_via you cannot really change its function.
That linked implementation is the one provided with CPython, but also the documentation expects: key/value pairs, so that behavior really is as specified / documented:
The resulting string is a series of key=value pairs separated by '&' characters...

python variable body API

Good Morning,
I need to understand how to insert a variable into this variable (CHANGEME).
payload = "{\n\t"client": {\n\t\t"clientId": "name"\n\t},\n\t"contentFieldOption": {\n\t\t"returnLinkedContents": false,\n\t\t"returnLinkedCategories": false,\n\t\t"returnEmbedCodes": false,\n\t\t"returnThumbnailUrl": false,\n\t\t"returnItags": false,\n\t\t"returnAclInfo": false,\n\t\t"returnImetadata": false,\n\t\t"ignoreITagCombining": false,\n\t\t"returnTotalResults": true\n\t},\n\t"criteria": {\n\t\t"linkedCategoryOp": {\n\t\t\t"linkedCategoryIds": [\n\t\t\t\t" CHANGEME ",\n\t\t\t\t"!_TRASH"\n\t\t\t],\n\t\t\t"cascade": true\n\t\t}\n\t},\n\t"numberOfresults": 50,\n\t"offset": 0,\n\t"orderBy": "creationDate_A"\n}"
It is part of the body to be inserted inside API POST request.
I have tried various alternatives, but to no avail it led me to solve my problem

Don't try to hack this string with regexes; you'll end up with invalid data in no time. Use json.loads() to convert it into a dictionary, find the key CHANGEME, and do whatever you need to do (which you do not really explain).
>>> paydict = json.loads(payload)
>>> print(json.dumps(paydict, indent=4)
{
"criteria": {
"linkedCategoryOp": {
"linkedCategoryIds": [
" CHANGEME ",
"!_TRASH"
...
API objects usually have a consistent structure, so your variable is probably always in the list paydict["criteria"]["linkedCategoryOp"]["linkedCategoryIds"]. Find the index of " CHANGEME " in this list, and take it from there.

You can use re - Python's regular expressions module :
import re
payload = '{\n\t"client": {\n\t\t"clientId": "name"\n\t},\n\t"contentFieldOption": {\n\t\t"returnLinkedContents": false,\n\t\t"returnLinkedCategories": false,\n\t\t"returnEmbedCodes": false,\n\t\t"returnThumbnailUrl": false,\n\t\t"returnItags": false,\n\t\t"returnAclInfo": false,\n\t\t"returnImetadata": false,\n\t\t"ignoreITagCombining": false,\n\t\t"returnTotalResults": true\n\t},\n\t"criteria": {\n\t\t"linkedCategoryOp": {\n\t\t\t"linkedCategoryIds": [\n\t\t\t\t" CHANGEME ",\n\t\t\t\t"!_TRASH"\n\t\t\t],\n\t\t\t"cascade": true\n\t\t}\n\t},\n\t"numberOfresults": 50,\n\t"offset": 0,\n\t"orderBy": "creationDate_A"\n}'
payload = re.sub("\n|\t","",payload).strip() # do some cleanup
payload = re.sub("\s+CHANGEME\s+","NEW VALUE",payload) # Replace the value
print(payload) # CHANGEME is replaced with NEW VALUE

You could use a simple string replace to swap "CHANGEME" with something else.
new_str = 'IMCHANGED'
payload.replace('CHANGEME', new_str)
This solves your stated problem, unless there are extra constraints about what the payload looks like (right now you're assuming it's a string, or how many times the word CHANGEME occurs). Please clarify if that is the case.

What kind of data structure is this? Python

Studying Python, I am following an excellent Corey Schafer tutorial on Flask, he does this (I have extracted and summarized it for obvious reasons):
from folder_app import app # I did it to follow the structure and that the code is equal to the original
s = Serializer(app.config['SECRET_KEY'], 1800) # key, seconds
token = s.dumps({'user_id': 1}).decode('utf-8')
s = Serializer(app.config['SECRET_KEY'])
user_id = s.loads(token)['user_id'] # This is where I have the doubt
print(user_id)
print(type(s.loads(token)))
The code works, the problem I have is that although as you can see (s.loads (token)) is a dict, I expected to see something like this s.loads ({token ['user_id']}), or s.loads (token ['user_id']) or something like that. That is, it is a dict but it does not seem so. And my doubt goes in the sense if this comes from a greater concept of those they call "pythonic" (which I have not seen so far), or is something that only happens particularly as in this case. Incidentally, https://itsdangerous.palletsprojects.com/en/1.1.x/jws/ this appears: loads (self, s, salt = None, return_header = False) the arguments are in parentheses. I hope it is clear what my doubt is :)

I know this is not answer per say but just to add to my comment. This is an example of how the loads function works on dictionaries with the json module. https://docs.python.org/3/library/json.html#json.loads. What it does is take a json string and return the dictionary type object in Python. Your Serializer is doing something similar. It takes the token string and represents it as an object like dict
The s.dumps I am assuming is similar to json.dumps which gives you the json string representation of python dictionary.
import json
my_dict = json.loads('{"user_id": "Mane", "name": "Joe"}')
my_dict['user_id']
So you could just do json.loads('{"user_id": "Mane", "name": "Joe"}')['user_id'] which is just chaining the operations.

json change dictionary item to a list with one dictionary

I'm working with a Rest Api for finding address details. I pass it an address and it passes back details for that address: lat/long, suburb etc. I'm using the requests library with the json() method on the response and adding the json response to a list to analyse later.
What I'm finding is that when there is a single match for an address the 'FoundAddress' key in the json response contains a dictionary but when more than one match is found the 'FoundAddress' key contains a list of dictionaries.
The returned json looks something like:
For a single match:
{
'FoundAddress': {AddressDetails...}
}
For multiple matches:
{
'FoundAddress': [{Address1Details...}, {Address2Details...}]
}
I don't want to write code to handle a single match and then multiple matches.
How can I modify the 'FoundAddress' so that when there is a single match it changes it to a list with a single dictionary entry? Such that I get something like this:
{
'FoundAddress': [{AddressDetails...}]
}

If it's the external API sending responses in that format then you can't really change FoundAddress itself, since it will always arrive in that format.
You can change the response if you want to, since you have full control over what you've received:
r = json.parse(response)
fixed = r['FoundAddress'] if (type(r['FoundAddress']) is list) else [r['FoundAddress']]
r['FoundAddress'] = fixed
Alternatively you can do the distinction at address usage time:
def func(foundAddress):
# work with a single dictionary instance here
then:
result = map(func, r['FoundAddress']) if (type(r['FoundAddress']) is list) else [func(r['FoundAddress'])]
But honestly I'd take a clear:
if type(r['FoundAddress']) is list:
result = map(func, r['FoundAddress'])
else:
result = func(r['FoundAddress'])
or the response fix-up over the a if b else c one-liner any day.

If you can, I would just change the API. If you can't there's nothing magical you can do. You just have to handle the special case. You could probably do this in one place in your code with a function like:
def handle_found_addresses(found_addresses):
if not isinstance(found_addresses, list):
found_addresses = [found_addreses]
...
and then proceed from there to do whatever you do with found addresses as if the value is always a list with one or more items.

How to read and assign variables from an API return that's formatted as Dictionary-List-Dictionary?

So I'm trying to learn Python here, and would appreciate any help you guys could give me. I've written a bit of code that asks one of my favorite websites for some information, and the api call returns an answer in a dictionary. In this dictionary is a list. In that list is a dictionary. This seems crazy to me, but hell, I'm a newbie.
I'm trying to assign the answers to variables, but always get various error messages depending on how I write my {},[], or (). Regardless, I can't get it to work. How do I read this return? Thanks in advance.
{
"answer":
[{"widgets":16,
"widgets_available":16,
"widgets_missing":7,
"widget_flatprice":"156",
"widget_averages":15,
"widget_cost":125,
"widget_profit":"31",
"widget":"90.59"}],
"result":true
}
Edited because I put in the wrong sample code.

You need to show your code, but the de-facto way of doing this is by using the requests module, like this:
import requests
url = 'http://www.example.com/api/v1/something'
r = requests.get(url)
data = r.json() # converts the returned json into a Python dictionary
for item in data['answer']:
print(item['widgets'])

Assuming that you are not using the requests library (see Burhan's answer), you would use the json module like so:
data = '{"answer":
[{"widgets":16,
"widgets_available":16,
"widgets_missing":7,
"widget_flatprice":"156",
"widget_averages":15,
"widget_cost":125,
"widget_profit":"31",
"widget":"90.59"}],
"result":true}'
import json
data = json.loads(data)
# Now you can use it as you wish
data['answer'] # and so on...

First I will mention that to access a dictionary value you need to use ["key"] and not {}. see here an Python dictionary syntax.
Here is a step by step walkthrough on how to build and access a similar data structure:
First create the main dictionary:
t1 = {"a":0, "b":1}
you can access each element by:
t1["a"] # it'll return a 0
Now lets add the internal list:
t1["a"] = ["x",7,3.14]
and access it using:
t1["a"][2] # it'll return 3.14
Now creating the internal dictionary:
t1["a"][2] = {'w1':7,'w2':8,'w3':9}
And access:
t1["a"][2]['w3'] # it'll return 9
Hope it helped you.