I'm using the requests module to collect some data from a website. This application runs once every day. The amount of rows of data I get changes every time, per request I can get a maximum 250 rows of data. If there is more then 250 rows of data the API gives me a follow uplink which can be used to get the rows 251 >- 500 etc.
Now I have a problem, sometimes the amount of data is < 250 rows, this means there is no followuplink to use and that's exactly where my program gives the following error:
KeyError: #odata.nextLink
This is a piece of the application:
proxies = {'https': 'proxy.***.***.com:8080'}
headers = {"grant_type": "password",
"username": "****",
"password": "****",
"persistent": "true",
"device": '{"DeviceUniqueId":"b680c452","Name":"Chrome","DeviceVersion":"36","PlatformType":"Browser"}'}
url1 = 'https://****-***.com/odata/Results'
params_1 = (
('$filter', mod_date),
('$count', 'true'),
('$select', 'Status'),
('$expand', 'Result($select=ResultId),Specification($select=Name), SpecificationItem($select=Name,MinimumValue, MaximumValue)\n\n'),)
response_1 = requests.get(url_1, headers=headers, proxies=proxies, params=params_1)
q_1 = response_1.json()
next_link_1 = q_1['#odata.nextLink']
q_1 = [tuple(q_1.values())]
while next_link_1:
new_response_1 = requests.get(next_link_1, headers=headers, proxies=proxies)
new_data_1 = new_response_1.json()
q_1.append(tuple(new_data_1.values()))
next_link_1 = new_data_1.get('#odata.nextLink', None)
Now I actually want Python to only read the variable next_link_1 if its available otherwise it should just ignore it and collect what is available...
You only want to enter the while loop when q_1 has the key '#odata.nextLink' Inside the while loop, this is already accomplished in the line next_link_1 = new_data_1.get('#odata.nextLink', None) You could use the same approach -- setting next_link_1 to None if there is no next link -- before the while loop:
next_link_1 = q_1.get('#odata.nextLink', None)
This can be simplified to
next_link_1 = q_1.get('#odata.nextLink')
as None is already the default default value of dict.get().
NB: The question title is wrong. The variable always exists, as you are setting it. Only the existence of the key #odata.nextLink is fragile. So, what you actually want to do is check the existence of a key in a dictionary. To understand what is going on, you should familiarize yourself with the dict.get() method.
There is also some obvious refactoring possible here, getting rid of the repetition of the first iteration, and moving it into the loop:
proxies = {'https': 'proxy.***.***.com:8080'}
headers = {
'grant_type': 'password',
'username': '****',
'password': '****',
'persistent': 'true',
'device': '{"DeviceUniqueId":"b680c452","Name":"Chrome","DeviceVersion":"36","PlatformType":"Browser"}'
}
params = (
('$filter', mod_date),
('$count', 'true'),
('$select', 'Status'),
('$expand', 'Result($select=ResultId),Specification($select=Name), SpecificationItem($select=Name,MinimumValue, MaximumValue)\n\n'),
)
url = 'https://****-***.com/odata/Results'
data = []
while url:
response = requests.get(
url,
headers=headers,
proxies=proxies,
params=params,
)
response_data = response.json()
data.append(tuple(response_data.values()))
url = response_data.get('#odata.nextLink')
params = tuple()
Use get in both places. Better yet, restructure your loop so that you only need one call.
proxies = {'https': 'proxy.***.***.com:8080'}
headers = {...}
url1 = 'https://****-***.com/odata/Results'
params = (...)
qs = []
next_link = url
get_args = {'headers': headers, 'proxies': proxies, 'params': params}
while True:
response = requests.get(next_link, **get_args)
q = response.json()
qs.append(tuple(q.values())
if (next_link := q.get('#odata.nextLink', None)) is None:
break
if 'params' in get_args:
del get_args['params'] # Only needed in the first iteration
(I'm not terribly excited about how we ensure params is used only on the first iteration, but I think it's better than duplicating the process of defining next_link before the loop starts. Maybe something like this would be an improvement?
get_args = {...} # As above
new_get_args = dict(headers=..., proxies=...) # Same, but without params
while True:
...
if (next_link := ...) is None:
break
get_args = new_get_arg
Repeated assignment to get_args is probably cheaper than repeatedly testing for and deleting the params key, at the cost of having a second dict in memory. You could even drop that after the first iteration by adding a second assignment new_get_args = get_args to the end of the loop, which would result in a pair of do-nothing assignments for later iterations.)
I'd like to get the balance of a TRC20 token (in this case the WIN token) for a Tron wallet address that owns WIN tokens.
I'm using the Python module tron-api-python and based on what i've read on GitHub and the docs the code for that should look something like this:
from tronapi import Tron
# Source for ABI: https://tronscan.org/#/token20/TLa2f6VPqDgRE67v1736s7bJ8Ray5wYjU7/code
contract_abi = '[{"constant":true,"inputs":[],"name":"name","outputs":[{"name":"","type":"string"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":false,"inputs":[{"name":"spender","type":"address"},{"name":"value","type":"uint256"}],"name":"approve","outputs":[{"name":"","type":"bool"}],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":true,"inputs":[],"name":"totalSupply","outputs":[{"name":"","type":"uint256"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":false,"inputs":[{"name":"from","type":"address"},{"name":"to","type":"address"},{"name":"value","type":"uint256"}],"name":"transferFrom","outputs":[{"name":"","type":"bool"}],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":true,"inputs":[],"name":"decimals","outputs":[{"name":"","type":"uint8"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":false,"inputs":[{"name":"spender","type":"address"},{"name":"addedValue","type":"uint256"}],"name":"increaseAllowance","outputs":[{"name":"","type":"bool"}],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":false,"inputs":[{"name":"to","type":"address"},{"name":"value","type":"uint256"}],"name":"mint","outputs":[{"name":"","type":"bool"}],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":false,"inputs":[{"name":"value","type":"uint256"}],"name":"burn","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":true,"inputs":[{"name":"owner","type":"address"}],"name":"balanceOf","outputs":[{"name":"","type":"uint256"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":false,"inputs":[{"name":"from","type":"address"},{"name":"value","type":"uint256"}],"name":"burnFrom","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":true,"inputs":[],"name":"symbol","outputs":[{"name":"","type":"string"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":false,"inputs":[{"name":"account","type":"address"}],"name":"addMinter","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":false,"inputs":[],"name":"renounceMinter","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":false,"inputs":[{"name":"spender","type":"address"},{"name":"subtractedValue","type":"uint256"}],"name":"decreaseAllowance","outputs":[{"name":"","type":"bool"}],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":false,"inputs":[{"name":"to","type":"address"},{"name":"value","type":"uint256"}],"name":"transfer","outputs":[{"name":"","type":"bool"}],"payable":false,"stateMutability":"nonpayable","type":"function"},{"constant":true,"inputs":[{"name":"account","type":"address"}],"name":"isMinter","outputs":[{"name":"","type":"bool"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":true,"inputs":[{"name":"owner","type":"address"},{"name":"spender","type":"address"}],"name":"allowance","outputs":[{"name":"","type":"uint256"}],"payable":false,"stateMutability":"view","type":"function"},{"inputs":[{"name":"name","type":"string"},{"name":"symbol","type":"string"},{"name":"decimals","type":"uint8"}],"payable":false,"stateMutability":"nonpayable","type":"constructor"},{"anonymous":false,"inputs":[{"indexed":true,"name":"account","type":"address"}],"name":"MinterAdded","type":"event"},{"anonymous":false,"inputs":[{"indexed":true,"name":"account","type":"address"}],"name":"MinterRemoved","type":"event"},{"anonymous":false,"inputs":[{"indexed":true,"name":"from","type":"address"},{"indexed":true,"name":"to","type":"address"},{"indexed":false,"name":"value","type":"uint256"}],"name":"Transfer","type":"event"},{"anonymous":false,"inputs":[{"indexed":true,"name":"owner","type":"address"},{"indexed":true,"name":"spender","type":"address"},{"indexed":false,"name":"value","type":"uint256"}],"name":"Approval","type":"event"}]'
tron = Tron()
contract = tron.trx.contract("TLa2f6VPqDgRE67v1736s7bJ8Ray5wYjU7", abi=contract_abi)
balance = contract.functions.balanceOf("TXRZqGMEXsGTX6AQtcSgYknos93hqw18P7")
print(balance)
But the result i get is:
eth_abi.exceptions.NoEntriesFound: No matching entries for 'address' in encoder registry
You can use this API:
import requests
import json
def get_balance(address, token_symbol):
url = "https://apilist.tronscan.org/api/account"
payload = {
"address": address,
}
res = requests.get(url, params=payload)
trc20token_balances = json.loads(res.text)["trc20token_balances"]
token_balance = next((item for item in trc20token_balances if item["symbol"] == token_symbol), None)
if token_balance == None:
return 0
else:
return int(token_balance["balance"])
The previous response broadcasts a transaction to read a constant function; you don't need to do that if you're only going to read the balance.
Found it myself. This is not the code to get the balance but to send the WIN token so for this to return the balance the function_selector and the parameters would need to change but the rest should be fine since both is based on triggering a smart contract.
tron_kwargs = dict()
tron_kwargs["private_key"] = your_privkey
tron_kwargs["default_address"] = your_base58_address
tron = Tron(**tron_kwargs)
kwargs = dict()
kwargs["contract_address"] = tron.address.to_hex(wink_base58_contract_address)
kwargs["function_selector"] = "transfer(address,uint256)"
kwargs["fee_limit"] = 100000
kwargs["call_value"] = 0
kwargs["parameters"] = [
{
'type': 'address',
'value': tron.address.to_hex(recipients_base58_address)
},
{
'type': 'uint256',
'value': 8000000
}
]
raw_tx = tron.transaction_builder.trigger_smart_contract(**kwargs)
sign = tron.trx.sign(raw_tx["transaction"])
result = tron.trx.broadcast(sign)
CONSUMER_KEY = "*******"
CONSUMER_SECRET = "***"
ACCESS_KEY = "**"
ACCESS_SECRET = ****"
url = 'https://api.twitter.com/1.1/account/verify_credentials.json'
auth = OAuth1(CONSUMER_KEY, CONSUMER_SECRET, ACCESS_KEY, ACCESS_SECRET)
requests.get(url, auth=auth)
r = requests.get('https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name=coderFord&count=2',auth=auth)
res=r.json()
def id_al():
for item in res:
b=item['id']
print(b)
req_ya(b)
def req_ya(b):
params = {"ids": b}
ponse = requests.get("https://api.twitter.com/labs/1/tweets/metrics/private", params=params,auth=auth)
ak=ponse.json()
for itema in ak['data']:
for items in res:
k=itema['tweet']
p=itema['video']
data = {"created_at":items['created_at'],"text":items['text'],"tweet_id": itema['tweet_id'], "like_count": k['like_count'],"retweet_count": k['retweet_count'],"quote_count": k['quote_count'],"reply_count": k['reply_count'],"impression_count": k['impression_count'],"view_count":p['view_count']}
data_dict = json.dumps(data, indent = 4, sort_keys = True)
print(json.loads(data_dict))
collection.insert_one(data)
print(id_al())
Error: KeyError: 'video'
If the tweet I send is text, it doesn't send me the video view. And it gives an error.How can I control this
And I can't check because the type part of the tweet doesn't exist.
Response sample like this
https://developer.twitter.com/en/docs/labs/tweet-metrics/quick-start/get-tweets-metrics-private
You can first check there is 'video' in itema.keys() and then perform your logic inside this condition.
Replace this part,
for itema in ak['data']:
for items in res:
k=itema['tweet']
p=itema['video'] # here you are getting error.
with
for itema in ak['data']:
for items in res:
k=itema['tweet']
if 'video' in itema.keys():
p = itema['video']
Looking for some advice customizing requests for json style syntax.
json.dumps will format the like this:
{"tenantid": 1, "name": "NewRoleName", "users": "[djones]"}
But the servers wants something like this:
{"tenantid": 1, "name": "NewRoleName", "users": ["djones"]}
Any easy way to manipulate the double quotes to be placed with in the square brackets?
payload = {}
funcURL = "roles"
if name is not None: payload.update({'name': name})
if userStore is not None: payload.update({'userStore': userStore})
if users is not None: payload.update({'users': "[" + users + "]"})
if tenantid is not None: payload.update({'tenantid': tenantid})
if permissions is not None: payload.update({'userClaims': permissions})
print users
print payload
payload = json.dumps(payload)
return self._call_url('POST', funcURL, payload)
_call_url uses requests to send the payload out:
response = requests.request( methodType, baseURL + "/" + funcURL, data=payload, headers=self.headers, timeout=timeout, verify=False)
Have you tried:
if users is not None: payload.update({'users': users})
This should work if users is something like users = ["djones","mjones","todd"]
The square brackets denote a list. So you have to put users into a list:
funcURL = "roles"
payload = {}
if name is not None:
payload['name'] = name
if userStore is not None:
payload['userStore'] = userStore
if users is not None:
payload['users'] = [users]
if tenantid is not None:
payload['tenantid'] = tenantid
if permissions is not None:
payload['userClaims'] = permissions
payload = json.dumps(payload)
return self._call_url('POST', funcURL, payload)