I have a web application route in flask. The route is supposed to give the user an analysis of their data after 30 minutes. To do this, they initiate the route. This triggers an API request that takes gets their initial data. The route then sleeps for 30 minutes, and then triggers the same exact API response. The initial data is subtracted from the final data and result the data that is 'attained' in that 30 minutes and is upserted to a database. An example of how it would look is below. I would like my users to be able to use the other features of the website, or more importantly even initiate another instance of this function, while this runs. My question is: is this an async problem or a multithread problem and why? I would like to note that my question is more conceptual then about the actual code.
#app.route("/data/", methods=['GET', 'POST', 'UPDATE'])
#login_required
def data():
Url = "https://data-api.com"
payload={}
headers = {
'Authorization': AUTH
}
response = requests.request("GET", Url, headers=headers, data=payload)
data_json = json.loads(response.text)
data1_init = data_json["a"]["b"]["c"]
data2_init = data_json["e"]["f"]["g"]
data3_init = data_json["h"]["i"]["j"]
sleep(1800)
Url = "https://data-api.com"
payload={}
headers = {
'Authorization': AUTH
}
response = requests.request("GET", Url, headers=headers, data=payload)
data_json = json.loads(response.text)
data1_final = data_json["a"]["b"]["c"]
data2_final = data_json["e"]["f"]["g"]
data3_final = data_json["h"]["i"]["j"]
data1 = data1_final - data1_initial
data2 = data2_final - data2_initial
data3 = data3_final - data3_initial
data_info = DBModel(data1=data1, data2=data2, data3=data3)
db.session.commit()
return render_template('loading.html')
def run_func():
data = { 'some': 'data', 'any': 'data' }
thr = Thread(target= *your_function*, args=[app, data])
thr.start()
return thr
You can use threading. Put your function name in place of "your_function", pass the function parameters in args and there you go. Just call the run_func() wherever you want to execute and it starts running in background in an other thread.
Related
I'm finding the Kraken Futures API confusing compared to other providers. Using a demo account I'm trying to make basic private requests and not working so far with authentication error. The code mainly comes from Kraken docs (non-futures)
Futures auth doc: https://support.kraken.com/hc/en-us/articles/360022635592-Generate-authentication-strings-REST-API-
api_sec = "MxA2FwIQxCxsfy2XDa4R8PwTjwLKjzT8GSOw+qOVuWGh3Lx6PtyW0f94J5XXKz9mP8bztRJSDQJVKBsHFicrDr/N"
api_url = "https://futures.kraken.com/derivatives/api/v3"
api_key = 'Y7kVv/hW0JWRRAhJtA8BuJkUX+E0gWmTL5NWf4lRPN8f+iYoJp9AoYwW'
def get_kraken_signature(urlpath, data, secret):
postdata = urllib.parse.urlencode(data)
encoded = (str(data['nonce']) + postdata).encode()
message = urlpath.encode() + hashlib.sha256(encoded).digest()
mac = hmac.new(base64.b64decode(secret), message, hashlib.sha256)
sigdigest = base64.b64encode(mac.digest())
return sigdigest.decode()
# Attaches auth headers and returns results of a get request
def kraken_request(uri_path, data, api_key, api_sec):
headers = {}
headers['API-Key'] = api_key
# get_kraken_signature() as defined in the 'Authentication' section
headers['API-Sign'] = get_kraken_signature(uri_path, data, api_sec)
req = requests.get((api_url + uri_path), headers=headers, data=data)
return req
# Construct the request and print the result
resp = kraken_request('/accounts', {
"nonce": str(int(1000*time.time()))
}, api_key, api_sec)
Output
{"result":"error","error":"authenticationError","serverTime":"2022-05-13T10:14:50.838Z"}
"""
#Collects basic metrics from Matomo installation and returns a pandas dataframe
"""
token = os.getenv("token")
# Build url string
base_url = 'https://matomo.___.com/index.php?module=API'
site_num = '&idSite=1'
return_format = '&format=json'
period = '&period=day'
date_range = '&date=last30'
method = '&method=VisitsSummary.get'
token_string = "&token_auth=" + token
my_url = base_url + site_num + return_format + period + date_range + method + token_string
# send request for report
r = requests.get(my_url)
# parse and tidy collected data
data = pd.DataFrame(r.json()).T
data = data.reset_index()
data.columns = [
"date",
"uniq_visitors",
"users",
"visits",
"actions",
"visits_converted",
"bounces",
"sum_visit_length",
"max_actions",
"bounce_rate",
"actions_per_visit",
"avg_time_on_site",
]
return data
I am trying to get data from the matomo API using an auth_token and parameters by using above code but i am not able to access it and my url is not taking token code any one has idea how i can solve this
Given that you are using the request library, passing parameters and headers can be done using the following params in your get call:
r = requests.get(my_url, params=payload)
In the same way, an auth token is usually passed within headers:
r = requests.get(my_url, params=payload, headers=headers)
Using this format you can simply create a headers object which contains your token_auth and directly pass your parameters in a payload object:
headers = {'token_auth': token}
payload = {'module':'API', 'idSite':1, 'format':'json', 'period':'day', 'date':'last30', 'method':'VisitsSummary.get'}
Since you are now passing your parameters in you get request, there is no need to add them to the end of your url. Thus, your url should stay as https://matomo.___.com/index.php. These can then be used within your params and headers respectively. Please note that this assumes that the matomo API places the token_auth in its headers such as most APIs do. If this is not the case you could pass it directly within the params payload.
Here is a global overview:
token = os.getenv("token")
# Get url, headers and params
my_url = 'https://matomo.___.com/index.php'
payload = {'module':'API', 'idSite':1, 'format':'json', 'period':'day', 'date':'last30', 'method':'VisitsSummary.get'}
headers = {'token_auth': token}
# send request for report
r = requests.get(my_url, params=payload, headers=headers)
Note this answers your question specifically regarding the API call and not the processing after.
i'm working on winrest api and since my session token will change from time to time i would like to ad it in a variable, when run my code i get a return request('get', url, params=params, **kwargs)
here the code :
sessionToken = 'session_token': 'gggg6gsl68l2vdim5fgggggg'}
headers = {
'App-Token': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
'Session-Token': sessionToken
}
response = requests.get(urlUnencryptedWorkstation, headers=headers)
I believe this is what you're looking for:
import requests as req
urlUnencryptedWorkstation = 'https://www.google.com' # Whatever the url is
sessionToken = 'gggg6gsl68l2vdim5fgggggg'
headers = {
'App-Token': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
'Session-Token': sessionToken
}
res = req.get(urlUnencryptedWorkstation, headers=headers)
print(res.status_code, res.text)
Hope this helps :) Cheers!
This is the url I'm trying to parse.
url = 'https://www.ncsl.org/research/energy/energy-legislation-tracking-database.aspx'
I'm trying to select "all state" and "all topic" option, and retrieve the search output.
def Generate_params(tn=22, sn=55, searchyear=2008, vr=vrstring):
topicstring = "dnn$ctr85406$StateNetDB$ckBxTopics$"
statestring = 'dnn$ctr85406$StateNetDB$ckBxStates$'
params = {}
params['dnn$ctr85406$StateNetDB$ckBxAllTopics']='on'
params['dnn$ctr85406$StateNetDB$ckBxAllStates']='on'
for t in range(tn+1):
topic_n = topicstring+"{}".format(t)
params[topic_n]='on'
for s in range(sn+1):
state_n = statestring+"{}".format(s)
params[state_n]='on'
# vrstring is a string parameter I have outside the function.
params['__VIEWSTATE']=vrstring
params['dnn$ctr85406$StateNetDB$ddlYear']=str(searchyear)
params['dnn$ctr85406$StateNetDB$btnSearch']='Search'
params['__VIEWSTATEGENERATOR']='CA0B0334'
return params
from requests import Request, Session
post_params=Generate_params()
session = Session()
response = session.post(url, headers=headers,data=post_params)
But the returned response don't have the search output. What did I do wrong?
I am using this API to list users. One of the parameters I could specify is a team id which is placed in an array. When I try to specify a team id it doesn't work when I put it in the payload, but it works when I change the url to include the team id.
This is the API reference: https://api-reference.pagerduty.com/#!/Users/get_users
Here is what I am basing my code off of: https://github.com/PagerDuty/API_Python_Examples/blob/master/REST_API_v2/Users/list_users.py
This is my code when I try to specify team id in the payload. It doesn't work like this for some reason, but it works when I change the url to url = 'https://api.pagerduty.com/users?team_ids%5B%5D=TEAMID&team_ids%5B%5D=' where in TEAMID I have an actual team id.
with open('config/config.json') as f:
config = json.load(f)
API_KEY = config['API_KEY']
TEAM_IDS = ['TEAMID']
def list_users():
url = 'https://api.pagerduty.com/users'
headers = {
'Accept': 'application/vnd.pagerduty+json;version=2',
'Authorization': 'Token token={token}'.format(token=API_KEY)
}
payload = {
'team_ids[]': TEAM_IDS
}
r = requests.get(url, headers=headers)
result = []
if r.status_code == 200:
# loops for each user and retrieves their email
result = [user['email'] for user in r.json()['users']]
return result
else:
return None
I want to get this work by listing team id's in the array and sending it in the payload so that I can list more than one team id and not clutter them all in the url.
Looks like you just need something like this
payload = {
'team_ids[]': TEAM_IDS
}
r = requests.get(url, headers=headers, params=payload)