Develop Reference

Api call using python and token_auth

"""
#Collects basic metrics from Matomo installation and returns a pandas dataframe
"""
token = os.getenv("token")
# Build url string
base_url = 'https://matomo.___.com/index.php?module=API'
site_num = '&idSite=1'
return_format = '&format=json'
period = '&period=day'
date_range = '&date=last30'
method = '&method=VisitsSummary.get'
token_string = "&token_auth=" + token
my_url = base_url + site_num + return_format + period + date_range + method + token_string
# send request for report
r = requests.get(my_url)
# parse and tidy collected data
data = pd.DataFrame(r.json()).T
data = data.reset_index()
data.columns = [
"date",
"uniq_visitors",
"users",
"visits",
"actions",
"visits_converted",
"bounces",
"sum_visit_length",
"max_actions",
"bounce_rate",
"actions_per_visit",
"avg_time_on_site",
]
return data
I am trying to get data from the matomo API using an auth_token and parameters by using above code but i am not able to access it and my url is not taking token code any one has idea how i can solve this

Given that you are using the request library, passing parameters and headers can be done using the following params in your get call:
r = requests.get(my_url, params=payload)
In the same way, an auth token is usually passed within headers:
r = requests.get(my_url, params=payload, headers=headers)
Using this format you can simply create a headers object which contains your token_auth and directly pass your parameters in a payload object:
headers = {'token_auth': token}
payload = {'module':'API', 'idSite':1, 'format':'json', 'period':'day', 'date':'last30', 'method':'VisitsSummary.get'}
Since you are now passing your parameters in you get request, there is no need to add them to the end of your url. Thus, your url should stay as https://matomo.___.com/index.php. These can then be used within your params and headers respectively. Please note that this assumes that the matomo API places the token_auth in its headers such as most APIs do. If this is not the case you could pass it directly within the params payload.
Here is a global overview:
token = os.getenv("token")
# Get url, headers and params
my_url = 'https://matomo.___.com/index.php'
payload = {'module':'API', 'idSite':1, 'format':'json', 'period':'day', 'date':'last30', 'method':'VisitsSummary.get'}
headers = {'token_auth': token}
# send request for report
r = requests.get(my_url, params=payload, headers=headers)
Note this answers your question specifically regarding the API call and not the processing after.

Formated output

I'm making a request in restapi to show application version, but the output i got is not the expected, i want to format this data.
from requests.api import request
from requests.packages.urllib3.exceptions import InsecureRequestWarning
import re
import requests
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
bipTmb='https://api1-foo'
bipAws='https://api2-foo'
def requestGet(bipTmb, bipAws):
cont = []
for urls in [bipTmb, bipAws]:
url = urls + '/mgmt/tm/sys?$top=4'
headers = {
'accept': '*/*',
'Content-Type': 'application/json',
}
response = requests.get(url, headers=headers, verify=False, auth=('auth', 'pass'))
data = response.json()
data = data['items']
reference = data[0]
version = reference['reference']
find = re.search("ver=.*",format(version))
content = (urls, find.group())
cont.append(content)
return cont
cont = requestGet(bipTmb, bipAws)
for item in cont:
treated_data = (item)
print(treated_data)
output:
[('https://api1-foo', "ver=13.1.3.6'}"), ('https://api2-foo', "ver=13.1.3.6'}")]
Formated output expected:
https://api1-foo ver=13.1.3.6,
https://api2-foo ver=13.1.3.6
How can i transform this data?

Try this.
print "\n".join(treated_data)

I reach the expected output looping through data.
for i in range(len(cont)):
print(cont[i])

How do I save API data into a csv file? Also how do I fix a traceback error?

This is part of my code. It successfully makes an API call and receives the data from the API endpoint. I am trying to save this JSON data into a csv file, but I am not sure how. Also the API data is printed as unicode instead of a string - how do I fix that?
I have tried these lines of code:
Trial 1:
with open('data.json', 'w', encoding = 'utf-8') as file:
json.dump(response, file, ensure_ascii=False, indent=4)
Trial 2:
data = response.text
file_csv = open("File.csv", "w")
writer = csv.writer(file_csv, delimiter = ' ')
for rows in basketball_data.split('\n'):
writer.writerow(rows)
Trial 3:
I tried using Pandas, but that didn't work as well. Any recommendations?
The code below is for fetching the API data which works.
basketball_data = " "
URL = "https://api.sportsdata.io/v3/nba/stats/json/PlayerGameStatsByDate/2020-FEB7" #API
# endpoint
# Dictionary to map HTTP authenticator and API key
Headers = {'Ocp-Apim-Subscription-Key': 'd22e84f5c1fa4f4ab47bf1419bd94221', 'accept':
"application/json", 'accept': "text/csv"}
response = requests.get(url = URL , headers = Headers) #get request parameters to
print(response.status_code) #Status code tells us if API call is successful
print(response.json()) #JSON object is returned

Try this:
import pandas as pd
import requests
url = "https://api.sportsdata.io/v3/nba/stats/json/PlayerGameStatsByDate/2020-FEB7"
headers = {
"Ocp-ApimKey": ";wld4221",
"accept": "application/json",
}
data = requests.get(url=url, headers=headers).json()
pd.DataFrame(data).to_csv("basketball_data.csv", index=False)
Output:

Here is one of the solutions given by the pandas documentation:
import requests
import pandas as pd
basketball_data = " "
URL = "https://api.sportsdata.io/v3/nba/stats/json/PlayerGameStatsByDate/2020-FEB7" #API
Headers = {'Ocp-Apim-Subscription-Key': 'd22e84f5c1fa4f4ab47bf1419bd94221', 'accept':
"application/json", 'accept': "text/csv"}
response = requests.get(url = URL , headers = Headers) #get request parameters to
print(response.status_code) #Status code tells us if API call is successful
print(response.json()) #JSON object is returned
df = pd.json_normalize(response.json())
df.to_csv("data1.csv")

Find specific keyword in API JSON response - Python

I am trying to fetch a JSON response of multiple issues from an API and I am able to get the response successfully. My next part which I want to perform is to fetch/print only those lines which have specific keywords as "moviepass" and "login" in JSON tag "body". Here is my code
import json
import requests
api_url = '***************************************'
headers = {'Content-Type': 'application/json',
'Authorization':'Basic **************************'}
response = requests.get(api_url, headers=headers)
#print(response.text)
words = ('moviepass', 'login')
def lookingfor(words):
data = response.text
for line in data:
for word in words:
match = re.findall(word, line['body'])
if match:
print((word, line[]))
lookingfor(words)
My JSON looks like:
[{"tags":["moviepass"],"assignee_name":null,"app_id":"*******","hs_user_id":"*******","title":"1234","redacted":false,"updated_at":1611753805497,"messages":[{"body":"moviepass - Not '
'sure if this is what you guys meant or not but here '
'haha.","created_at":********,"author":{"name":"abc","id":"*****","emails":["abc#qwerty.com"]},"origin":"end-user","id":"*********"}]

You dont need regular expression.You can use json_data['tags']
But if you want to use regular expression, you need to convert json to string by using
import json
json.dumps(json_obj) #returns same object but type of string.

Convert JSON response and parse it - it's a list of [nested] dicts. You can use Response.json() method, no need to import json.
import requests
api_url = '***************************************'
headers = {'Content-Type': 'application/json',
'Authorization':'Basic **************************'}
words = ('moviepass', 'login')
response = requests.get(api_url, headers=headers)
data = response.json()
for item in data:
if any(word in item.get('tags', []) for word in words):
print(item)

How to preserve the Ascii encoding type with the POST request in Python?

How do I send the ASCII encoded text via POST request in Python? The length of true_input I received via the POST is always different from the length I sent.
def insert_true_input(level, iteration, true_input):
url = master_url + "/insert_true_input?"
data = {'level': level, 'iteration': iteration, 'true_input': true_input}
headers = {'Content-Type': 'text/plain'}
res = requests.post(url, params=data, headers=headers).text
return res
The sample true_input that I want to send is directly from numpy.ndarray.tostring() and looks like
'\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00\x00#\x00\x00\x00\x00\x00\x00\x08#\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x007#\x00\x00\x00\x00\x00\xc0^#\x00\x00\x00\x00\x00\xc0^#\x00\x00\x00\x00\x00\xc0^#\x00\x00\x00\x00\x00\x00(#\x00\x00\x00\x00\x00\x00?#'

As explained in the comments, the null characters \x00 are not sendable in raw text. You have to encode them one way or another (URL encoded, Base64, json, etc.). But then the other side that will receive the request must be adapted to decode them accordingly.
Actually requests will use URL encoding automatically for the parameters passed in the query string, but I suspect that your java code is not able to decode them properly.
Please post your Java code for the receiving side to see what we can do.
Suggestions on python side, using base64:
import base64
def insert_true_input(level, iteration, true_input):
url = master_url + "/insert_true_input?"
data = {'level': level, 'iteration': iteration, 'true_input': base64.b64encode(true_input)}
res = requests.post(url, params=data, headers=headers).text
return res
Using json (requests will do the work for you if you use the json parameter to .post()):
def insert_true_input(level, iteration, true_input):
url = master_url + "/insert_true_input?"
data = {'level': level, 'iteration': iteration, 'true_input': true_input}
res = requests.post(url, json=data, headers=headers).text
return res

You have to encode your string using str.encode('ascii'):
def insert_true_input(level, iteration, true_input):
url = master_url + "/insert_true_input?"
data = {'level': level, 'iteration': iteration, 'true_input': true_input.encode('ascii')}
headers = {'Content-Type': 'text/plain'}
res = requests.post(url, params=data, headers=headers).text
return res