I need to fetch KPIs (revenue change, total cost etc.) from a PBI report.
So far I have tried calling few PowerBI Rest APIs like this-
url_groups = 'https://api.powerbi.com/v1.0/myorg/reports'
header = {'Content-Type':'application/json','Authorization': f'Bearer {access_token}'}
api_out = requests.get(url=url_groups, headers=header)
However, only the report name, type , urls etc. are getting returned in the output, not the KPIs. Any help would be highly appreciated.
PowerBI has many API endpoints (https://learn.microsoft.com/en-us/rest/api/power-bi/). You are using the Reports endpoint, that's why you are getting the output you see.
There are other endpoints you might be interested in checking: Dashboard endpoint (https://api.powerbi.com/v1.0/myorg/dashboards/{dashboardId}) and Dataset one (https://api.powerbi.com/v1.0/myorg/datasets/{datasetId}).
The only thing is that in the documentation looks like there is no specific place where you can get your data from by using the API.
It might be necessary to schedule csv exports from PowerBI and then reading those in Python.
To get at the data in the report you need to send a DAX query to the report's Dataset using the ExecuteQueries API.
In Power BI Desktop you can use the Performance Analyzer to see the DAX query sent by a visual, instead of writing it from scratch.
Related
Given an existing report on SalesForce that I created, can I update the content of the report using simple_salesforce package in python?
# A Pseudo code to get the expected result
from simple_salesforce import Salesforce
sf = Salesforce(username='my_user_name',
password='my_password',
security_token='my_security_token')
export_url = "link to the report I want to change"
session = requests.Session()
response = requests.get(export_url,
headers=sf.headers,
cookies={'sid': sf.session_id})
# this part is pseudo-code, for instance, I want to change a column of the report, I change it and then upload the new version as the current report
response.update(response, data)
You're bit out of Simple's comfort zone. You don't manipulate data, you don't even use restful call for "normal" API operations. You just (ab)use simple's login functionality to get the session id and pretend you're a browser. So you can experiment with requests below in Postman for example and then go back to your Python program.
I wonder if you're overcomplicating it. Are you sure you absolutely need a report?
Maybe it can be done with a normal SOQL query?
There's proper API for reports and a way to filter a report on the fly. It won't save the changes to the report. But that API will return results as JSON, you probably coded your stuff around CSVs.
If you need to actually save changes - there are proper Tooling API/Metadata API calls to fetch, modify and deploy back the report definition as XML file. Again - simple might not be best tool for the task, sfdx might fare better.
I suspect you'll ignore all these and stick to faking a browser. Google around for "url hacking", for example here. As it becomes more config than code issue - you can find some good stuff in dedicated sister site that has more admins than stackoverflow. For example https://salesforce.stackexchange.com/q/110219/799
get_smart_campaign (Marketo API)
Hi all,
The picture above is currently the process I am using to extract Smart_Campaign data using the .execute() method. It returns all the smart campaigns listed up to 200 objects. I have over 3000 objects that I need to get but the API limits are complicating the process.
I read about the Marketo Bulk Extract API but I am not sure if I will be able to extract the smart_campaign id and name.
Any help would be greatly appreciated.
I am trying to access historical weather data from an API. I obtained the API key from here: https://www.visualcrossing.com/weather/weather-data-services#/timeline
I am trying this but I keep getting an Error 404. I am not sure if this is because of a problem with the API or my code.
import requests
r = requests.get("https://weather.visualcrossing.com/VisualCrossingWebServices/rest/services/timeline/London,UK/2021-01-01/2021-03-28?key=AXSSS")
print(r)
Documentation: https://www.visualcrossing.com/resources/documentation/weather-api/timeline-weather-api/
How can I obtain the data?
i tested the site you gave and created a account and api key to get London whether data, you can use it too
Code :
import requests
r = requests.get("https://weather.visualcrossing.com/VisualCrossingWebServices/rest/services/timeline/London?unitGroup=metric&key=PPKBBJ7637X5SNDUG6HZA23X7")
print(r)
Output :
<Response [200]>
now you can access data by json() method too:
print(r.json())
the output is so Huge, but your problem is 2 things:
1-API key is not correct (i tested)
2-You should buy premium plan
for get a range of dates url will be like this:
https://weather.visualcrossing.com/VisualCrossingWebServices/rest/services/timeline/London/2021-1-1/2021-1-5?unitGroup=us&key=PPKBBJ7637X5SNDUG6HZA23X7
and the date range you give , has too much row per request, you should buy premium plan
Otherwise you will get this error in their own website:
Your plan allows up to 100 rows per request. This query will return (yyy) rows. Please smaller date range or fewer locations.
Our paid plans offer increased query limits
If you don`t want to pay for paid plan you can use this link to use github public apis, a lot of free and without api key you can use for it
Github public apis
I am trying to get data from a REST API into BigQuery on the Google Cloud Platform (GCP). What is the best way to achieve that (without using any third party tools such as Funnel.io or Supermetrics)?
Most tutorials I could find suggest to write the data as CSV files to Cloud Storage and then use DataFlow to load the data into BigQuery. This however seems to be a bit cumbersome. There should be a way to do that without the intermediate step to write to CSV. Can this be achieved (within GCP) and if so, what is the best way?
PS: If the size of the data is relevant for the answer: I'm trying to load a total of about 10,000 rows of data (one-off) with about 100 new columns coming in every day - ideally updating every hour.
Following up on the hint by #Kolban above, loading data from an API into BigQuery without using third party tools and without writing an intermediate file to Google Cloud Storage is possible, and indeed quite simple, by "streaming" data into BigQuery:
rows_to_insert = [(u"Phred Phlyntstone", 32), (u"Wylma Phlyntstone", 29)]
errors = client.insert_rows(table, rows_to_insert) # Make an API request.
if errors == []:
print("New rows have been added.")
(From the BQ documentation)
In order to prepare the JSON data, it has to be turned into tuples. Here's an excerpt from my code to achieve this:
# Turn JSON into tuples
data_tuples = []
for key,value in resp_json[product_id].items():
data_tuples.append((
value["product_id"],
value["downloads"]
)
)
# Insert into BQ
errors = client.insert_rows(table, data_tuples)
if errors == []:
print("New rows have been added.")
else:
print(errors)
According to the documentation:
Currently, you can load data into BigQuery only from Cloud Storage or
a readable data source (such as your local machine).
Therefore, unless you are loading Datastore or Firestore exports, it is required that the files are in Google Cloud Storage. There are the following available readable formats from GCS:
Avro
CSV
JSON (newline delimited only)
ORC
Parquet
Datastore exports
Firestore exports
You should be aware of the limitations for each format. In addition, there are also limitations for load jobs, they are described here.
I would advice you to fetch the data from your Rest API, in one of the readable formats, store it in Google Cloud Storage then use Google Transfer Service to load it into BigQuery. Thus, it would not be necessary to use DataFlow.
Cloud Storage Transfer is used to schedule recurring data loads directly into BigQuery. According to the documentation, the minimum load interval is 1 hour, which I believe suits your need. You can read more about this service here.
I hope it helps.
Im currently trying to retrieve data from the google trends site directly into python. By inspecting the "download csv"-button I was able to extract the underlying url of the file, which looks like this:
https://trends.google.com/trends/api/widgetdata/multiline/csv?req=%7B%22time%22%3A%222018-08-30%202019-08-30%22%2C%22resolution%22%3A%22WEEK%22%2C%22locale%22%3A%22de%22%2C%22comparisonItem%22%3A%5B%7B%22geo%22%3A%7B%7D%2C%22complexKeywordsRestriction%22%3A%7B%22keyword%22%3A%5B%7B%22type%22%3A%22BROAD%22%2C%22value%22%3A%22trump%22%7D%5D%7D%7D%5D%2C%22requestOptions%22%3A%7B%22property%22%3A%22%22%2C%22backend%22%3A%22IZG%22%2C%22category%22%3A0%7D%7D&token=APP6_UEAAAAAXWrEbGVLW-ssfeQvOJgr9938DRgYO1sm&tz=-120
Unquoted :
https://trends.google.com/trends/api/widgetdata/multiline/csv?req={"time":"2018-08-30 2019-08-30","resolution":"WEEK","locale":"de","comparisonItem":[{"geo":{},"complexKeywordsRestriction":{"keyword":[{"type":"BROAD","value":"trump"}]}}],"requestOptions":{"property":"","backend":"IZG","category":0}}&token=APP6_UEAAAAAXWrEbGVLW-ssfeQvOJgr9938DRgYO1sm&tz=-120
I can now easily get this csv into a pandas dataframe. Ideally, I would now just manipulate the url in order to make custom requests and load new data. The problem I have is that I cannot use the same token parameter, since it is somehow generated newly for each individual csv-request. I think that the answer from shaochuancs in Origin of tokens in Google trends API call describes the problem as I am facing it. Can anyone explain how I can request this token which I could then use for the second request ( the actual csv download )? :/