Develop Reference

is there a direct explanation for each google-ads.yaml item?

I am looking for collect data from Google ADS API into GCP by using Python scripts and it requires to fill these items for authentication in the google-ads.yaml file:
developer_token:
client_id:
client_secret:
refresh_token:
login_customer_id:
I was able to fill these items by asking people in my company or generating it with google python scripts in GitHub but I need to understand the role of each, the docs seems to be disperse with a long learning path.

You can follow this guidebook to make your google-ads.yaml file. And for the sample role you provided, below are the definitions of each but you can check this sample template for more details about it.
Developer token
A developer token is required when making requests to the Google Ads API regardless of whether you're using the OAuth2 or Service Account configurations. To obtain a developer token see: https://developers.google.com/google-ads/api/docs/first-call/dev-token
developer_token: INSERT_DEVELOPER_TOKEN_HERE
OAuth2 configuration
The below configuration parameters are used to authenticate using the recommended OAuth2 flow. For more information on authenticating with OAuth2 see:
https://developers.google.com/google-ads/api/docs/oauth/overview
client_id: INSERT_OAUTH2_CLIENT_ID_HERE
client_secret: INSERT_OAUTH2_CLIENT_SECRET_HERE
refresh_token: INSERT_REFRESH_TOKEN_HERE
Login customer ID configuration
Required for manager accounts only: Specify the login customer ID used to authenticate API calls. This will be the customer ID of the authenticated manager account. It should be set without dashes, for example: 1234567890 instead of 123-456-7890. You can also specify this later in code if your application uses multiple manager account + OAuth pairs.
login_customer_id: INSERT_LOGIN_CUSTOMER_ID_HERE

Reading Outlook calendar events in Python

I need to get the events for the current day from a personal Outlook calendar. I have found next to no feasible resources online besides maybe Microsoft's tutorial (https://learn.microsoft.com/en-us/graph/tutorials/python), but I do not want to build a Django app. Can anyone provide some other resources?
also: I have seen a lot of ppl calling APIs by using a GET <url> command. I cannot for the life of me understand how or where you can use this? Am I missing something crucial when it comes to using APIs?

First you should know that if you wanna call ms graph api, you need to get the access token first and add it to the request header like screenshot below. What I showed in the screenshot is create calendar events but they're similar. Therefore, you can't avoid to generate the token.
Then there're 2 ways lie in front of you, if you are composing a web app, then you can follow this section to find a suitable sample for you, and if you are composing a daemon application, that means you need to use clientcredentialflow here and you may refer to this section.
Anyway, whatever you use SDK or sending http request to call the api, you all need to choose a suitable flow to obtain access token.

For this purpose without using Microsoft Graph API via request in python, there is a PyPI package named O365.
By the following procedure you can easily read a Microsoft calendar:
install the package: pip install O365
register an application in the Microsoft Azure console and keep the application (client) id as well as client secret — this article can help you up.
check the signInAudience, it should be AzureADandPersonalMicrosoftAccount not PersonalMicrosoftAccount within Microsft Azure Manifest, otherwise, you can edit that.
next you should set delegated permission to what scopes you want, in your case it's Calendars.Read. Here's a snapshot of my configuration in Azure:
Now it's time to dive into the code:
from O365 import Account
CLIENT_ID = "xxx"
CLIENT_SECRET = "xxx"
credentials = (CLIENT_ID, CLIENT_SECRET)
scopes = ['Calendars.Read']
account = Account(credentials)
if not account.is_authenticated:
account.authenticate(scopes=scopes)
print('Authenticated!')
schedule = account.schedule()
calendar = schedule.get_default_calendar()
events = calendar.get_events(include_recurring=False)
for event in events:
print(event)

Azure python SDK ComputerManagementClient error

I get an error when trying to deallocate a virtual machine with the Python SDK for Azure.
Basically I try something like:
credentials = ServicePrincipalCredentials(client_id, secret, tenant)
compute_client = ComputeManagementClient(credentials, subscription_id, '2015-05-01-preview')
compute_client.virtual_machines.deallocate(resource_group_name, vm_name)
pprint (result.result())
-> exception:
msrestazure.azure_exceptions.CloudError: Azure Error: AuthorizationFailed
Message: The client '<some client UUID>' with object id '<same client UUID>' does not have authorization to perform action 'Microsoft.Compute/virtualMachines/deallocate/action' over scope '/subscriptions/<our subscription UUID>/resourceGroups/<resource-group>/providers/Microsoft.Compute/virtualMachines/<our-machine>'.
What I don't understand is that the error message contains an unknown client UUID that I have not used in the credentials.
Python is version 2.7.13 and the SDK version was from yesterday.
What I guess I need is a registration for an Application, which I did to get the information for the credentials. I am not quite sure which exact permission(s) I need to register for the application with IAM. For adding an access entry I can only pick existing users, but not an application.
So is there any programmatic way to find out which permissions are required for an action and which permissions our client application has?
Thanks!

As #GauravMantri & #LaurentMazuel said, the issue was caused by not assign role/permission to a service principal. I had answered another SO thread Cannot list image publishers from Azure java SDK, which is similar with yours.
There are two ways to resolve the issue, which include using Azure CLI & doing these operations on Azure portal, please see the details of my answer for the first, and I update below for the second way which is old.
And for you want to find out these permissions programmatically, you can refer to the REST API Role Definition List to get all role definitions that are applicable at scope and above, or refer to Azure Python SDK Authentication Management to do it via the code authorization_client.role_definitions.list(scope).
Hope it helps.

Thank you all for your answers! The best recipe for creating an application and to register it with the right role - Virtual Machine Contributor - is presented indeed on https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal
The main issue I had was that there is a bug in the adding a role within IAM. I use add. I select "Virtual Machine Contributor". With "Select" I get presented a list of users, but not the application that I have created for this purpose. Entering the first few letters of the name of my application will give a filtered output that includes my application this time though. Registration is then finished and things can proceed.

Instagram api does not return data

The instagram api returs this when triying to get latest post by hashtag:
{"pagination":{"deprecation_warning":"next_max_id and min_id are deprecated for this endpoint; use min_tag_id and max_tag_id instead"},"meta":{"code":200},"data":[]}
This is the url I use for the query:
https://api.instagram.com/v1/tags/paris/media/recent?access_token=ACCESTOKENHERE
I tried using someone else access token and it works well. What is causing this?

I'm having the same issue. It seems that Instagram apps in "Sandbox Mode", i.e. those created after 17 November do not have access to public_content.
This is why using an access code of another app (presumably one that isn't in Sandbox mode) works.

For apps registered after Nov 17, 2015. A few endpoints return different data in Sandbox mode. The endpoint you are using is also one of them. Unless your app goes live, you can't fetch public content
As another example, let's consider an endpoint that returns a list of media: /tags/{tag-name}/media/recent. The response returned by this endpoint will contain only media with the given tag, as expected. But instead of returning media from any public Instagram user, it will return only media that belongs to your sandbox users, restricted to the last 20 for each user.
Source: http://instagram.com/developer/sandbox/

I'm having the same problem with PHP (language doesn't matter by the way). I saw on the Sandbox page these 3 conditions:
Apps in sandbox are restricted to 10 users
Data is restricted to the 10 users and the 20 most recent media from each of those users
Reduced API rate limits
You can only get the data from sandbox users, whom you can invite via the Manage Clients section.
Visit Manage Clients.
Click edit for your app.
Go to the Sandbox tab.
Add some users to invite.
Get them to accept the invitation and auth (make sure you have the scopes set up properly!).
Rejoice in the glory of their data.

Use an access_token with permission public_content. Just direct the user to your authorization URL with scopes:
https://api.instagram.com/oauth/authorize/?client_id=CLIENT-ID&redirect_uri=REDIRECT-URI&response_type=code&scope=basic+public_content
to get the permission.
Read more about Login Permissions (Scopes).

SPA (Ember) authentication vs backend

I am trying to wrap my head around implementing proper authentication in an SPA with a backend.
I am looking at 2 different frontends:
1. SPA with Ember.js
2. Mobile application
I am trying to design a backend that serves both (either in Rails or in Python, have not decided yet). I want the authentication to take place either via Google or Facebook, i.e. I do not need to maintain separate user registrations but I still need to maintain users as I want to be able to 'merge' authentications from Google and Facebook at the end of the day. I also want the backend to be fully stateless for scalability reasons. Also I would prefer to do as much as possible on the client side to spare the load from the backend.
My questions are as follows:
I will request an authentication token in the SPA or in the mobile app. Should I convert this to an access token on the server side or on the client side? I need to authorize backend requests and I need to do that statelessly. My idea was to do everything on the frontend side and pass along the access token to each and every backend request, to be validated on the server side as well but I am not sure if this is an efficient approach.
I want to be able to merge Google and Facebook authentications, what are the best practices for that? I was planning to maintain an user registry on the server side and check the email addresses coming from the authorized backend requests and merge the users if there is a match in email addresses. What are the best practices and are there any libraries supporting this in Python/Flask or in Ruby or Rails?
Thanks!

I'm not really sure what do you mean by 'stateless'. You obviously need some database to store user's data (otherwise you don't need backend at all). So the database is your state. HTTP protocol is stateless by definition, so you can't really be very stateful by other means than storing data in DB.
I will request an authentication token in the SPA or in the mobile app. Should I convert this to an access token on the server side or on the client side?
If you don't need to use Google/Facebook on behalf of your users (and your wording suggests that you don't), you don't need to convert auth_token to server_token at all.
You just need to call Google/Facebook API with that (Ruby has libraries for both, so it's basically a one line of code) and get social network's user ID and user email.
Then you save that ID+email in your database and give your internal server token (just random string) to your user. You generate that string yourself and give it to the client.
If user logs in from another device (i.e. it gives you auth_token with which you find out that user's email belongs to one of already-registered users), you either return existing internal token, or generate new one and bind it to the existing user (depends on what you prioritize – high security of simplicity of implementation/maintenance).
I want to be able to merge Google and Facebook authentications, what are the best practices for that?
Facebook guarantees that if it gives you user email, then it's ensured that that email belongs to the given user. Google, obviously, does the same. So you just merge them by emails.
You don't need some special libraries for that, as it is simple operation with you code on the language of your choice.
I'd organize all the things in database in the following manner:
Users table
id
email
Authentications table
user_id
email
social_uid # facebook number or google email
social_network # string, 'facebook' or 'google'
device # user agent, e.g. 'android'
ip # last login IP address
token # internal token
When user logs in, Authentication object is created. If there is no user with such email, the user is created. If there is a user, it's bind to the authentication object (both via user_id field).
Notes on access tokens
If you do plan to interact with social network (in other ways than just authenticating user), you should exchange auth_token for server_token. server_token is a 'permanent' (well, kind of) authorization token for accessing APIs of social network, whereas auth_token has a very limited lifespan (and some API calls may be restricted if you didn't obtain server_token).
Still, server_token can expire (or user can recall their access for your application), so you should plan ahead to detect that situation and re-acquire token/authorization if needed.
Key points when building Rails app
In Rails, in order to create tables, you need to write migrations:
gem install rails
rails new my_project
cd my_project
rails generate migration create_users
rails generate migration create_authentications
This will generate project folder structure and two migration files, which you need to fill in:
# db/migrate/xxx_create_users.rb
def change
create_table :users do |t|
t.string :email
end
end
# db/migrate/xxx_create_authentications.rb
def change
create_table :authentications do |t|
t.integer :user_id
t.index :user_id
t.string :social_uid
# etc., all other fields
# ...
end
end
Then you generate 'models' to handle database-related manipulations:
# app/models/user.rb
class User < ActiveRecord::Base
has_many :authentications
end
# app/models/authentication.rb
class Authentication < ActiveRecord::Base
belongs_to :user
before_create :set_token
after_commit :create_user_if_needed
private
def set_token
self.token = SecureRandom.urlsafe_base64(20)
end
def create_user_if_needed
unless self.user.present?
self.user.create(email: self.email)
end
end
end
And write 'controller' to handle request from user with a single method inside of it:
# app/controllers/login_controller.rb
class LoginController < ActionController
# Login via Facebook method
def facebook
token = params.require(:auth_token)
# we will use Koala gem
profile = Koala::Facebook::API.new(token).get_object('me', fields: 'email')
# user is created automatically by model
authentication = Authentication.where(social_network: 'facebook', social_uid: profile['id'], email: profile['email']).first_or_create
authentication.update(...) # set ip and device
render json: {token: authentication.token}
end
# This one you'll have to write yourself as an exercise :)
def google
end
end
Of course, you will need to setup routes for your action:
# config/routes.rb
post "login/:action", controller: 'login'
And add Koala (or whatever else you'll use to manage external APIs for which good Ruby packages already exist) to Gemfile:
# Gemfile
gem 'koala'
Then in your terminal run:
bundle install
rails server
And your app is up and running. Well, you'll need to setup your Facebook and Google applications first, get developer keys, and authorize localhost to accept auth_tokens.
Basically, that's it.