Develop Reference

Get and parse data from the last message of a specific sender with python and Gmail API

Everyday, a sender "sender#sender.com" send me a message with a number inside.
I need to save this number everyday.
I want to write a python script with gmail API to get data from last mail from this sender, and then parse it.
I followed the Gmail API "Quickstart Guide" : here
I also check the page about User.message : here
However, I don't understand how to synchronize all of this to get the data.
Could someone explain me the process ?

If you where you able to complete the Gmail API quickstart, then you already have a GCP project, credentials and have authorized some Gmail API scopes for you app.
The above is the first step (being able to authenticate and be allowed to make requests for the API scope you need).
Since you need to pass a message's Id as a parameter for Users.messages.get you need to first retrieve it using listing messages for example.
So the next step is to make a request to Users.messages.list to list all messages from a user.
You could use the query (q) parameter to filter the messages by user like: q="from:someuser#example.com is:unread".
This will return a list of messages from someuser#example.com that are unread.
Try things out in the API explorer sidebar from the documentation until you have defined the request as you want, and then implement it into you app.

As aerials said.
users().messages().list(userId='me',q=("<parameters>"))).execute()
The above code will fulfill the exact same function as typing in a search request on the gmail website. You dont actually have to worry about labels or anything if you are operating at a small scale. Just follow the same syntax as the search bar on gmail.
However, I am not sure about the usage quotas on the q parameter for list. It may be more expensive for a bigger scale operation to use the q parameter instead of using the other api methods.

Serialisation and timed web tokens in python flask

Hi I am struggling to understand the following example.
I want to add functionality for a user to reset password for my website.
As a lot of sites do I want to send a token to the users email that will let them reset their password.
I am following a guide that suggests using a python module called itsdangerous.
I have been given the following code as a simple example from the tutorial to understand how the module works before deploying to my website:
from itsdangerous import TimedJSONWebSignatureSerializer as Serializer
s = Serializer('secret_key',30)
token = s.dumps({'usr_id': 1}).decode('utf-8')
s.loads(token)
Now if I run this here is what happens:
I use s to create a token that allows me to take a dictionary {'usr_id':1} then if I run s.loads(token) within 30 seconds I can get this dictionary {'usr_id':1} back otherwise I get an error.
Can anyone explain (in a simple way for a beginner) what is going on here?
I don't really understand what is happening and I don't see what the secret_key argument to the Serializer is doing?
Also if someone could explain how this kind of code can help me with allowing users to get an email to reset their password that would be great. Thanks!

So this kind of serialization provided by itsdangerous library is JSON Web Token based. In order to create a JSON Web Token you need a secret key, which is a signature to certify that the information tokenized was provided by you and can only be edited with this signature.
A JSON Web Token can be read by anyone but can only be edited by someone who knows the secret key - so you shouldn't tokenize sensitive information like a password, but a user id is ok, which alongside a expiry time set is just enough to check its identity. You must hide your secret key and keep it safe.
A good usage in your case would be to send a expiring token - let's say one day - to the user email, to proof it has authorization to change the password. And after one day it'll be invalid, so you won't compromise your system.
What is the JSON Web Token structure?
In its compact form, JSON Web Tokens consist of three parts separated by dots (.), which are:
Header: The header typically consists of two parts: the type of the token, which is JWT, and the signing algorithm being used, such as HMAC SHA256 or RSA.
Payload: The information and the expiry time is held here.
Signature: The signature is used to verify the message wasn't changed along the way, and, in the case of tokens signed with a secret key, it can also verify that the sender of the JWT is who it says it is.
Therefore, a JWT typically looks like the following. xxxxx.yyyyy.zzzzz
You can play around seeing what is inside a JWT by pasting it on the link below:
https://jwt.io/
More info on:
https://jwt.io/introduction/

boto3 get available actions per service

I want to programatically get all the actions a user is allowed to do across aws services.
I've tried to fiddle with simulate_principal_policy but it seems this method expects a list of all actions, and I don't want to maintain a hard-coded list.
I also tried to call it with iam:* for example and got a generic 'implicitDeny' response so I know the user is not permitted all the actions but I require a higher granularity of actions.
Any ideas as to how do I get the action list dynamically?
Thanks!

To start with, there is no programmatic way to retrieve all possible actions (regardless of whether they are permitted to use an action).
You would need to construct a list of possible actions before checking the security. As an example, the boto3 SDK for Python contains an internal list of commands that it uses to validate commands before sending them to AWS.
Once you have a particular action, you could use Access the Policy Simulator API to validate whether a given user would be allowed to make a particular API call. This is much easier than attempting to parse the various Allow and Deny permissions associated with a given user.
However, a call might be denied based upon the specific parameters of the call. For example, a user might have permissions to terminate any Amazon EC2 instance that has a particular tag, but cannot terminate all instances. To correctly test this, an InstanceId would need to be provided to the simulation.
Also, permissions might be restricted by IP Address and even Time of Day. Thus, while a user would have permission to call an Action, where and when they do it will have an impact on whether the Action is permitted.
Bottom line: It ain't easy! AWS will validate permissions at the time of the call. Use the Policy Simulator to obtain similar validation results.

I am surprised no one has answered this question correctly. Here is code that uses boto3 that addresses the OP's question directly:
import boto3
session = boto3.Session('us-east-1')
for service in session.get_available_services ():
service_client = session.client (service)
print (service)
print (service_client.meta.service_model.operation_names)
IAM, however, is a special case as it won't be listed in the get_available_services() call above:
IAM = session.client ('iam')
print ('iam')
print (IAM.meta.service_model.operation_names)

Simple Access API (Developer Key) with Google Cloud Endpoint (Python)

Is there a way to use Simple Access API (Developer Key) instead of oAuth2 key with Google Cloud Endpoint?

Extra fields in your protorpc request object that aren't part of the definition are still stored with the request.
If you wanted to use a key field as a query parameter, you could access it via
request.get_unrecognized_field_info('key')
even if key is not a field in your message definition.
This is done in users_id_token.py (the Auth part of the endpoints library) to allow sending bearer_token or access_token as query parameters instead of as header values.
Unfortunately, the nice quota checking and other associated pieces that a "Simple API Access" key gives are not readily available. However, you could issue your own keys and manually check a key against your list and potentially check against quotas that you have defined.

For those looking to use #bossylobster's answer in Java, use the the SO Answer here:
Getting raw HTTP Data (Headers, Cookies, etc) in Google Cloud Endpoints
P.S.
I tried to make this a comment in #bossylobster's answer, but I don't have the reputation to do that. Feel free to clean up this answer so that other's can follow the path

Security measures for controlling access to web-services/API

I have a webapp with some functionality that I'd like to be made accessible via an API or webservice. My problem is that I want to control where my API can be accessed from, that is, I only want the apps that I create or approve to have access to my API. The API would be a web-based REST service. My users do not login, so there is no authentication of the user. The most likely use case, and the one to work with now, is that the app will be an iOS app. The API will be coded with django/python.
Given that it is not possible to view the source-code of an iOS app (I think, correct me if I'm wrong), my initial thinking is that I could just have some secret key that is passed in as a parameter to the API. However, anyone listening in on the connection would be able to see this key and just use it from anywhere else in the world.
My next though is that I could add a prior step. Before the app gets to use API it must pass a challenge. On first request, my API will create a random phrase and encrypt it with some secret key (RSA?). The original, unencrypted phrase will be sent to the app, which must also encrypt the phrase with the same secret key and send back the encrypted text with their request. If the encryptions match up, the app gets access but if not they don't.
My question is: Does this sound like a good methodology and, if so, are there any existing libraries out there that can do these types of things? I'll be working in python server-side and objective-c client side for now.

The easiest solution would be IP whitelisting if you expect the API consumer to be requesting from the same IP all the time.
If you want to support the ability to 'authenticate' from anywhere, then you're on the right track; it would be a lot easier to share an encryption method and then requesting users send a request with an encrypted api consumer handle / password / request date. Your server decodes the encrypted value, checks the handle / password against a whitelist you control, and then verifies that the request date is within some timeframe that is valid; aka, if the request date wasnt within 1 minute ago, deny the request (that way, someone intercepts the encrypted value, it's only valid for 1 minute). The encrypted value keeps changing because the request time is changing, so the key for authentication keeps changing.
That's my take anyways.

In addition to Tejs' answer, one known way is to bind the Product ID of the OS (or another unique ID of the client machine) with a specific password that is known to the user, but not stored in the application, and use those to encrypt/decrypt messages. So for example, when you get the unique no. of the machine from the user, you supply him with password, such that they complete each other to create a seed X for RC4 for example and use it for encryption / decryption. this seed X is known to the server as well, and it also use it for encryption / decryption. I won't tell you this is the best way of course, but assuming you trust the end-user (but not necessarily any one who has access to this computer), it seems sufficient to me.
Also, a good python library for cryptography is pycrypto

On first request, my API will create a random phrase and encrypt it with some secret key (RSA?)
Read up on http://en.wikipedia.org/wiki/Digital_signature to see the whole story behind this kind of handshake.
Then read up on
http://en.wikipedia.org/wiki/Lamport_signature
And it's cousin
http://en.wikipedia.org/wiki/Hash_tree
The idea is that a signature can be used once. Compromise of the signature in your iOS code doesn't matter since it's a one-use-only key.
If you use a hash tree, you can get a number of valid signatures by building a hash tree over the iOS binary file itself. The server and the iOS app both have access to the same
file being used to generate the signatures.