Develop Reference

Fetching URL from a redirected target using Python

I'm building a Twitch chat-bot, integrating some Spotify features using Spotipy library.
The goal behind the implementation is to achieve full-automated Spotipfy API Authentication for the bot.
How the Spotify API and Spotipy library work is, an authorization token is needed first in order to do anything over Spotify-end. So that's why, whenever the bot is initially run over my VPS, it prompts me to copy a URL from the console, locate it on a browser to wait for its redirect and paste on the console the redirected URL including the desired token. That's how the authentication object retrieves the token data.
To automate this process, I've seen several solutions via Flask or Django.
Django implementation would be useful for me, since I also have Django environment active on the same VPS, except that Django environment runs on Python 2.7 while my Twitch chat-bot runs on a separate Python 3.6 environment. Hence, I would like to keep them separate unless there is no way to implement such automation without listening redirects over Django, Flask or any other web-framework. Unfortunately, my bot can only run on Python 3.6 or higher.
I'm specifically curious if there is any built-in function or a lightweight library to handle such operation.
The function which I'm using to fetch Spotify Auth token is:
def fetchSpotiToken():
global spotiToken, spoti
spotiToken = spotifyAuth.get_cached_token()
if not spotiToken:
spAuthURL = spotifyAuth.get_authorize_url()
print(spAuthURL)
# Prints the URL that Spotify API will redirect to
authResp = input("Enter URL")
# Console user is expected to visit the URL and submit the new redirected URL on console
respCode = spotifyAuth.parse_response_code(authResp)
spotiToken = spotifyAuth.get_access_token(respCode)
elif spotifyAuth.is_token_expired(spotifyAuth.get_cached_token()):
spotiToken = spotifyAuth.refresh_access_token(spotiToken["refresh_token"])
spoti = spotipy.Spotify(auth=spotiToken["access_token"])
return [spotiToken, spoti]
PS: I've been developing Python only for couple of weeks, even after doing some research, I wasn't able to find a solution to this problem in a way that I need. I'm not sure if it's even possible to achieve it that way. So, if that's impossible, please excuse me for my lack of knowledge.

I've found the solution myself.
It seems that requests is a good match for this example.
Following snippet works perfectly for now.
def tryFetchSpotiToken():
global spotiToken, spoti
try:
spotiToken = spotifyAuth.get_cached_token()
except:
if not spotiToken:
spAuthURL = spotifyAuth.get_authorize_url()
htReq = requests.get(spAuthURL)
htRed = htReq.url
respCode = spotifyAuth.parse_response_code(htRed)
spotiToken = spotifyAuth.get_access_token(respCode)
elif spotifyAuth.is_token_expired(spotifyAuth.get_cached_token()):
spotiToken = spotifyAuth.refresh_access_token(spotiToken["refresh_token"])
spoti = spotipy.Spotify(auth=spotiToken["access_token"])

Twilio python flask try to get status call back event but it returns "none"

I am new to twilio python and flask. I tried to follow the twilio example for tracking sms status, but as I mentionned I get none as the return for this statement status=request.values.get('callstatus', None).
I want to track the call progress status and see its different status. I follow all the documentation but I am blocked. Thanks for your help.
from flask import Flask,request
from twilio.rest import Client
from twilio.twiml.voice_response import Dial, VoiceResponse
from pprint import pprint
import logging
logging.basicConfig(level=logging.INFO)
app = Flask(__name__)
twilioClient = Client('*******************', '*****************************')
#app.route("/voice")
def voice():
call=twilioClient.calls.create(
method='GET',
status_callback='http://*****.ngrok.io/response',
status_callback_event='initiated ringing answered completed',
status_callback_method='POST',
from_='+**********',
to='+*********',
url='http://demo.twilio.com/docs/voice.xml''
)
return call.sid
#app.route('/response', methods=['POST'])
def outbound():
status=request.values.get('callstatus', None)
logging.info('Status: {}'.format(status))
return ('', 204)
if __name__ == "__main__":
app.run(debug=true)

I understand that this is a relatively old post and that you might have figured out the solution as of now, but still I am posting my answer so that it will benefit other users who may face the same issue in future.
There are 2 issues that I can see in the code posted above:
Inside the function voice()
When you are making a request to the twilio server by using the API twilioClient.calls.create(), you are feeding the status_callback_event parameter with a single string.
Instead of passing a single string, you have to pass a list of strings like:
['initiated', 'answered', 'ringing', 'completed']
Inside the function outbound()
The parameter sent by twilio to the call back url is CallStatus and not callstatus. They are case sensitive.
status=request.values.get('CallStatus', None)
You can refer this link to check out all the call resources

Tornado routes no picked up by Heroku

i follow the tutorial based on a service i wan't to use ( Drone Deploy) :
https://dronedeploy.gitbooks.io/dronedeploy-apps/content/server_example.html
i made a topic on their forum for this issue 2 days ago.( and repo on Github)
But maybe someone can help me here.
I got an issue with an Heroku server app i do.
i use Python and the Tornado module.
The fact is i got a 404 error
My terminal ( when test it localy ) ask me :
"WARNING:tornado.access:404 GET / (::1) 0.75ms"
so the "GET" function is not working , maybe because of HTTP access control (CORS) i try many fix , but none worked.
Maybe i made something wrong ,or forgot something.
update :
Recently someone of the Drone deploy said my tornado routes aren't being picked up by heroku.
After following many tutorial and read the Tornado documentation i have no idea how to routes this.
the url of the server : https://stardrone-server.herokuapp.com/
this the code :
import os
import requests
import tornado.ioloop
import tornado.web
GEOCODE_FMT = 'https://maps.googleapis.com/maps/api/geocode/json?address={address}&key={key}'
class GeocodeHandler(tornado.web.RequestHandler):
"""Proxy a call to the Google Maps geocode API"""
def set_default_headers(self):
# allow cross-origin requests to be made from your app on DroneDeploy to your web server
self.set_header("Access-Control-Allow-Origin", "https://www.dronedeploy.com")
self.set_header("Access-Control-Allow-Headers", "x-requested-with")
# add more allowed methods when adding more handlers (POST, PUT, etc.)
self.set_header("Access-Control-Allow-Methods", "GET, OPTIONS")
def get(self):
api_key = os.environ.get("MyApiKey")
address = self.get_query_argument("address")
url = GEOCODE_FMT.format(address=address, key=api_key)
# fetch results of the geocode from Google
response = requests.get(url)
# send the results back to the client
self.write(response.content)
def options(self):
# no body
self.set_status(204)
self.finish()
def main():
application = tornado.web.Application([
(r"/geocode/", GeocodeHandler)
])
port = int(os.environ.get("PORT", 5000))
application.listen(port)
tornado.ioloop.IOLoop.current().start()
if __name__ == "__main__":
main()
Maybe i missed something.
Any help, advices are appreciated
Thanks
John

How do I receive Github Webhooks in Python

Github offers to send Post-receive hooks to an URL of your choice when there's activity on your repo.
I want to write a small Python command-line/background (i.e. no GUI or webapp) application running on my computer (later on a NAS), which continually listens for those incoming POST requests, and once a POST is received from Github, it processes the JSON information contained within. Processing the json as soon as I have it is no problem.
The POST can come from a small number of IPs given by github; I plan/hope to specify a port on my computer where it should get sent.
The problem is, I don't know enough about web technologies to deal with the vast number of options you find when searching.. do I use Django, Requests, sockets,Flask, microframeworks...? I don't know what most of the terms involved mean, and most sound like they offer too much/are too big to solve my problem - I'm simply overwhelmed and don't know where to start.
Most tutorials about POST/GET I could find seem to be concerned with either sending or directly requesting data from a website, but not with continually listening for it.
I feel the problem is not really a difficult one, and will boil down to a couple of lines, once I know where to go/how to do it. Can anybody offer pointers/tutorials/examples/sample code?

First thing is, web is request-response based. So something will request your link, and you will respond accordingly. Your server application will be continuously listening on a port; that you don't have to worry about.
Here is the similar version in Flask (my micro framework of choice):
from flask import Flask, request
import json
app = Flask(__name__)
#app.route('/',methods=['POST'])
def foo():
data = json.loads(request.data)
print "New commit by: {}".format(data['commits'][0]['author']['name'])
return "OK"
if __name__ == '__main__':
app.run()
Here is a sample run, using the example from github:
Running the server (the above code is saved in sample.py):
burhan#lenux:~$ python sample.py
* Running on http://127.0.0.1:5000/
Here is a request to the server, basically what github will do:
burhan#lenux:~$ http POST http://127.0.0.1:5000 < sample.json
HTTP/1.0 200 OK
Content-Length: 2
Content-Type: text/html; charset=utf-8
Date: Sun, 27 Jan 2013 19:07:56 GMT
Server: Werkzeug/0.8.3 Python/2.7.3
OK # <-- this is the response the client gets
Here is the output at the server:
New commit by: Chris Wanstrath
127.0.0.1 - - [27/Jan/2013 22:07:56] "POST / HTTP/1.1" 200 -

Here's a basic web.py example for receiving data via POST and doing something with it (in this case, just printing it to stdout):
import web
urls = ('/.*', 'hooks')
app = web.application(urls, globals())
class hooks:
def POST(self):
data = web.data()
print
print 'DATA RECEIVED:'
print data
print
return 'OK'
if __name__ == '__main__':
app.run()
I POSTed some data to it using hurl.it (after forwarding 8080 on my router), and saw the following output:
$ python hooks.py
http://0.0.0.0:8080/
DATA RECEIVED:
test=thisisatest&test2=25
50.19.170.198:33407 - - [27/Jan/2013 10:18:37] "HTTP/1.1 POST /hooks" - 200 OK
You should be able to swap out the print statements for your JSON processing.
To specify the port number, call the script with an extra argument:
$ python hooks.py 1234

I would use:
https://github.com/carlos-jenkins/python-github-webhooks
You can configure a web server to use it, or if you just need a process running there without a web server you can launch the integrated server:
python webhooks.py
This will allow you to do everything you said you need. It, nevertheless, requires a bit of setup in your repository and in your hooks.
Late to the party and shameless autopromotion, sorry.

If you are using Flask, here's a very minimal code to listen for webhooks:
from flask import Flask, request, Response
app = Flask(__name__)
#app.route('/webhook', methods=['POST'])
def respond():
print(request.json) # Handle webhook request here
return Response(status=200)
And the same example using Django:
from django.http import HttpResponse
from django.views.decorators.http import require_POST
#require_POST
def example(request):
print(request.json) # Handle webhook request here
return HttpResponse('Hello, world. This is the webhook response.')
If you need more information, here's a great tutorial on how to listen for webhooks with Python.

If you're looking to watch for changes in any repo...
1. If you own the repo that you want to watch
In your repo page, Go to settings
click webhooks, new webhook (top right)
give it your ip/endpoint and setup everything to your liking
use any server to get notified
2. Not your Repo
take the url you want i.e https://github.com/fire17/gd-xo/
add /commits/master.atom to the end such as:
https://github.com/fire17/gd-xo/commits/master.atom
Use any library you want to get that page's content, like:
filter out the keys you want, for example the element
response = requests.get("https://github.com/fire17/gd-xo/commits/master.atom").text
response.split("<updated>")[1].split("</updated>")[0]
'2021-08-06T19:01:53Z'
make a loop that checks this every so often and if this string has changed, then you can initiate a clone/pull request or do whatever you like

how do i make an api call in python (GAE)?

I have made a simple web form in Google app engine where I have added a recaptcha component.
The component is showing up on my web page. But i have no idea how to make the api call.
my code is;
def post(self):
challenge = self.request.get('recaptcha_challenge_field')
response = self.request.get('recaptcha_response_field')
remoteip = os.environ['REMOTE_ADDR']
private_key = 'xxx'
cResponse = self.request.submit(http://www.google.com/recaptcha/api/verify?privatekey="private_key"&remoteip="remoteip"&challenge="challenge"&response="response")
if cResponse.is_valid:
# response was valid
# other stuff goes here
pass
else:
error = cResponse.error_code
its pretty clear that my api call is completely wrong but i have no idea how to make it.
The examples i have seen use the plugin.

Use the URL Fetch API documented here, the first example in the linked page should be suitable for your needs.
Notice that url fetches have a quota and are billable.