I am trying to connect to the twitter streaming API on Python anywhere, but always get a connection refused error.
I use Tweepy in my application, and to test the connection I am using the streaming example that can be found in the repo.
HEre is a sum-up of the code :
from tweepy.streaming import StreamListener
from tweepy import OAuthHandler
from tweepy import Stream
# Go to http://dev.twitter.com and create an app.
# The consumer key and secret will be generated for you after
consumer_key=""
consumer_secret=""
# After the step above, you will be redirected to your app's page.
# Create an access token under the the "Your access token" section
access_token=""
access_token_secret=""
class StdOutListener(StreamListener):
""" A listener handles tweets are the received from the stream.
This is a basic listener that just prints received tweets to stdout.
"""
def on_data(self, data):
print data
return True
def on_error(self, status):
print status
if __name__ == '__main__':
l = StdOutListener()
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
stream = Stream(auth, l)
stream.filter(track=['basketball'])
When I run this line in a bash console in python anywhere (after having filled the tokens of course)
12:02 ~/tweepy/examples (master)$ python streaming.py
I get the following error :
Traceback (most recent call last):
File "streaming.py", line 33, in <module>
stream.filter(track=['basketball'])
File "/usr/local/lib/python2.7/site-packages/tweepy/streaming.py", line 228, in filter
self._start(async)
File "/usr/local/lib/python2.7/site-packages/tweepy/streaming.py", line 172, in _start
self._run()
File "/usr/local/lib/python2.7/site-packages/tweepy/streaming.py", line 106, in _run
conn.connect()
File "/usr/local/lib/python2.7/httplib.py", line 1157, in connect
self.timeout, self.source_address)
File "/usr/local/lib/python2.7/socket.py", line 571, in create_connection
raise err
socket.error: [Errno 111] Connection refused
The domain .twitter.com is in the pythonanywhere whithelist though, so i don't understand why the connection would be refused :s.
The very same code works like a charm on my Ubuntu.
Any idea would be more than welcome, thanks !!
If you're using a Free account, tweepy won't work. It does not use the proxy settings from the environment.
There is a fork of tweepy that you might be able to use (http://github.com/ducu/tweepy) until the main line uses the proxy settings correctly.
As Glenn said, there currently isn't proxy support in tweepy.
For a reason I cannot explain (and isn't documented), a pull request was closed without any merge about a month ago.
https://github.com/tweepy/tweepy/pull/152
There apparently is a fork available on github (see Glenn's answer), but I didn't test it.
Knowing that I would need to use my own domain name in the end, I finally got a paid account on pythonanywhere and got rid of the proxy stuff all together.
Related
I'm currently trying to make a system that can transcribe a phone call in real time and then display the conversation in my command line. To do this, im using a Twilio phone number which sends out a http request when called. Then using Flask, Ngrok and Websockets to compile my server code, make my local port public and to transfer the data, the TwiML verb "Stream" is used to stream the audio data to the Google Cloud Speech-Text API. I have so far used Twilio's python demo on GitHub (https://github.com/twilio/media-streams/tree/master/python/realtime-transcriptions).
My server code:
from flask import Flask, render_template
from flask_sockets import Sockets
from SpeechClientBridge import SpeechClientBridge
from google.cloud.speech_v1 import enums
from google.cloud.speech_v1 import types
import json
import base64
import os
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "./<KEY>.json"
HTTP_SERVER_PORT = 8080
config = types.RecognitionConfig(
encoding=enums.RecognitionConfig.AudioEncoding.MULAW,
sample_rate_hertz=8000,
language_code='en-US')
streaming_config = types.StreamingRecognitionConfig(
config=config,
interim_results=True)
app = Flask(__name__)
sockets = Sockets(app)
#app.route('/home')
def home():
return render_template("index.html")
#app.route('/twiml', methods=['POST'])
def return_twiml():
print("POST TwiML")
return render_template('streams.xml')
def on_transcription_response(response):
if not response.results:
return
result = response.results[0]
if not result.alternatives:
return
transcription = result.alternatives[0].transcript
print("Transcription: " + transcription)
#sockets.route('/')
def transcript(ws):
print("WS connection opened")
bridge = SpeechClientBridge(
streaming_config,
on_transcription_response
)
while not ws.closed:
message = ws.receive()
if message is None:
bridge.terminate()
break
data = json.loads(message)
if data["event"] in ("connected", "start"):
print(f"Media WS: Received event '{data['event']}': {message}")
continue
if data["event"] == "media":
media = data["media"]
chunk = base64.b64decode(media["payload"])
bridge.add_request(chunk)
if data["event"] == "stop":
print(f"Media WS: Received event 'stop': {message}")
print("Stopping...")
break
bridge.terminate()
print("WS connection closed")
if __name__ == '__main__':
from gevent import pywsgi
from geventwebsocket.handler import WebSocketHandler
server = pywsgi.WSGIServer(('', HTTP_SERVER_PORT), app, handler_class=WebSocketHandler)
print("Server listening on: http://localhost:" + str(HTTP_SERVER_PORT))
server.serve_forever()
streams.xml:
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<Say> Thanks for calling!</Say>
<Start>
<Stream url="wss://<ngrok-URL/.ngrok.io/"/>
</Start>
<Pause length="40"/>
</Response>
Twilio WebHook:
http://<ngrok-URL>.ngrok.io/twiml
Im am getting the following error when I run the server code and then call the Twilio number:
C:\Users\Max\Python\Twilio>python server.py
Server listening on: http://localhost:8080
POST TwiML
WS connection opened
Media WS: Received event 'connected': {"event":"connected","protocol":"Call","version":"0.2.0"}
Media WS: Received event 'start': {"event":"start","sequenceNumber":"1","start":{"accountSid":"AC8abc5aa74496a227d3eb489","streamSid":"MZe6245f23e2385aa2ea7b397","callSid":"CA5864313b4992607d3fe46","tracks":["inbound"],"mediaFormat":{"encoding":"audio/x-mulaw","sampleRate":8000,"channels":1}},"streamSid":"MZe6245f2397c1285aa2ea7b397"}
Exception in thread Thread-4:
Traceback (most recent call last):
File "C:\Users\Max\AppData\Local\Programs\Python\Python37\lib\site-packages\google\api_core\grpc_helpers.py", line 96, in next
return six.next(self._wrapped)
File "C:\Users\Max\AppData\Local\Programs\Python\Python37\lib\site-packages\grpc\_channel.py", line 416, in __next__
return self._next()
File "C:\Users\Max\AppData\Local\Programs\Python\Python37\lib\site-packages\grpc\_channel.py", line 689, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.OUT_OF_RANGE
details = "Audio Timeout Error: Long duration elapsed without audio. Audio should be sent close to real time."
debug_error_string = "{"created":"#1591738676.565000000","description":"Error received from peer ipv6:[2a00:1450:4009:807::200a]:443","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Audio Timeout Error: Long duration elapsed without audio. Audio should be sent close to real time.","grpc_status":11}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\Max\AppData\Local\Programs\Python\Python37\lib\threading.py", line 917, in _bootstrap_inner
self.run()
File "C:\Users\Max\AppData\Local\Programs\Python\Python37\lib\threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\Max\Python\Twilio\SpeechClientBridge.py", line 37, in process_responses_loop
for response in responses:
File "C:\Users\Max\AppData\Local\Programs\Python\Python37\lib\site-packages\google\api_core\grpc_helpers.py", line 99, in next
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "<string>", line 3, in raise_from
google.api_core.exceptions.OutOfRange: 400 Audio Timeout Error: Long duration elapsed without audio. Audio should be sent close to real time.
Media WS: Received event 'stop': {"event":"stop","sequenceNumber":"752","streamSid":"MZe6245f2397c125aa2ea7b397","stop":{"accountSid":"AC8abc5aa74496a60227d3eb489","callSid":"CA5842bc6431314d502607d3fe46"}}
Stopping...
WS connection closed
I cant work out why im getting the audio timeout error? Is it a firewall issue with Twilio and Google? An encoding issue?
Any help would be greatly appreciated.
System:
Windows 10
Python 3.7.1
ngrok 2.3.35
Flask 1.1.2
As your streams.xml returned socket url "wss://<ngrok-URL/.ngrok.io/", please make sure it matches with your routing (e.g. #sockets.route('/'))
If your socket starting with '/', then your should rewrite the streams.xml, see below as an example.
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<Say> Thanks for calling!</Say>
<Start>
<Stream url="wss://YOUR_NGROK_ID.ngrok.io/"/>
</Start>
<Pause length="40"/>
</Response>
I ran some tests on this to try to establish what was happening. I put a timer over the
bridge = SpeechClientBridge(
streaming_config,
on_transcription_response)
section of code and found that it was taking ~10.9s to initialize. I believe the google API has a timeout of 10s. I tried running this on my google cloud instance which has more oomph than my laptop and it works perfectly well. Either this, or there are some different versions of libraries/code etc installed on the GCP instance, which I need to check.
This is related to gevent (used by flask_sockets) and grpc (used by google cloud speech) conflict described in this issue https://github.com/grpc/grpc/issues/4629
the solution is to add the following code
import grpc.experimental.gevent as grpc_gevent
grpc_gevent.init_gevent()
I am trying to create a P2P node using python (pyp2p) but I am getting this error:
Eamons-MacBook-Pro:blockchain eamonwhite$ python3 serveralice.py
HTTP Error 404: Not Found
HTTP Error 404: Not Found
HTTP Error 404: Not Found
HTTP Error 404: Not Found
Traceback (most recent call last):
File "/Users/eamonwhite/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pyp2p/net.py", line 732, in start
rendezvous_con = self.rendezvous.server_connect()
File "/Users/eamonwhite/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pyp2p/rendezvous_client.py", line 92, in server_connect
con.connect(server["addr"], server["port"])
File "/Users/eamonwhite/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pyp2p/sock.py", line 189, in connect
self.s.bind((src_ip, 0))
TypeError: str, bytes or bytearray expected, not NoneType
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "serveralice.py", line 10, in <module>
alice.start()
File "/Users/eamonwhite/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pyp2p/net.py", line 735, in start
raise Exception("Unable to connect to rendezvous server.")
Exception: Unable to connect to rendezvous server.
My relevant code looks like this:
from uuid import uuid4
from blockchain import Blockchain
from flask import Flask, jsonify, request
from pyp2p.net import *
import time
#Setup Alice's p2p node.
alice = Net(passive_bind="192.168.1.131", passive_port=44444, interface="en0", node_type="passive", debug=1)
alice.start()
alice.bootstrap()
alice.advertise()
while 1:
for con in alice:
for reply in con:
print(reply)
time.sleep(1)
...
It is getting stuck on the Net function right at the beginning - something to do with the rendezvous package. The IP is my IP on the my network, and I port forwarded 44444 although I'm not sure if I need to do that or not. Thanks.
I am new to this, apparently with the way the server code was configured, it needed a rendezvous server to work (a node that handles all the other nodes). It is in net.py of the pyp2p package:
# Bootstrapping + TCP hole punching server.
rendezvous_servers = [
{
"addr": "162.243.213.95",
"port": 8000
}
]
The address was the problem, obviously it is just a placeholder IP. So then I realized I needed my own rendezvous server, and I used this code - https://raw.githubusercontent.com/StorjOld/pyp2p/master/pyp2p/rendezvous_server.py.
However I had to debug this file a little, it ended up needing to have import sys, import time and import re statements at the top before it would work. Now I am going to host it on my raspberry pi so that it is always up to handle nodes :)
I have my RabbitMQ Server running on AWS EC2
I have run the producer and consumer code locally. It is working.
I am able to access the rabbitMQ management web app as well.
When I am trying to push data from my laptop to EC2
I am getting this error on this line:
connection = pika.BlockingConnection(pika.ConnectionParameters('xx.xx.xx.xx',5672,'/',credentials))
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 339, in init
self._process_io_for_connection_setup()
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 374, in _process_io_for_connection_setup
self._open_error_result.is_ready)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 395, in _flush_output
raise exceptions.ConnectionClosed()
pika.exceptions.ConnectionClosed
where xx.xx.xx.xx: public IP address of my instance
Pls tell me, if I am using the correct parameters. What should be the IP address , virtual hostname.
I have checked the credentials , the user that I am using exists and it has the rights to access '/' virtual host
I have made the needed changes in the Security Groups.
This is a screenshot of it:
When I am running the same producer code from within the instance it's working properly. No Exceptions and the consumer is able to receive it as well.
This is my complete code for reference:
import pika
print("Start")
credentials=pika.PlainCredentials('manish','manish')#RabbitMQ user created on EC2
connection=pika.BlockingConnection(pika.ConnectionParameters('xx.xx.xx.xx',5672,'/',credentials))
channel = connection.channel()
channel.queue_declare(queue='hello')
channel.basic_publish(exchange='',
routing_key='hello',
body='Hello World!')
print(" [x] Sent 'Hello World!'")
connection.close()
I tried socket_timeout and it worked for me, you could try something like :
credentials = pika.PlainCredentials('username,'password')
connection = pika.BlockingConnection(pika.ConnectionParameters('hostname',port,'virtual host',credentials,**socket_timeout=10000**))
I have been trying to configure Redis Cluster with my python wepapp (which is using single redis instance currently)
Since redis clustering is supported from 3.x, upgraded the redis server version to 3.0.7
Using redis-py-clusterV1.2.0 module in my python app.
PubSub command to publish and subscribe to a particular channel does not seem to be working with the KeyError as mentioned below in the stack trace. It would be great if somebody can help on this.
from rediscluster import StrictRedisCluster
startup_nodes = [{"host": "127.0.0.1", "port": "6379"}]
redis_conn = StrictRedisCluster(startup_nodes=startup_nodes,
decode_responses=True)
redis_conn.pubsub()
pubsub.subscribe(['a_b:c:d']) //tried with diff keys, getting the same error
Stack Trace of the Error:
File "/home/<username>/.pex/install/redis-2.10.2-py2-none-
any.whl.621ec5075459e032e02a8654e15c5ca320969b0b/redis-2.10.2-py2-none-
any.whl/redis/client.py",line 2196, in subscribe
ret_val = self.execute_command('SUBSCRIBE', *iterkeys(new_channels))
File "/home/<username>/.pex/install/redis_py_cluster-1.2.0-py2-none-
any.whl.5a485619e4e267bf90b972cf0e736baba918e3cc/redis_py_cluster-1.2.0-py2-
none-any.whl/rediscluster/pubsub.py",line 29, in execute_command
channel=args[1],
File "/home/<username>/.pex/install/redis_py_cluster-1.2.0-py2-none-
any.whl.5a485619e4e267bf90b972cf0e736baba918e3cc/redis_py_cluster-1.2.0-py2-
none-any.whl/rediscluster/connection.py",line 141, in get_connection
node = self.get_master_node_by_slot(slot)
File "/home/<username>/.pex/install/redis_py_cluster-1.2.0-py2-none-
any.whl.5a485619e4e267bf90b972cf0e736baba918e3cc/redis_py_cluster-1.2.0-py2-
none-any.whl/rediscluster/connection.py",line 267, inget_master_node_by_slot
return self.nodes.slots[slot][0]
KeyError: 7226
When I debug , got to know self.nodes.slots[] seems to be empty. No clues why I get this error.
Thanks in Advance
Priya
Trying to use Google App Engine's remote_api so that we can do line-by-line debugging through the IDE.
The remote api works great at first. The application is able to successfully retrieve information from the database. The error occurs when wepapp responds to the client browser.
The Code:
It is very similar to the example given in app engine's documentation:
from model import My_Entity
from google.appengine.ext.remote_api import remote_api_stub
# Test database calls
def get(w_self):
remote_api_stub.ConfigureRemoteApi(None, '/_ah/remote_api', auth_func, 'myapp.appspot.com')
t_entity = My_Entity.get_by_key_name('the_key')
w_self.response.set_status(200)
# The error occurs AFTER this code executes, when webapp actually responds to the browser
Error Traceback:
The error seems to be related to the blobstore.
Is the remote api initialized too late into the code?
...After webapp has done something with the blobstore through the localhost server? Then the remote api might be re-directing requests to the blobstore in the server instead of the localhost debug server where webapp is expecting it to be?
Traceback (most recent call last):
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver.py", line 2795, in _HandleRequest
login_url)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver.py", line 3622, in CreateImplicitMatcher
get_blob_storage)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver_blobstore.py", line 420, in CreateUploadDispatcher
return UploadDispatcher()
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver_blobstore.py", line 307, in __init__
get_blob_storage())
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver_blobstore.py", line 79, in GetBlobStorage
return apiproxy_stub_map.apiproxy.GetStub('blobstore').storage
AttributeError: 'RemoteStub' object has no attribute 'storage'
Should the remote api be initialized somewhere else in the code?
Or does this problem have to do with something else?
Thanks so much!
To get this working you can use the help of the testbed to start the stubs that are missing:
ADDRESS=....
remote_api_stub.ConfigureRemoteApi(None, '/_ah/remote_api', auth_func, ADDRESS)
# First, create an instance of the Testbed class.
myTestBed = testbed.Testbed()
# Then activate the testbed, which prepares the service stubs for use.
myTestBed.activate()
# Next, declare which service stubs you want to use.
myTestBed.init_blobstore_stub()
myTestBed.init_logservice_stub()