I was trying to use Linkedin API to scrape information about companies but i had a socket error:
Traceback (most recent call last):
File "/Users/anhvangiang/Desktop/PY/test.py", line 10, in <module>
application = server.quick_api(key, secret)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/linkedin/server.py", line 25, in quick_api
_wait_for_user_to_enter_browser(app)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/linkedin/server.py", line 39, in _wait_for_user_to_enter_browser
httpd = BaseHTTPServer.HTTPServer(server_address, MyHandler)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 417, in __init__
self.server_bind()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/BaseHTTPServer.py", line 108, in server_bind
SocketServer.TCPServer.server_bind(self)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 431, in server_bind
self.socket.bind(self.server_address)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 228, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 48] Address already in use
My code is:
from linkedin import server
from socket import *
sock = socket()
sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
key = "key"
secret = "secret"
application = server.quick_api(key, secret)
Any suggestions how can i fix the problem ?
You could find and kill the process using the:
ps aux | grep python
Get the process id and kill using:
kill -9 PID
It will free the address.
OR you can try :
application = server.quick_api(key, secret, 5557)
Related
I tried to get all the active/scheduled/reserved tasks in redis:
from celery.task.control import inspect
inspect_obj = inspect()
inspect_obj.active()
inspect_obj.scheduled()
inspect_obj.reserved()
But was greeted with a list of errors as follows:
My virtual environment ==> HubblerAPI.
Iam using this from the ec2 console
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/celery/app/control.py", line 81, in active
return self._request('dump_active', safe=safe)
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/celery/app/control.py", line 71, in _request
timeout=self.timeout, reply=True,
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/celery/app/control.py", line 316, in broadcast
limit, callback, channel=channel,
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/kombu/pidbox.py", line 283, in _broadcast
chan = channel or self.connection.default_channel
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/kombu/connection.py", line 771, in default_channel
self.connection
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/kombu/connection.py", line 756, in connection
self._connection = self._establish_connection()
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/kombu/connection.py", line 711, in _establish_connection
conn = self.transport.establish_connection()
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/kombu/transport/pyamqp.py", line 116, in establish_connection
conn = self.Connection(**opts)
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/amqp/connection.py", line 165, in __init__
self.transport = self.Transport(host, connect_timeout, ssl)
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/amqp/connection.py", line 186, in Transport
return create_transport(host, connect_timeout, ssl)
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/amqp/transport.py", line 299, in create_transport
return TCPTransport(host, connect_timeout)
File "/home/ec2-user/HubblerAPI/local/lib/python3.4/site-
packages/amqp/transport.py", line 95, in __init__
raise socket.error(last_err)
**OSError: [Errno 111] Connection refused**
My celery config file is as follows:
BROKER_TRANSPORT = 'redis'
BROKER_TRANSPORT_OPTIONS = {
'queue_name_prefix': 'dev-',
'wait_time_seconds': 10,
'polling_interval': 30,
# The polling interval decides the number of seconds to sleep
between unsuccessful polls
'visibility_timeout': 3600 * 5,
# If a task is not acknowledged within the visibility_timeout, the
task will be redelivered to another worker and executed.
}
CELERY_MESSAGES_DB = 6
BROKER_URL = "redis://%s:%s/%s" % (AWS_REDIS_ENDPOINT, AWS_REDIS_PORT,
CELERY_MESSAGES_DB)
What am i doing wrong here as the error log suggests that its not using the redis broker.
Looks like your python code doesn't recognize your configs since it is attempting to use RabbitMQ's ampq protocol instead of the configured broker.
I suggest the following
https://docs.celeryq.dev/en/stable/getting-started/backends-and-brokers/redis.html
Your configs look similar to Django configs for Celery yet it doesn't seem you are using Celery with Django.
https://docs.celeryq.dev/en/latest/django/first-steps-with-django.html
The issue is using "BROKER_URL" instead of "CELERY_BROKER_URL" in settings.py. Celery wasn't finding the URL and was defaulting to the rabbitmq port instead of the redis port.
I am new to neo4j world. I have successfully used it on my macbook. Now I am deploying it on a remote Linux machine with the same setup. But I keep getting this Protocol error. What caused this issue? How to fix this? I have been banging my head on this error for days.
Traceback (most recent call last):
File "/root/dev/knowledgeGraphH/knowledge/media_entity_mapper.py", line 31, in <module>
main()
File "/root/dev/knowledgeGraphH/knowledge/media_entity_mapper.py", line 28, in main
map_media_to_entities()
File "/root/dev/knowledgeGraphH/knowledge/media_entity_mapper.py", line 7, in map_media_to_entities
data_manager = DataManager()
File "/root/dev/knowledgeGraphH/knowledge/data_manager/data_manager.py", line 13, in __init__
self.graphDB = Neo4jManager()
File "/root/dev/knowledgeGraphH/knowledge/neo4j_manager.py", line 10, in __init__
self.session = self.driver.session()
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/session.py", line 148, in session
session = Session(self)
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/session.py", line 461, in __init__
self.connection = connect(driver.host, driver.port, driver.ssl_context, **driver.config)
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/connection.py", line 465, in connect
return Connection(s, der_encoded_server_certificate=der_encoded_server_certificate, **config)
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/connection.py", line 237, in __init__
self.fetch()
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/connection.py", line 326, in fetch
self.acknowledge_failure()
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/connection.py", line 273, in acknowledge_failure
fetch()
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/connection.py", line 311, in fetch
raw.writelines(self.channel.chunk_reader())
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/connection.py", line 169, in chunk_reader
chunk_header = self._recv(2)
File "/root/dev/knowledgeGraphH/env/lib/python2.7/site-packages/neo4j/v1/connection.py", line 152, in _recv
raise ProtocolError("Server closed connection")
neo4j.v1.exceptions.ProtocolError: Server closed connection
Seems to be port issue. Is the bolt port open or not? You have access to the port or not?
Check the output of the following command:
lsof -i tcp:7687
change the port number if you have changed bolt port address.
It turns out it was because I used the wrong credentials for this connection.
I have a problem with my most recent openshift gear , i get the python error errno 13 , Permission denied. I have checked around on here and found that most people who have gotten the same issue have just forgot the if __name__ == "__main__": statement before the app.run(), but i still have the same issue. According to the log on openshift, the error seems to be rooting from the socket.py built in to python.
This is my second gear on openshift and both this and my first one is built using flask. The first one works just fine but my second one wont work ...
This is the error message I get in the python log from openshift:
* Running on http://127.0.0.1:5000/
Traceback (most recent call last):
File "app.py", line 24, in <module>
app.run()
File "/var/lib/openshift/54d242655973ca23980001d1/python/virtenv /lib/python2.7/site-packages/flask/app.py", line 772, in run
run_simple(host, port, self, **options)
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/werkzeug/serving.py", line 617, in run_simple
inner()
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/werkzeug/serving.py", line 599, in inner
passthrough_errors, ssl_context).serve_forever()
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/werkzeug/serving.py", line 408, in make_server
passthrough_errors, ssl_context)
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/werkzeug/serving.py", line 334, in __init__
HTTPServer.__init__(self, (host, int(port)), handler)
File "/opt/rh/python27/root/usr/lib64/python2.7/SocketServer.py", line 419, in __init__
self.server_bind()
File "/opt/rh/python27/root/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind
SocketServer.TCPServer.server_bind(self)
File "/opt/rh/python27/root/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind
self.socket.bind(self.server_address)
File "/opt/rh/python27/root/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 13] Permission denied
EDIT 2:
So i changed the ip to OPENSHIFT_PYTHON_IP but the errorlog looked exactly the same. This is my wsgi.py code
#!/usr/bin/python
import os
virtenv = os.environ['OPENSHIFT_PYTHON_DIR'] + '/virtenv/'
virtualenv = os.path.join(virtenv, 'bin/activate_this.py')
try:
execfile(virtualenv, dict(__file__=virtualenv))
except IOError:
pass
def run_gevent_server(app, ip, port=8080):
from gevent.pywsgi import WSGIServer
WSGIServer((ip, port), app).serve_forever()
def run_simple_httpd_server(app, ip, port=8080):
from wsgiref.simple_server import make_server
make_server(ip, port, app).serve_forever()
from server import app as application
#
# Below for testing only
#
if __name__ == '__main__':
ip = os.environ['OPENSHIFT_PYTHON_IP']
port = 8080
zapp = impo.load_source('application', 'wsgi/application')
try:
run_gevent_server(application, ip, port)
except:
run_simple_httpd_server(zapp.application, ip, port)
What port/ip address are you using? You must bind to port 8080 of your openshift ip address, you can't bind to 127.0.0.1, it is not allowed.
I am trying to right a Google Cloud Platform app in python with Flask that makes an MQTT connection. I have included the paho python library by doing pip install paho-mqtt -t libs/. However, when I try to run the app, even if I don't try to connect to MQTT. I get a weird error about IP address checking:
RuntimeError: error('illegal IP address string passed to inet_pton',)
It seems something in the remote_socket lib is causing a problem. Is this a security issue? Is there someway to disable it?
Relevant code:
from flask import Flask
import paho.mqtt.client as mqtt
import logging as logger
app = Flask(__name__)
# Note: We don't need to call run() since our application is embedded within
# the App Engine WSGI application server.
#callback to print out connection status
def on_connect(mosq, obj, rc):
logger.info('on_connect')
if rc == 0:
logger.info("Connected")
mqttc.subscribe('test', 0)
else:
logger.info(rc)
def on_message(mqttc, obj, msg):
logger.info(msg.topic+" "+str(msg.qos)+" "+str(msg.payload))
mqttc = mqtt.Client("mqttpy")
mqttc.on_message = on_message
mqttc.on_connect = on_connect
As well as full stack trace:
ERROR 2014-06-03 15:14:57,285 wsgi.py:262]
Traceback (most recent call last):
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 239, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 298, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 84, in LoadObject
obj = __import__(path[0])
File "/Users/cbarnes/code/ignite/tank-demo/appengine-flask-demo/main.py", line 24, in <module>
mqttc = mqtt.Client("mqtthtpp")
File "/Users/cbarnes/code/ignite/tank-demo/appengine-flask-demo/lib/paho/mqtt/client.py", line 403, in __init__
self._sockpairR, self._sockpairW = _socketpair_compat()
File "/Users/cbarnes/code/ignite/tank-demo/appengine-flask-demo/lib/paho/mqtt/client.py", line 255, in _socketpair_compat
listensock.bind(("localhost", 0))
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/dist27/socket.py", line 222, in meth
return getattr(self._sock,name)(*args)
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/remote_socket/_remote_socket.py", line 668, in bind
self._SetProtoFromAddr(request.mutable_proxy_external_ip(), address)
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/remote_socket/_remote_socket.py", line 632, in _SetProtoFromAddr
proto.set_packed_address(self._GetPackedAddr(address))
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/remote_socket/_remote_socket.py", line 627, in _GetPackedAddr
AI_NUMERICSERV|AI_PASSIVE):
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/remote_socket/_remote_socket.py", line 338, in getaddrinfo
canonical=(flags & AI_CANONNAME))
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/remote_socket/_remote_socket.py", line 211, in _Resolve
canon, aliases, addresses = _ResolveName(name, families)
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/remote_socket/_remote_socket.py", line 229, in _ResolveName
apiproxy_stub_map.MakeSyncCall('remote_socket', 'Resolve', request, reply)
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 94, in MakeSyncCall
return stubmap.MakeSyncCall(service, call, request, response)
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 328, in MakeSyncCall
rpc.CheckSuccess()
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/api/apiproxy_rpc.py", line 156, in _WaitImpl
self.request, self.response)
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 200, in MakeSyncCall
self._MakeRealSyncCall(service, call, request, response)
File "/Users/cbarnes/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 234, in _MakeRealSyncCall
raise pickle.loads(response_pb.exception())
RuntimeError: error('illegal IP address string passed to inet_pton',)
INFO 2014-06-03 15:14:57,291 module.py:639] default: "GET / HTTP/1.1" 500 -
Thanks!
You're trying to connect to host test and port 0 where typically you should use localhost and 1883
See the getting started example here:
https://pypi.python.org/pypi/paho-mqtt/0.9#usage-and-api
I am trying to access linkedin API using python code. Here's the code that I am running on my windows machine:
from linkedin import server
import webbrowser
API_KEY = "<API_KEY>"
API_SECRET = "<API_SECRET>"
application = server.quick_api(API_KEY, API_SECRET)
I am executing these statements one-by-one in the console, but when I execute server.quick_api(API_KEY, API_SECRET), I receive the following error:
>>> application = server.quick_api(API_KEY, API_SECRET)
https://www.linkedin.com/uas/oauth2/authorization?scope=r_basicprofile%20rw_nus%20r_network%20r_contactinfo%20w_messages%20rw_groups%20r_emailaddress%20r_fullprofile&state=0b0290ff6e51e14c5409434a6e4bf52f&redirect_uri=http%3A//localhost%3A8000/&response_type=code&client_id=<API_KEY>
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\IPython\core\interactiveshell.py", line 2731, in run_code
exec code_obj in self.user_global_ns, self.user_ns
File "<ipython-input-9-eb867f0fb231>", line 1, in <module>
application = server.quick_api(API_KEY, API_SECRET)
File "C:\Users\sony\AppData\Roaming\Python\Python27\site-packages\linkedin\server.py", line 24, in quick_api
_wait_for_user_to_enter_browser(app)
File "C:\Users\sony\AppData\Roaming\Python\Python27\site-packages\linkedin\server.py", line 38, in _wait_for_user_to_enter_browser
httpd = BaseHTTPServer.HTTPServer(server_address, MyHandler)
File "C:\Python27\lib\SocketServer.py", line 419, in __init__
self.server_bind()
File "C:\Python27\lib\BaseHTTPServer.py", line 108, in server_bind
SocketServer.TCPServer.server_bind(self)
File "C:\Python27\lib\SocketServer.py", line 430, in server_bind
self.socket.bind(self.server_address)
File "C:\Python27\lib\socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
error: [Errno 10013] An attempt was made to access a socket in a way forbidden by its access permissions
I get a url which I understand I am supposed to get, but what's the cause of this error?
Also, when I paste the url in the browser, it takes me to linkedin page and asks for my credentials which it never accepts (I am sure I am entering the correct credentials). It always shows:
"HTTP Error 404. The requested resource is not found."
I am otherwise able to login to linkedin using the same credentials.