I'm trying to figure out how to deploy a Bokeh slider chart over an IIS server.
I recently finished up a Flask application, so I figured I'd try the route where you embed through flask:
https://github.com/bokeh/bokeh/tree/master/examples/howto/server_embed
It's nice and easy when I launch the script locally.. but I can't seem to set it up properly over IIS. I believe the complexity is stemming from the fact that the wfastcgi.py module I'm using to deploy over IIS can't easily multi-thread without some sort of hack-like work around.
So, my second attempt was to wrap the flask app in tornado as below OPTION B
(without much success, but still think this is my best lead here)
Run Flask as threaded on IIS 7
My third attempt was to try and run Bokeh server standalone on a specific port. I figured I'd be able to run the server via standalone_embed.py using wfastcgi.py on say port 8888 & while using port 5000 for the server callbacks. However, the Server function:
from bokeh.server.server import Server
still launches it locally on the host machine
server = Server({'/': bokeh_app}, io_loop=io_loop, port=5000)
server.start()
So this actually works if I go to http://localhost:5000/ on the host,
but fails if I go to http://%my_host_ip%:5000/ from a remote machine.
I even tried manually setting the host but get an "invalid host" error:
server = Server({'/': bokeh_app}, io_loop=io_loop, host='%my_host_ip_address_here%:5000')
server.start()
ERR:
Error occurred while reading WSGI handler: Traceback (most recent call last): File "C:\Python34\lib\site-packages\bokeh\server\server.py", line 45, in _create_hosts_whitelist int(parts[1]) ValueError: invalid literal for int() with base 10: '' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\WebsitesFlask\bokehTest\wfastcgi.py", line 711, in main env, handler = read_wsgi_handler(response.physical_path) File "C:\WebsitesFlask\bokehTest\wfastcgi.py", line 568, in read_wsgi_handler return env, get_wsgi_handler(handler_name) File "C:\WebsitesFlask\bokehTest\wfastcgi.py", line 537, in get_wsgi_handler handler = import(module_name, fromlist=[name_list[0][0]]) File ".\app.py", line 41, in server = Server({'/': bokeh_app}, io_loop=io_loop, host='%my_host_ip_address_here%:5000') File "C:\Python34\lib\site-packages\bokeh\server\server.py", line 123, in init tornado_kwargs['hosts'] = _create_hosts_whitelist(kwargs.get('host'), self._port) File "C:\Python34\lib\site-packages\bokeh\server\server.py", line 47, in _create_hosts_whitelist raise ValueError("Invalid port in host value: %s" % host) ValueError: Invalid port in host value: : StdOut: StdErr:
First off, the --host parameter should no longer be needed in the next 0.12.5 release. It's probably been the most confusing stumbling block for people trying to deploy a Bokeh server app in a "production" environment. You can follow the discussion on this issue on GitHub for more details.
Looking at the actual implementation in Bokeh that generates the error you are seeing, it is just this:
parts = host.split(':')
if len(parts) == 1:
if parts[0] == "":
raise ValueError("Empty host value")
hosts.append(host+":80")
elif len(parts) == 2:
try:
int(parts[1])
except ValueError:
raise ValueError("Invalid port in host value: %s" % host)
The exception you are reporting that states that int(parts[1]) is failing:
Traceback (most recent call last):
File "C:\Python34\lib\site-packages\bokeh\server\server.py", line 45,
in _create_hosts_whitelist int(parts[1])
ValueError: invalid literal for int() with base 10:
So, there is something amiss with the string you are passing for hosts that's causing the part after the colon to not be able to be converted to in int But without seeing the actual string, it's impossible to say much more. Maybe there is some encoding issue that needs to be handled differently or better. If you can provide a concrete string example that reproduces the problem I can take a closer look.
Related
I am trying to check if a domain name has MX records resolved using dnspython module. I am getting the following error while connecting to the mx record server. Can anyone explain why I am facing this issue?
Traceback (most recent call last):
File "c:\Users\iamfa\OneDrive\Desktop\test\email_mx.py", line 26, in <module>
dns.resolver.resolve("cmrit.ac.in", 'MX')
File "c:\Users\iamfa\OneDrive\Desktop\test\env1\lib\site-packages\dns\resolver.py", line 1193, in resolve
return get_default_resolver().resolve(qname, rdtype, rdclass, tcp, source,
File "c:\Users\iamfa\OneDrive\Desktop\test\env1\lib\site-packages\dns\resolver.py", line 1066, in resolve
timeout = self._compute_timeout(start, lifetime,
File "c:\Users\iamfa\OneDrive\Desktop\test\env1\lib\site-packages\dns\resolver.py", line 879, in _compute_timeout
raise LifetimeTimeout(timeout=duration, errors=errors)
dns.resolver.LifetimeTimeout: The resolution lifetime expired after 5.001 seconds: Server 10.24.0.1 UDP port 53 answered The DNS operation timed out.; Server 198.51.100.1 UDP port 53 answered The DNS operation timed out.; Server 10.95.11.110 UDP port 53 answered The DNS operation timed out.
This is my code:
import dns.resolver
if dns.resolver.resolve("cmrit.ac.in", 'MX'):
print(True)
else:
print(False)
However it was working fine till yesterday but when I try to run the same code today I am facing this issue.
If the remote DNS server takes a long time to respond, or accepts the connection but does not respond at all, the only thing you can really do is move along. Perhaps try again later. You can catch the error with try/except:
import dns.resolver
try:
if dns.resolver.resolve("cmrit.ac.in", 'MX'):
print(True)
else:
print(False)
except dns.resolver.LifetimeError:
print("timed out, try again later maybe?")
If you want to apply a longer timeout, the resolve method accepts a lifetime keyword argument which is documented in the Resolver.resolve documentation.
The Resolver class (documented at the top of the same page) also has a timeout parameter you can tweak if you build your own resolver.
For production code, you should probably add the other possible errors to the except clause; the exemplary documentation shows you precisely which exceptions resolve can raise.
...
except (dns.resolver.LifetimeTimeout, dns.resolver.NXDOMAIN,
dns.resolver.YXDOMAIN, dns.resolver.NoAnswer,
dns.resolver.NoNameservers) as err:
print("halp, something went wrong:", err)
Probably there is a base exception class which all of these inherit from; I was too lazy to go back and check. Then you only have to list the base class in the except statement.
It's probably more useful to extract the actual MX record and display it, rather than just print True, but that's a separate topic.
Your error message indicates that you were able to connect to your own resolver at 10.24.0.1 but in the general case, this error could also happen if your network (firewall etc) prevents you from accessing DNS for some reason.
I am trying to create a P2P node using python (pyp2p) but I am getting this error:
Eamons-MacBook-Pro:blockchain eamonwhite$ python3 serveralice.py
HTTP Error 404: Not Found
HTTP Error 404: Not Found
HTTP Error 404: Not Found
HTTP Error 404: Not Found
Traceback (most recent call last):
File "/Users/eamonwhite/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pyp2p/net.py", line 732, in start
rendezvous_con = self.rendezvous.server_connect()
File "/Users/eamonwhite/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pyp2p/rendezvous_client.py", line 92, in server_connect
con.connect(server["addr"], server["port"])
File "/Users/eamonwhite/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pyp2p/sock.py", line 189, in connect
self.s.bind((src_ip, 0))
TypeError: str, bytes or bytearray expected, not NoneType
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "serveralice.py", line 10, in <module>
alice.start()
File "/Users/eamonwhite/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pyp2p/net.py", line 735, in start
raise Exception("Unable to connect to rendezvous server.")
Exception: Unable to connect to rendezvous server.
My relevant code looks like this:
from uuid import uuid4
from blockchain import Blockchain
from flask import Flask, jsonify, request
from pyp2p.net import *
import time
#Setup Alice's p2p node.
alice = Net(passive_bind="192.168.1.131", passive_port=44444, interface="en0", node_type="passive", debug=1)
alice.start()
alice.bootstrap()
alice.advertise()
while 1:
for con in alice:
for reply in con:
print(reply)
time.sleep(1)
...
It is getting stuck on the Net function right at the beginning - something to do with the rendezvous package. The IP is my IP on the my network, and I port forwarded 44444 although I'm not sure if I need to do that or not. Thanks.
I am new to this, apparently with the way the server code was configured, it needed a rendezvous server to work (a node that handles all the other nodes). It is in net.py of the pyp2p package:
# Bootstrapping + TCP hole punching server.
rendezvous_servers = [
{
"addr": "162.243.213.95",
"port": 8000
}
]
The address was the problem, obviously it is just a placeholder IP. So then I realized I needed my own rendezvous server, and I used this code - https://raw.githubusercontent.com/StorjOld/pyp2p/master/pyp2p/rendezvous_server.py.
However I had to debug this file a little, it ended up needing to have import sys, import time and import re statements at the top before it would work. Now I am going to host it on my raspberry pi so that it is always up to handle nodes :)
I have coded a wireless probe sniffer using python + scapy. I'm want to use this script in openwrt routers.
Everytime it captures a probe request from nearby devices, the information is send to a webservice. (mac, power and probe).
My problem is the high consumption of CPU. The script runs quite good in my laptop (it takes between 50-70% of cpu), but when I run it in an openwrt router (400mhz cpu,16 ram) it takes 99%.
It's a well known bug in scapy the lost of packets with high loads (I have tested at the same time the script in my laptop and in the router and the router do not catched all the available packets in the air)
I already made some optimizations to the code, but I think there's more room for improvement.
This is the script.
#!/usr/bin/python
from scapy.all import *
import time
import thread
import requests
from datetime import datetime
PROBE_REQUEST_TYPE=0
PROBE_REQUEST_SUBTYPE=4
buf={'arrival':0,'source':0,'dest':0,'pwr':0,'probe':0}
uuid='1A2B3'
def PacketHandler(pkt):
global buf
if pkt.haslayer(Dot11):
if pkt.type==PROBE_REQUEST_TYPE and pkt.subtype == PROBE_REQUEST_SUBTYPE:
arrival= int(time.mktime(time.localtime()))
try:
extra = pkt.notdecoded
except:
extra=None
if extra!=None:
signal_strength = -(256-ord(extra[-4:-3]))
else:
signal_strength = -100
source = pkt.addr2
dest= pkt.addr3
pwr=signal_strength
probe=pkt.getlayer(Dot11).info
if buf['source']!=source and buf['probe']!=probe:
print 'launch %r %r %r' % (source,dest,probe)
buf={'arrival':arrival,'source':source,'dest':dest,'pwr':pwr,'probe':probe}
try:
thread.start_new_thread(exporter,(arrival,source,dest,pwr,probe))
except:
print 'Error launching the thread %r' % source
def exporter (arrival,source,dest,pwr,probe):
global uuid
urlg='http://webservice.com/?arrival='+str(arrival)+'&source='+str(source)+'&dest='+str(dest)+'&pwr='+str(pwr)+'&probe='+str(probe)+'&uuid='+uuid
try:
r=requests.get(urlg)
print r.status_code
print r.content
except:
print 'ERROR in Thread:::::: %r' % source
def main():
print "[%s] Starting scan"%datetime.now()
sniff(iface=sys.argv[1],prn=PacketHandler,store=0)
if __name__=="__main__":
main()
[UPDATE]
After a lot of reading and deep searching (it seems not many people have found full solution to the same issue or something similar).
I have found you can filter directly from the sniff function, so I've added a filter to just catch probe requests.
def main():
print "[%s] Starting scan"%datetime.now()
sniff(iface=sys.argv[1],prn=PacketHandler, filter='link[26] = 0x40',store=0)
In my laptop runs really smooth, using between 1%-3% of cpu and catching most of the available packets in the air.
but when I run this on the router, the script throws an error and crash.
Traceback (most recent call last):
File "snrV2.py", line 66, in <module>
main()
File "snrV2.py", line 63, in main
sniff(iface=sys.argv[1],prn=PacketHandler, filter='link[26] = 0x40',store=0)
File "/usr/lib/python2.7/site-packages/scapy/sendrecv.py", line 550, in sniff
s = L2socket(type=ETH_P_ALL, *arg, **karg)
File "/usr/lib/python2.7/site-packages/scapy/arch/linux.py", line 460, in __init__
attach_filter(self.ins, filter)
File "/usr/lib/python2.7/site-packages/scapy/arch/linux.py", line 132, in attach_filter
s.setsockopt(SOL_SOCKET, SO_ATTACH_FILTER, bpfh)
File "/usr/lib/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 99] Protocol not available
I have tried using bpf filter syntax (the same used in tcpdump http://biot.com/capstats/bpf.html) and it is supposed you can also use it in scapy but I get a filter syntax error.
Sniff fucntion:
def main():
print "[%s] Starting scan"%datetime.now()
sniff(iface=sys.argv[1],prn=PacketHandler, filter='type mgt subtype probe-req', store=0)
error:
Traceback (most recent call last):
File "snrV2.py", line 66, in <module>
main()
File "snrV2.py", line 63, in main
sniff(iface=sys.argv[1],prn=PacketHandler, filter='type mgt subtype probe-req', store=0)
File "/usr/lib/python2.7/site-packages/scapy/sendrecv.py", line 550, in sniff
s = L2socket(type=ETH_P_ALL, *arg, **karg)
File "/usr/lib/python2.7/site-packages/scapy/arch/linux.py", line 460, in __init__
attach_filter(self.ins, filter)
File "/usr/lib/python2.7/site-packages/scapy/arch/linux.py", line 120, in attach_filter
raise Scapy_Exception("Filter parse error")
NameError: global name 'Scapy_Exception' is not defined
In the router I have installed the last version of scapy and tcpdump.
Now I really don't know what to do.
I encountered a similar error (socket.error: [Errno 99] Protocol not available) when I tried to use sniff() with a filter on my NETGEAR WNDR4300.
After a lot of searching on Google, I found that the reason is that the Linux kernel of my router does not enable CONFIG_PACKET. It is mentioned in the Scapy installation guide, as follows:
Make sure your kernel has Packet sockets selected (CONFIG_PACKET)
If your kernel is < 2.6, make sure that Socket filtering is selected CONFIG_FILTER)
If you set CONFIG_PACKET=y when you compile the kernel, then it will enable the BPF for underlying socket.
I'm using localtunnel v1. But I found that v2 allows you to customize the subdomain, and I need this feature.
I followed the tutorial described in the README from the repository, but it confused me in several parts and, in the end, it did not work.
First step is to run some web-app: checked, on port no. 8000.
Then, it says something about hostnames:
Localtunnel does some stuff with the hostname, so you want to set up two
hostnames. One for localtunnel registration, one for your localtunnel.
Normally it expects a wildcard, but we'll just hardcode a hostname for
this example tunnel.
example.localtunnel.local -> 127.0.0.1
localtunnel.local -> 127.0.0.1
You can do this in /etc/hosts or use that fancy ghost utility.
I've got lost here, but still I edited my /etc/hosts:
127.0.0.1 localhost
127.0.1.1 my-pc-name
127.0.0.1 example.localtunnel.local
127.0.0.1 localtunnel.local
Next step...
Now you can start the server. It's based on a configuration file in the
config directory. You can make your own, but this one is configured to
run the server on port 9999 and expects the hostname localtunnel.local
ginkgo config/default.conf.py
Which one? Anyway... I created myconfig.conf.py based on the files in localtunnel repo's dir /deploy:
port = 9999
hostname = 'localtunnel.local'
service = 'localtunnel.server.TunnelBroker'
But, when I run:
lt --broker 127.0.0.1:9999 --name example 8000
I got:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/gevent/greenlet.py", line 390, in run
result = self._run(*self.args, **self.kwargs)
File "/usr/local/lib/python2.7/dist-packages/localtunnel/client.py", line 53, in listen
msg = self.ws.receive(msg_obj=True)
TypeError: receive() got an unexpected keyword argument 'msg_obj'
<Greenlet at 0xb6e0db1cL: <bound method TunnelClient.listen of <localtunnel.client.TunnelClient object at 0xb6def52c>>> failed with TypeError
And in the ginkgo process:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/gevent/pywsgi.py", line 438, in handle_one_response
self.run_application()
File "/usr/local/lib/python2.7/dist-packages/ws4py/server/geventserver.py", line 85, in run_application
self.result = self.application(self.environ, start_response_for_upgrade)
File "/usr/local/lib/python2.7/dist-packages/ws4py/server/wsgi/middleware.py", line 131, in __call__
environ.copy()))
TypeError: handle_websocket() takes exactly 3 arguments (2 given)
<BrokerFrontend fileno=6 address=0.0.0.0:9999>: Failed to handle request:
request = GET /t/example HTTP/1.1 from ('127.0.0.1', 35907)
application = <ws4py.server.wsgi.middleware.WebSocketUpgradeMiddleware object at 0x95bc2ac>
127.0.0.1 - - [2012-05-14 17:18:18] "GET /t/example HTTP/1.1" 101 162 0.000933
And, obviously, http://example.localtunnel.local:9999 does not work.
How to fix this? And where I have to modify to change the final subdomain?
Sorry about the creepy english.
Edit
I've followed the paul suggestion and did the downgrading. But although changes have happened, errors still occur. ginkgo process:
$ ginkgo eco.conf.py
Starting process with eco.conf.py...
127.0.0.1 - - [2012-05-22 20:21:11] "GET /t/example HTTP/1.1" 400 116 0.000190
localtunnel process:
$ lt --broker 127.0.0.1:9999 --name example 8000
Traceback (most recent call last):
File "/usr/local/bin/lt", line 9, in <module>
load_entry_point('localtunnel==0.4.0', 'console_scripts', 'lt')()
File "/usr/local/lib/python2.7/dist-packages/localtunnel/client.py", line 31, in main
client.serve_forever()
File "/usr/local/lib/python2.7/dist-packages/ginkgo/core.py", line 188, in serve_forever
self.start()
File "/usr/local/lib/python2.7/dist-packages/ginkgo/core.py", line 124, in start
ready = not self.do_start()
File "/usr/local/lib/python2.7/dist-packages/localtunnel/client.py", line 42, in do_start
self.ws.connect()
File "/usr/local/lib/python2.7/dist-packages/ws4py-0.1.5-py2.7.egg/ws4py/client/threadedclient.py", line 72, in connect
self.process_response_line(response_line)
File "/usr/local/lib/python2.7/dist-packages/ws4py-0.1.5-py2.7.egg/ws4py/client/__init__.py", line 61, in process_response_line
raise HandshakeError("Invalid response status: %s %s" % (code, status))
ws4py.exc.HandshakeError: Invalid response status: 400 Bad Handshake
Although ginkgo does not give any error now, localtunnel still raising errors different from previous errors. Apparently it tries to GET "/t/example" in the connecting process.
It looks like this software is expecting an older version of ws4py. The current version (0.2.1) of ws4py matches what it looks like you have, while the 0.1.5 version of ws4py matches what localtunnel is trying to use.
Downgrading to ws4py 0.1.5 may be sufficient to solve the problems you're having.
On the other hand, though, this doesn't seem like the best-supported software in the world. Are you sure it's the right solution for your problem? I've looked through the code and all the docs provided in that repo, and what I get is that it sets up this weird tcp-tunnel-over-json-over-websockets (yeah, websockets, for something with python on both the server and client side!) thing, without even providing any particular security or encryption or robustness capabilities of its own, and it appears to do nothing that other more common tools can't do better. But granted, I may be missing something important.
I think you must be following the instructions to set up the localtunnel.com server (ie. if you wanted to run your own localtunnel server on some other domain).
Installing localtunnel v2 for normal use should be as easy as running pip install localtunnel (possibly with sudo).
Once that's done, just run localtunnel-beta -n <subdomain> 8000
For more details, see Jeff's blog post.
This question is related to How do we handle Python xmlrpclib Connection Refused?
When I try to use the following code, with my RPC server down, _get_rpc() returns False and I'm good to go. However if the server is running, it fails with unknown method. Is it trying to execute .connect() on the remote server? How can I get around this, when I needed to use .connect() to detect if the returned proxy worked (see related question)?
import xmlrpclib
import socket
def _get_rpc():
try:
a = xmlrpclib.ServerProxy('http://dd:LNXFhcZnYshy5mKyOFfy#127.0.0.1:9001')
a.connect() # Try to connect to the server
return a.supervisor
except socket.error:
return False
if not _get_rpc():
print "Failed to connect"
Here is the issue:
ahiscox#lenovo:~/code/dd$ python xmlrpctest2.py
Failed to connect
ahiscox#lenovo:~/code/dd$ supervisord -c ~/.supervisor # start up RPC server
ahiscox#lenovo:~/code/dd$ python xmlrpctest2.py
Traceback (most recent call last):
File "xmlrpctest2.py", line 13, in <module>
if not _get_rpc():
File "xmlrpctest2.py", line 7, in _get_rpc
a.connect() # Try to connect to the server
File "/usr/lib/python2.6/xmlrpclib.py", line 1199, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python2.6/xmlrpclib.py", line 1489, in __request
verbose=self.__verbose
File "/usr/lib/python2.6/xmlrpclib.py", line 1253, in request
return self._parse_response(h.getfile(), sock)
File "/usr/lib/python2.6/xmlrpclib.py", line 1392, in _parse_response
return u.close()
File "/usr/lib/python2.6/xmlrpclib.py", line 838, in close
raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 1: 'UNKNOWN_METHOD'>
Well i was just looking in to it ; my old method suck because xmlrpclib.ServerProxy try to connect to the XmlRPC server when you call a method, not before !!!
Try this instead :
import xmlrpclib
import socket
def _get_rpc():
a = xmlrpclib.ServerProxy('http://dd:LNXFhcZnYshy5mKyOFfy#127.0.0.1:9001')
try:
a._() # Call a fictive method.
except xmlrpclib.Fault:
# connected to the server and the method doesn't exist which is expected.
pass
except socket.error:
# Not connected ; socket error mean that the service is unreachable.
return False, None
# Just in case the method is registered in the XmlRPC server
return True, a
connected, server_proxy = _get_rpc():
if not connected
print "Failed to connect"
import sys
sys.exit(1)
To summarize this we have 3 cases :
XmlRPC server is up and in it we defined a method called _():
(EDIT : i did choose the name _ because it unlikely to have a method with this name, but this case still can happen)
In this case no exception will be catch and the code will execute
the return True
XmlRPC server is up and in it we don't have any method methoded call
_():
This time xmlrpclib.Fault will be raised and we will also pass to the return True
XmlRPC server is down:
Now the socket.error exception will be raised and that
when we call a._() so we should return False
I don't know if there is an easy way to do this and i will love to see it until then , hope this can fix thing this time :)
N.B: when you do if a: python will again search for a method __nonzero__() to test the boolean value of a and this will fail to.
N.B 2: Some xmlrpc service offer a rpc path specialized to do an authentication , in this path the service offer methods like login() ... , this kinds of method can replace the _() method in our case, so just calling login(), will be enough to know if the service is up or down (socket.error), and in the same time this login() method authenticate the user if the service is up .