Let's run a simple Bottle server:
from bottle import route, run
#route('/')
def index():
return 'Hello'
if __name__ == "__main__":
run(port=80)
I wanted to test the speed of the requests with:
import requests
for i in range(10):
requests.get('http://localhost/?param=%i' % i)
and then I was about to use multithreading/parallel to also test what happens if 10 requests arrive at the same time, and not one after another... but I was stopped by something strange:
Animated console of the incoming requests
Each request seems to take precisely 1 second!
Why such a long time for each request on a Bottle server? (I know it's a micro-framework, but this is surely not the reason for 1-second to respond!)
PS: it's the same with urllib.request.urlopen. This code uses the default WSGIRefServer:
Bottle v0.12.18 server starting up (using WSGIRefServer())...
and was tested on Windows 7 + Python 3.7 64 bit.
Related
import sys
import os
import logging
import requests
import json
import pytest
from multiprocessing import Process
from flask_file import main
#pytest.fixture
def endpoint():
return "http://127.0.0.1:8888/"
def test_send_request(endpoint: str):
server = Process(target=main)
server.start()
time.sleep(30)
# check that the service is up and running
service_up = requests.get(f"{endpoint}")
server.terminate()
server.join()
I wanted to spin up and spin down a server locally from within a test to test some requests. I know the server itself works because I can run main() from the flask_file itself using python flask_file and it will spin up the server...and I can ping it just fine. When I use the above method, the test does seem to do the full 30s sleep without failing, but in those 30s I cannot open the endpoint on my browser and see the expected "hello world".
When you run the Flask builtin development server (e.g. flask run or app.run(), only one connection is possible. So when your test accesses the app, you cannot access it via browser.
Anyway, you should rewrite your test and fixture to use the test_client, see the official documentation
https://flask.palletsprojects.com/en/1.1.x/testing/
My end goal is to have a button on my website (dashboard created in React) which allows me to run a Selenium test (written in python).
I am using socket.io in hopes that I can stream test results live back to dashboard, but I seem to be hitting some sort of time limit at about 29 seconds.
To debug I made this test case, which completes on the server side, but my connection is severed before emit('test_progress', 29) happens.
from flask import Flask, render_template
from flask_socketio import SocketIO, join_room, emit
import time
app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app)
#socketio.on('run_test')
def handle_run_test(run_test):
print('received run_test')
for x in range(1, 30):
time.sleep(1)
print(x)
emit('test_progress', x)
time.sleep(1)
print('TEST FINISHED')
emit('test_finished', {'data':None})
if __name__ == '__main__':
socketio.run(app, debug=True)
(Some of) my JavaScript
import settings from './settings.js';
import io from 'socket.io-client';
const socket = io(settings.socketio);
socket.on('test_progress', function(data){
console.log(data);
});
My console in browser
...
App.js:154 27
App.js:154 28
polling-xhr.js:269 POST http://127.0.0.1:5000/socket.io/?EIO=3&transport=polling&t=Mbl7mEI&sid=72903901182d49eba52a4a813772eb06 400 (BAD REQUEST)
...
(reconnects)
Eventually, I'll have a test running that could take 40-60 seconds instead of the arbitrary time.sleep(1) calls, so I would like the function to be able to use more than 29 seconds. Am I going about this wrong or is there a way to change this time limit?
My solution was to use threading as described in this question
I also needed to implement #copy_current_request_context so that the thread could communicate
I am new to Python programming and the Bottle framework as well. I wrote up a basic hello, world program which looks like this:
from bottle import run, route
#route('/')
def index():
return '<h1>Hello, World</h1>'
if __name__ == '__main__':
run(host='localhost', port=8080, debug=True)
The output of this code is
Bottle v0.12.13 server starting up (using WSGIRefServer())...
Listening on http://localhost:8080/
Hit Ctrl-C to quit.
But when I enter http://localhost:8080/ in a browser, i get the error- "The site cannot be reached"
Am I missing some configuration. I am learning using this youtube video
Make sure that no other programs are accessing the server. Are you using IPyhton as well? Just a sanity check nothing fancy.
I'm new to python and currently researching its viability to be used as a soap server. I currently have a very rough application that uses the mysql blocking api, but would like to try twisted adbapi. I've successfully used twisted adbapi on regular twisted code using reactors, but can't seem to make it work with code below using ZSI framework. It's not returning anything from mysql. Anyone ever used twisted adbapi with ZSI?
import os
import sys
from dpac_server import *
from ZSI.twisted.wsgi import (SOAPApplication,
soapmethod,
SOAPHandlerChainFactory)
from twisted.enterprise import adbapi
import MySQLdb
def _soapmethod(op):
op_request = GED("http://www.example.org/dpac/", op).pyclass
op_response = GED("http://www.example.org/dpac/", op + "Response").pyclass
return soapmethod(op_request.typecode, op_response.typecode,operation=op, soapaction=op)
class DPACServer(SOAPApplication):
factory = SOAPHandlerChainFactory
#_soapmethod('GetIPOperation')
def soap_GetIPOperation(self, request, response, **kw):
dbpool = adbapi.ConnectionPool("MySQLdb", '127.0.0.1','def_user', 'def_pwd', 'def_db', cp_reconnect=True)
def _dbSPGeneric(txn, cmts):
txn.execute("call def_db.getip(%s)", (cmts, ))
return txn.fetchall()
def dbSPGeneric(cmts):
return dbpool.runInteraction(_dbSPGeneric, cmts)
def returnResults(results):
response.Result = results
def showError(msg):
response.Error = msg
response.Result = ""
response.Error = ""
d = dbSPGeneric(request.Cmts)
d.addCallbacks(returnResults, showError)
return request, response
def main():
from wsgiref.simple_server import make_server
from ZSI.twisted.wsgi import WSGIApplication
application = WSGIApplication()
httpd = make_server('127.0.0.1', 8080, application)
application['dpac'] = DPACServer()
print "listening..."
httpd.serve_forever()
if __name__ == '__main__':
main()
The code you posted creates a new ConnectionPool per (some kind of) request and it never stops the pool. This means you'll eventually run out of resources and you won't be able to service any more requests. "Eventually" is probably after one or two or three requests.
If you never get any responses perhaps this isn't the problem you've encountered. It will be a problem at some point though.
On closer inspection, I wonder if this code even runs the Twisted reactor at all. On first read, I thought you were using some ZSI Twisted integration to run your server. Now I see that you're using wsgiref.simple_server. I am moderately confident that this won't work.
You're already using Twisted, use Twisted's WSGI server instead.
Beyond that, verify that ZSI executes your callbacks in the correct thread. The default for WSGI applications is to run in a non-reactor thread. Twisted APIs are not thread-safe, so if ZSI doesn't do something to correct for this, you'll have bugs introduced by using un-thread-safe APIs in threads.
I have this simple flask/gevent demo code.
#!/usr/bin/env python
import gevent
from gevent.pywsgi import WSGIServer
from gevent import monkey
monkey.patch_socket()
from flask import Flask, Response
app = Flask(__name__)
#app.route('/')
def stream():
def gen():
for i in range(10):
yield "data: %d\r\n" % i
gevent.sleep(1)
return Response(gen())
if __name__ == '__main__':
http = WSGIServer(('', 5000), app)
http.serve_forever()
When I run it and open multiple urls in the browser, all but one of them block. What am I doing wrong?
I have tried running it with monkey.patch_all(), and running it with gunicorn streaming:app -k gevent - it still blocks in the browser.
Multiple tabs in browsers will block. That doesn't mean gevent/gunicorn isn't running the requests concurrently. I tried it with concurrent curl requests and XmlHttpRequest - it works as expected. Also note that curl buffers output. "\r\n" is required to make it print line by line.
Sidenote: Thanks to mitsuhiko on #pocoo for resolving it. If you haven't tried flask, you should. Both mitushiko and flask are awesome.