Here is my python code.
from aiohttp import web
if __name__ == "__main__":
app = web.Application()
cors = aiohttp_cors.setup(app)
app.on_shutdown.append(on_shutdown)
app.router.add_get("/", index)
app.router.add_get("/client.js", javascript)
cors = aiohttp_cors.setup(app, defaults={
"*": aiohttp_cors.ResourceOptions(
allow_credentials=True,
expose_headers="*",
allow_headers="*",
)
})
resource = cors.add(app.router.add_resource("/offer"))
cors.add(resource.add_route("POST", offer))
resource = cors.add(app.router.add_resource("/image_makeup"))
cors.add(resource.add_route("POST", image_makeup))
web.run_app(
app, access_log=None, host=args.host, port=args.port, ssl_context=ssl_context
)
Command:
gunicorn main:app --bind localhost:8080 --worker-class aiohttp.GunicornWebWorker
error:
[2020-10-17 21:17:13 +0530] [47676] [INFO] Starting gunicorn 20.0.4
[2020-10-17 21:17:13 +0530] [47676] [INFO] Listening at: http://127.0.0.1:8080 (47676)
[2020-10-17 21:17:13 +0530] [47676] [INFO] Using worker: sync
[2020-10-17 21:17:13 +0530] [47678] [INFO] Booting worker with pid: 47678
usage: gunicorn [-h] [--cert-file CERT_FILE] [--key-file KEY_FILE] [--host HOST] [--port PORT] [--verbose]
gunicorn: error: unrecognized arguments: --bind localhost:8080 wsgi:main
[2020-10-17 21:17:18 +0530] [47678] [INFO] Worker exiting (pid: 47678)
[2020-10-17 21:17:19 +0530] [47680] [INFO] Booting worker with pid: 47680
I have main.py inside my root directory still m getting above error.
Please have a look where i am making the mistake.
As you are not running directly your file "main.py", the value of the variable __name__ is not __main__ but main. You need to change your if statement :
if __name__ == "main":
app = web.Application()
...
As given in the documentation,
When running Gunicorn, you provide the name of the module, in your case main, and the name of the app or application factory, along with other Gunicorn Settings provided as command line flags or in your config file.
The entry point could be either an application instance or a coroutine that accepts no parameters and returns an application instance
Your Code has to be like:
from aiohttp import web
async def my_web_app():
app = web.Application()
....
return app
or
from aiohttp import web
my_web_app = web.Application()
....
Gunicorn command becomes:
gunicorn main:my_web_app --bind localhost:8080 --worker-class aiohttp.GunicornWebWorker
Related
I'm having a lot of trouble to deal with Cloud Firestore in native mode. I'm used to work with NDB, but tried to change my app to Flexible Environment, since I need to work with websocket. But, I'm really having a nightmare to use Firestore. I had to use Tasks API to do a simple batch commit, that worked well. But I can't retrieve a single document from database, neither locally or deployed. Locally, I keep getting:
[2019-08-31 20:05:17 -0300] [1951] [INFO] Booting worker with pid: 1951
[2019-08-31 20:05:47 -0300] [1927] [CRITICAL] WORKER TIMEOUT (pid:1951)
[2019-08-31 20:05:48 -0300] [1964] [INFO] Booting worker with pid: 1964
[2019-08-31 20:06:18 -0300] [1927] [CRITICAL] WORKER TIMEOUT (pid:1964)
[2019-08-31 20:06:19 -0300] [1977] [INFO] Booting worker with pid: 1977
I have a collection called Contacts with a document called 5599999999 with a pair of values: "name":"test". Also, I'm running locally with gunicorn, since I need websockets.
This is the command to run locally:
gunicorn -b 127.0.0.1:8080 -k flask_sockets.worker main:app --reload
And my app.yaml file:
runtime: python
env: flex
entrypoint: gunicorn -b :$PORT -k flask_sockets.worker main:app
runtime_config:
python_version: 3
manual_scaling:
instances: 1
resources:
cpu: 1
memory_gb: 0.5
disk_size_gb: 10
main.py
import googleapiclient.discovery
import asyncio
import google.cloud.exceptions
import firebase_admin
from google.cloud import firestore
from google.oauth2 import service_account
from firebase_admin import credentials, firestore
from flask import Flask, render_template
from flask_sockets import Sockets
cred = credentials.Certificate('service.json')
firebase_admin.initialize_app(cred)
app = Flask(__name__)
sockets = Sockets(app)
#app.route('/')
def index():
teste = GetContacts()
return str(teste.to_dict())
def GetContacts():
db = firestore.client()
contacts = db.collection(u'Contacts').document(u'5599999999').get()
return contacts
I simply expect the output of this to be the pair of values. But it get's running indefinitely...
Tried now, with only Flask. Runs smoothly. No problems at all and real fast. But, I'm also getting this kind of error, besides getting the pair of values:
$ python main.py
* Serving Flask app "main" (lazy loading)
* Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://127.0.0.1:8080/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: 325-288-398
127.0.0.1 - - [01/Sep/2019 01:03:53] "GET / HTTP/1.1" 200 -
ERROR:root:An error occurred during a request.
Traceback (most recent call last):
File "/Users/dr/env/lib/python3.7/site-packages/flask_sockets.py", line 40, in __call__
handler, values = adapter.match()
File "/Users/dr/env/lib/python3.7/site-packages/werkzeug/routing.py", line 1799, in match
raise NotFound()
werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/dr/env/lib/python3.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/Users/dr/env/lib/python3.7/site-packages/flask/app.py", line 1791, in dispatch_request
self.raise_routing_exception(req)
File "/Users/dr/env/lib/python3.7/site-packages/flask/app.py", line 1774, in raise_routing_exception
raise request.routing_exception
File "/Users/dr/env/lib/python3.7/site-packages/flask/ctx.py", line 336, in match_request
self.url_adapter.match(return_rule=True)
File "/Users/dr/env/lib/python3.7/site-packages/werkzeug/routing.py", line 1799, in match
raise NotFound()
werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
127.0.0.1 - - [01/Sep/2019 01:03:53] "GET /favicon.ico HTTP/1.1" 200 -
INFO:werkzeug:127.0.0.1 - - [01/Sep/2019 01:03:53] "GET /favicon.ico HTTP/1.1" 200 -
This is related to gevent (used by flask_sockets) and grpc (used by firebase) conflict described in this issue https://github.com/grpc/grpc/issues/4629 the solution is to add the following code
import grpc.experimental.gevent as grpc_gevent
grpc_gevent.init_gevent()
Gunicorn's default timeout setting is 30 seconds, which is consistent with what I'm seeing in your logs. I don't think this is a gunicorn issue. Have you tried making the connection without gunicorn?
I have this from /home/myname/myapp/app.py:
from flask import Flask
app = Flask(__name__)
print __name__
#app.route('/')
def index():
return "Hello world!"
if __name__ == '__main__':
print 'in if'
app.run()
When I run:
$ gunicorn app:app -b 127.0.0.2:8000
It says:
2013-03-01 11:26:56 [21907] [INFO] Starting gunicorn 0.17.2
2013-03-01 11:26:56 [21907] [INFO] Listening at: http://127.0.0.2:8000 (21907)
2013-03-01 11:26:56 [21907] [INFO] Using worker: sync
2013-03-01 11:26:56 [21912] [INFO] Booting worker with pid: 21912
app
So the __name__ of the app is app. Not __main__ like I need it to be to run the if statement.
I tried putting an empty __init__.py in the directory. Here is my nginx sites-enabled default:
server {
#listen 80; ## listen for ipv4; this line is default and implied
#listen [::]:80 default_server ipv6only=on; ## listen for ipv6
root /home/myname/myapp;
# Make site accessible from http://localhost/
server_name localhost;
location / {
proxy_pass http://127.0.0.2:8000;
}
}
Edit... While this app does print 'Hello world' when I visit the site. The point is that I need __name__ to equal '__main__'. I also just want to know why it doesn't and how to make it equal __main__.
Edit 2... I just had the epiphany that I do not need to run app.run() since that is what Gunicorn is for. Duh. But I would still like to figure out why __name__ isn't '__main__'
Python sets __name__ to "__main__" when the script is the entry point for the Python interpreter. Since Gunicorn imports the script it is running that script will not be the entry point and so will not have __name__ set to "__main__".
I have implemented a simple microservice using Flask, where the method that handles the request calculates a response based on the request data and a rather large datastructure loaded into memory.
Now, when I deploy this application using gunicorn and a large number of worker threads, I would simply like to share the datastructure between the request handlers of all workers. Since the data is only read, there is no need for locking or similar. What is the best way to do this?
Essentially what would be needed is this:
load/create the large data structure when the server is initialized
somehow get a handle inside the request handling method to access the data structure
As far as I understand gunicorn allows me to implement various hook functions, e.g. for the time the server gets initialized, but a flask request handler method does not know anything about the gunicorn server data structure.
I do not want to use something like redis or a database system for this, since all data is in a datastructure that needs to be loaded in memory and no deserialization must be involved.
The calculation carried out for each request which uses the large data structure can be lengthy so it must happen concurrently in a truly independent thread or process for each request (this should scale up by running on a multi-core computer).
You can use preloading.
This will allow you to create the data structure ahead of time, then fork each request handling process. This works because of copy-on-write and the knowledge that you are only reading from the large data structure.
Note: Although this will work, it should probably only be used for very small apps or in a development environment. I think the more production-friendly way of doing this would be to queue up these calculations as tasks on the backend since they will be long-running. You can then notify users of the completed state.
Here is a little snippet to see the difference of preloading.
# app.py
import flask
app = flask.Flask(__name__)
def load_data():
print('calculating some stuff')
return {'big': 'data'}
#app.route('/')
def index():
return repr(data)
data = load_data()
Running with gunicorn app:app --workers 2:
[2017-02-24 09:01:01 -0500] [38392] [INFO] Starting gunicorn 19.6.0
[2017-02-24 09:01:01 -0500] [38392] [INFO] Listening at: http://127.0.0.1:8000 (38392)
[2017-02-24 09:01:01 -0500] [38392] [INFO] Using worker: sync
[2017-02-24 09:01:01 -0500] [38395] [INFO] Booting worker with pid: 38395
[2017-02-24 09:01:01 -0500] [38396] [INFO] Booting worker with pid: 38396
calculating some stuff
calculating some stuff
And running with gunicorn app:app --workers 2 --preload:
calculating some stuff
[2017-02-24 09:01:06 -0500] [38403] [INFO] Starting gunicorn 19.6.0
[2017-02-24 09:01:06 -0500] [38403] [INFO] Listening at: http://127.0.0.1:8000 (38403)
[2017-02-24 09:01:06 -0500] [38403] [INFO] Using worker: sync
[2017-02-24 09:01:06 -0500] [38406] [INFO] Booting worker with pid: 38406
[2017-02-24 09:01:06 -0500] [38407] [INFO] Booting worker with pid: 38407
I have a Flask app with the following route:
#app.route('/')
def index():
console = logging.StreamHandler()
log = logging.getLogger("asdasd")
log.addHandler(console)
log.setLevel(logging.DEBUG)
log.error("Something")
print >> sys.stderr, "Another thing"
return 'ok'
I run this using
python gunicorn --access-logfile /mnt/log/test.log --error-logfile /mnt/log/test.log --bind 0.0.0.0:8080 --workers 2 --worker-class gevent --log-level debug server:app
The Logs are as below:
2014-06-26 00:13:55 [21621] [INFO] Using worker: gevent
2014-06-26 00:13:55 [21626] [INFO] Booting worker with pid: 21626
2014-06-26 00:13:55 [21627] [INFO] Booting worker with pid: 21627
2014-06-26 00:14:05 [21626] [DEBUG] GET /
10.224.67.41 - - [26/Jun/2014:00:14:14 +0000] "GET / HTTP/1.1" 200 525 "-" "python-requests/2.2.1 CPython/2.7.5 Darwin/13.2.0"
2014-06-26 00:14:14 [21626] [DEBUG] Closing connection.
What's happening to my logs in index method?
As of Gunicorn 19.0, gunicorn has stopped redirecting stderr to its logs.
Refer to https://github.com/benoitc/gunicorn/commit/41523188bc05fcbba840ba2e18ff67cd9df638e9
I have this from /home/myname/myapp/app.py:
from flask import Flask
app = Flask(__name__)
print __name__
#app.route('/')
def index():
return "Hello world!"
if __name__ == '__main__':
print 'in if'
app.run()
When I run:
$ gunicorn app:app -b 127.0.0.2:8000
It says:
2013-03-01 11:26:56 [21907] [INFO] Starting gunicorn 0.17.2
2013-03-01 11:26:56 [21907] [INFO] Listening at: http://127.0.0.2:8000 (21907)
2013-03-01 11:26:56 [21907] [INFO] Using worker: sync
2013-03-01 11:26:56 [21912] [INFO] Booting worker with pid: 21912
app
So the __name__ of the app is app. Not __main__ like I need it to be to run the if statement.
I tried putting an empty __init__.py in the directory. Here is my nginx sites-enabled default:
server {
#listen 80; ## listen for ipv4; this line is default and implied
#listen [::]:80 default_server ipv6only=on; ## listen for ipv6
root /home/myname/myapp;
# Make site accessible from http://localhost/
server_name localhost;
location / {
proxy_pass http://127.0.0.2:8000;
}
}
Edit... While this app does print 'Hello world' when I visit the site. The point is that I need __name__ to equal '__main__'. I also just want to know why it doesn't and how to make it equal __main__.
Edit 2... I just had the epiphany that I do not need to run app.run() since that is what Gunicorn is for. Duh. But I would still like to figure out why __name__ isn't '__main__'
Python sets __name__ to "__main__" when the script is the entry point for the Python interpreter. Since Gunicorn imports the script it is running that script will not be the entry point and so will not have __name__ set to "__main__".