Develop Reference

How to return a jpg / jpeg with gunicorn

I can return an html file with gunicorn, but I am trying to return an image file being referenced in an <img> tag. The image is not rendering in the browser, and in fact I see no response being returned in the browser tools for the image.
This is my gunicorn code:
def render_jpeg(filename):
encoded_string = ""
with open(filename, "rb") as image_file:
encoded_string = base64.b64encode(image_file.read())
return encoded_string
def app(environ, start_response):
path = environ.get("PATH_INFO")
data = ""
encode = "utf-8"
content_type = 'text/html'
if path.endswith(".jpeg"):
content_type = 'image/jpeg'
encode = "byte"
data = render_jpeg(path)
else:
context = {"path": path, "pageTitle": "home"}
data = template("template.html", context)
if encode.endswith("utf-8"):
data = data.encode(encode)
status = '200 OK'
response_headers = [
('Content-type', content_type),
('Content-Length', str(len(data)))
]
start_response(status, response_headers)
return iter([data])
This is my command
gunicorn --workers=2 test:app --reload --log-file log.txt --log-level 'debug'
This is my log output
[2022-07-08 14:28:21 -0400] [91974] [INFO] Starting gunicorn 20.1.0
[2022-07-08 14:28:21 -0400] [91974] [DEBUG] Arbiter booted
[2022-07-08 14:28:21 -0400] [91974] [INFO] Listening at: http://127.0.0.1:8000 (91974)
[2022-07-08 14:28:21 -0400] [91974] [INFO] Using worker: sync
[2022-07-08 14:28:21 -0400] [91976] [INFO] Booting worker with pid: 91976
[2022-07-08 14:28:21 -0400] [91977] [INFO] Booting worker with pid: 91977
[2022-07-08 14:28:21 -0400] [91974] [DEBUG] 2 workers
[2022-07-08 14:28:47 -0400] [91977] [DEBUG] GET /media/zombie1.jpeg
[2022-07-08 14:28:47 -0400] [91977] [DEBUG] GET /favicon.ico
[2022-07-08 14:30:33 -0400] [91974] [INFO] Handling signal: int
[2022-07-08 14:30:33 -0400] [91976] [INFO] Worker exiting (pid: 91976)
[2022-07-08 14:30:33 -0400] [91977] [INFO] Worker exiting (pid: 91977)
[2022-07-08 14:30:33 -0400] [91974] [INFO] Shutting down: Master
Where am I going wrong?

As #tripleee noted, my issue was the base64 encoding. No encoding required! Corrected code:
def render_jpeg(filename):
data = ""
with open(filename, "rb") as image_file:
data = image_file.read()
return data
def app(environ, start_response):
path = environ.get("PATH_INFO")
data = ""
encode = "utf-8"
content_type = 'text/html'
if path.endswith(".jpeg"):
content_type = 'image/jpeg'
encode = ""
filename = get_filename_from_path(path)
data = render_jpeg(filename)
else:
context = {"path": path, "pageTitle": "home"}
data = template("template.html", context)
if encode.endswith("utf-8"):
data = data.encode(encode)
status = '200 OK'
response_headers = [
('Content-type', content_type),
('Content-Length', str(len(data)))
]
start_response(status, response_headers)
return iter([data])

Python/Flask: Stop Gunicorn from restarting when receiving HUP signal?

I have a Python/Flask app running on Gunicorn. For some reason, somewhere between 2 to 2.5 hours of starting the app, this always happens:
[2021-05-19 03:17:38 +0000] [26530] [DEBUG] Closing connection.
[2021-05-19 03:18:04 +0000] [26526] [WARNING] Worker with pid 26529 was terminated due to signal 1
[2021-05-19 03:18:04 +0000] [26526] [WARNING] Worker with pid 26530 was terminated due to signal 1
[2021-05-19 03:18:04 +0000] [26526] [WARNING] Worker with pid 26527 was terminated due to signal 1
[2021-05-19 03:18:04 +0000] [26526] [WARNING] Worker with pid 26528 was terminated due to signal 1
[2021-05-19 03:18:04 +0000] [27837] [INFO] Booting worker with pid: 27837
[2021-05-19 03:18:04 +0000] [26526] [DEBUG] 1 workers
[2021-05-19 03:18:04 +0000] [26526] [INFO] Handling signal: hup
[2021-05-19 03:18:04 +0000] [26526] [INFO] Hang up: Master
[2021-05-19 03:18:04 +0000] [26526] [DEBUG] Current configuration:
config: ./gunicorn.conf.py
wsgi_app: None
bind: ['0.0.0.0:8081']
backlog: 2048
workers: 4
worker_class: sync
threads: 1
worker_connections: 1000
max_requests: 0
max_requests_jitter: 0
timeout: 9223372036
graceful_timeout: 30
keepalive: 2
limit_request_line: 4094
limit_request_fields: 100
limit_request_field_size: 8190
reload: False
reload_engine: auto
reload_extra_files: []
spew: False
check_config: False
print_config: False
preload_app: False
sendfile: None
reuse_port: False
chdir: /home/ubuntu
daemon: False
raw_env: []
pidfile: None
worker_tmp_dir: None
user: 0
group: 0
umask: 0
initgroups: False
tmp_upload_dir: None
secure_scheme_headers: {'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'}
forwarded_allow_ips: ['127.0.0.1']
accesslog: None
disable_redirect_access_to_syslog: False
access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
errorlog: -
loglevel: debug
capture_output: False
logger_class: gunicorn.glogging.Logger
logconfig: None
logconfig_dict: {}
syslog_addr: udp://localhost:514
syslog: False
syslog_prefix: None
syslog_facility: user
enable_stdio_inheritance: False
statsd_host: None
dogstatsd_tags:
statsd_prefix:
proc_name: None
default_proc_name: __init__:create_app()
pythonpath: None
paste: None
on_starting: <function OnStarting.on_starting at 0x7f3f9ecc3ca0>
on_reload: <function OnReload.on_reload at 0x7f3f9ecc3dc0>
when_ready: <function WhenReady.when_ready at 0x7f3f9ecc3ee0>
pre_fork: <function Prefork.pre_fork at 0x7f3f9ec5a040>
post_fork: <function Postfork.post_fork at 0x7f3f9ec5a160>
post_worker_init: <function PostWorkerInit.post_worker_init at 0x7f3f9ec5a280>
worker_int: <function WorkerInt.worker_int at 0x7f3f9ec5a3a0>
worker_abort: <function WorkerAbort.worker_abort at 0x7f3f9ec5a4c0>
pre_exec: <function PreExec.pre_exec at 0x7f3f9ec5a5e0>
pre_request: <function PreRequest.pre_request at 0x7f3f9ec5a700>
post_request: <function PostRequest.post_request at 0x7f3f9ec5a790>
child_exit: <function ChildExit.child_exit at 0x7f3f9ec5a8b0>
worker_exit: <function WorkerExit.worker_exit at 0x7f3f9ec5a9d0>
nworkers_changed: <function NumWorkersChanged.nworkers_changed at 0x7f3f9ec5aaf0>
on_exit: <function OnExit.on_exit at 0x7f3f9ec5ac10>
proxy_protocol: False
proxy_allow_ips: ['127.0.0.1']
keyfile: /etc/letsencrypt/live/site.io/privkey.pem
certfile: /etc/letsencrypt/live/site.io/fullchain.pem
ssl_version: 2
cert_reqs: 0
ca_certs: None
suppress_ragged_eofs: True
do_handshake_on_connect: False
ciphers: None
raw_paste_global_conf: []
strip_header_spaces: False
[2021-05-19 03:18:04 +0000] [26526] [DEBUG] 4 workers
[2021-05-19 03:18:04 +0000] [27838] [INFO] Booting worker with pid: 27838
[2021-05-19 03:18:04 +0000] [27839] [INFO] Booting worker with pid: 27839
[2021-05-19 03:18:04 +0000] [27841] [INFO] Booting worker with pid: 27841
[2021-05-19 03:18:04 +0000] [27840] [INFO] Booting worker with pid: 27840
[2021-05-19 03:18:08 +0000] [27837] [INFO] Worker exiting (pid: 27837)
[2021-05-19 03:18:08 +0000] [26526] [WARNING] Worker with pid 27837 was terminated due to signal 15
I have no idea how or why Gunicorn is apparently receiving a HUP signal (https://docs.gunicorn.org/en/19.3/signals.html#reload-the-configuration) causing it to reload the app. Often nothing else is happening in the app - no users using it or anything else in the logs. This is a problem since if there is someone using the app, restarting it in the middle of some code running (there's a function that can take minutes to finish) will break it.
I'm running an AWS EC2 t2.medium instance, python3.9 and Gunicorn 20.1.0, launching the app as follows:
/usr/bin/python3.9 -m gunicorn.app.wsgiapp -w 4 -b 0.0.0.0:8081 -t 9223372036 '__init__:create_app()' --certfile=/etc/letsencrypt/live/site.io/fullchain.pem --keyfile=/etc/letsencrypt/live/site.io/privkey.pem --log-level 'debug'
How can I stop Gunicorn restarting like this?

I get a [DEBUG] worker: SIGWINCH ignored error when publishing my flask application with Guncorn3

I made a little app with flask. I run the app on my ubuntu system as python3 app.py and everything works fine. I wanted to publish the app with gunicorn, but some functions also get errors. I looked at the error logs and [DEBUG] worker: SIGWINCH ignored. I get this error. The reason for the error is I put python pexpect module
def botbaslatma():
child = pexpect.spawn('screen -S idle python3 bot_dosyalari/{}'.format(session["username"]+".py"))
i=child.expect(['Enter 2FA code: ',"Invalid password for '{}'. Enter password: ".format(session["username"]),pexpect.EOF])
if i==0:
child.sendline(Guard)
k=child.expect(['Logged on as:','Incorrect code. Enter 2FA code: ',pexpect.EOF])
if k==0:
cursor=Mysql.connection.cursor()
sorgu='UPDATE sxxxxeri SET status="Aktif" WHERE stxxxxci="{}" '.format(session["username"])
cursor.execute(sorgu)
Mysql.connection.commit()
cursor.close()
child.close()
flash("Done","success")
return redirect(url_for("xontrolxxxx",_external=True))
elif k==1:
cursor=Mysql.connection.cursor()
sorgu='UPDATE sxxxleri SET status="Bexxxyor" WHERE steaxxxici="{}" '.format(session["username"])
cursor.execute(sorgu)
Mysql.connection.commit()
cursor.close()
child.close()
flash("Wrong passwd","danger")
return redirect(url_for("xontrolxxxx",_external=True))
elif i==2:#timeout
flash("Timeout","danger")
child.close()
durdurma()
return redirect(url_for("xontrolxxxx",_external=True))
elif i==1:
child.close()
durdurma()
flash("Wrong passwd","danger")
return redirect(url_for("xontrolxxxx",_external=True))
elif i==2:
child.close()
flash("Timeout","danger")
return redirect(url_for("xontrolxxxx",_external=True))
return render_template("botxxxx.html",form=form)
#Always enters pexpect.EOF control i==2
######Gunicorn3 file
[Unit]
Description=Gunicorn service
After=network.target
[Service]
User=root
Group=www-data
WorkingDirectory=/root/flaskapp
ExecStart=/usr/bin/gunicorn3 --workers 3 --bind unix:flaskapp.sock -m 007 app:app --timeout 300 --config /root/flaskapp/gunicorn.conf.py
#Gunicorn3 Error log *************
[2020-07-03 15:38:55 +0200] [14261] [DEBUG] Current configuration:
config: /root/flaskapp/gunicorn.conf.py
bind: ['unix:flaskapp.sock']
backlog: 2048
workers: 3
worker_class: sync
threads: 1
worker_connections: 1000
max_requests: 0
max_requests_jitter: 0
timeout: 300
graceful_timeout: 30
keepalive: 2
limit_request_line: 4094
limit_request_fields: 100
limit_request_field_size: 8190
reload: False
reload_engine: auto
spew: False
check_config: False
preload_app: False
sendfile: None
chdir: /root/flaskapp
daemon: False
raw_env: []
pidfile: None
worker_tmp_dir: None
user: 0
group: 33
umask: 7
initgroups: False
tmp_upload_dir: None
secure_scheme_headers: {'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'}
forwarded_allow_ips: ['127.0.0.1']
accesslog: /root/flaskapp/flaskapp.gunicorn.access
access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
errorlog: /root/flaskapp/flaskapp.gunicorn.error
loglevel: debug
capture_output: False
logger_class: gunicorn.glogging.Logger
logconfig: None
syslog_addr: udp://localhost:xxx
syslog: False
syslog_prefix: None
syslog_facility: user
enable_stdio_inheritance: False
statsd_host: None
statsd_prefix:
proc_name: None
default_proc_name: app:app
pythonpath: None
paste: None
on_starting: <function OnStarting.on_starting at 0x7f66cff9c048>
on_reload: <function OnReload.on_reload at 0x7f66cff9c1e0>
when_ready: <function WhenReady.when_ready at 0x7f66cff9c378>
pre_fork: <function Prefork.pre_fork at 0x7f66cff9c510>
post_fork: <function Postfork.post_fork at 0x7f66cff9c6a8>
post_worker_init: <function PostWorkerInit.post_worker_init at 0x7f66cff9c840>
worker_int: <function WorkerInt.worker_int at 0x7f66cff9c9d8>
worker_abort: <function WorkerAbort.worker_abort at 0x7f66cff9cb70>
pre_exec: <function PreExec.pre_exec at 0x7f66cff9cd08>
pre_request: <function PreRequest.pre_request at 0x7f66cff9cea0>
post_request: <function PostRequest.post_request at 0x7f66cff9f048>
child_exit: <function ChildExit.child_exit at 0x7f66cff9f1e0>
worker_exit: <function WorkerExit.worker_exit at 0x7f66cff9f378>
nworkers_changed: <function NumWorkersChanged.nworkers_changed at 0x7f66cff9f510>
on_exit: <function OnExit.on_exit at 0x7f66cff9f6a8>
proxy_protocol: False
proxy_allow_ips: ['127.0.0.1']
keyfile: None
certfile: None
ssl_version: 2
cert_reqs: 0
ca_certs: None
suppress_ragged_eofs: True
do_handshake_on_connect: False
ciphers: TLSv1
raw_paste_global_conf: []
[2020-07-03 15:38:55 +0200] [14261] [INFO] Starting gunicorn 19.7.1
[2020-07-03 15:38:55 +0200] [14261] [DEBUG] Arbiter booted
[2020-07-03 15:38:55 +0200] [14261] [INFO] Listening at: unix:flaskapp.sock (14261)
[2020-07-03 15:38:55 +0200] [14261] [INFO] Using worker: sync
[2020-07-03 15:38:55 +0200] [14282] [INFO] Booting worker with pid: 14282
[2020-07-03 15:38:55 +0200] [14285] [INFO] Booting worker with pid: 14285
[2020-07-03 15:38:55 +0200] [14286] [INFO] Booting worker with pid: 14286
[2020-07-03 15:38:55 +0200] [14261] [DEBUG] 3 workers
[2020-07-03 15:39:03 +0200] [14285] [DEBUG] GET /kontrolpaneli
[2020-07-03 15:39:03 +0200] [14285] [DEBUG] GET /login
[2020-07-03 15:39:13 +0200] [14282] [DEBUG] POST /login
[2020-07-03 15:39:13 +0200] [14285] [DEBUG] GET /
[2020-07-03 15:39:17 +0200] [14286] [DEBUG] GET /kontrolpaneli
[2020-07-03 15:39:19 +0200] [14285] [DEBUG] GET /botbaslatma
[2020-07-03 15:39:43 +0200] [14285] [DEBUG] POST /botbaslatma
[2020-07-03 15:39:43 +0200] [14412] [DEBUG] worker: SIGWINCH ignored. **here it is**
[2020-07-03 15:39:43 +0200] [14285] [DEBUG] GET /kontrolpaneli

This message is harmless. The signal SIGWINCH marks that the terminal window was resized. There is no real terminal under which your app runs, so that seems normal. It is probably produced by screen.
Your code is very weird. Rather than creating a terminal through screen and driving a python command line application through pexpect, the normal approach would be to call bot_dosyalari functions directly. However, it should work.
Additionally, you shouldn't be runnning a web application as root. I guess you did that because www-data didn't have access to /root/flaskapp, but the solution is to set the permissions right (probably moving it to e.g. /var/www/flaskapp at the same time), not to change the webapp user.

aiohttp router - not writing to logs

I've got an aiohttp web app that uses routes as flask like decorators and gunicorn
I'm having some trouble getting the logs to work correctly though.
What am I missing here?
No error's are being thrown or logged and the app run smoothly, but nothing is being logged apart from the start-up logs:
[2018-10-16 09:41:18 +0000] [1] [INFO] Starting gunicorn 19.9.0
[2018-10-16 09:41:18 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
[2018-10-16 09:41:18 +0000] [1] [INFO] Using worker: aiohttp.worker.GunicornWebWorker
[2018-10-16 09:41:18 +0000] [16] [INFO] Booting worker with pid: 16
[2018-10-16 09:41:18 +0000] [17] [INFO] Booting worker with pid: 17
[2018-10-16 09:41:18 +0000] [18] [INFO] Booting worker with pid: 18
[2018-10-16 09:41:18 +0000] [19] [INFO] Booting worker with pid: 19
[2018-10-16 09:41:18 +0000] [20] [INFO] Booting worker with pid: 20
[2018-10-16 09:41:18 +0000] [21] [INFO] Booting worker with pid: 21
[2018-10-16 09:41:18 +0000] [22] [INFO] Booting worker with pid: 22
[2018-10-16 09:41:18 +0000] [23] [INFO] Booting worker with pid: 23
My app/init.py file is as such:
import logging
import os
from logging import handlers
from aiohttp.web import Application
from app.routes import routes
from utils.logging import CloggerFormatter
def create_app(app_config):
app = Application()
logger = logging.getLogger('aiohttp.web')
log_level = logging.DEBUG
if os.environ.get('LOG_LEVEL'):
log_level = os.environ['LOG_LEVEL']
app.router.add_routes(routes)
logger.setLevel(log_level)
logger.addHandler(CloggerFormatter)
app['config'] = app_config
return app
And then in my app/routes.py file I access the logger with request.app.logger from within the rout definition like:
from aiohttp.web import Response, RouteTableDef
routes = RouteTableDef()
#routes.post('/background-checks')
async def api_background_check(request):
request_identifier = request.headers.get('X-Request-ID')
if not request_identifier:
request_identifier = uuid.uuid4()
request.app.logger.info('Checking background for request: %s', request_identifier)
This is my utils/handlers/logging.py file:
from time import strftime, gmtime
from logging import Formatter
class CloggerFormatter(Formatter):
"""
Logging module formatter in accordance with the yoti clogger manual
guidelines.
"""
converter = gmtime
def __init__(self, datefmt=None):
fmt = ('level:%(levelname)s'
'\ttime:%(asctime)s'
'\tmessage:%(message)s')
Formatter.__init__(self, fmt=fmt, datefmt=datefmt)
def formatTime(self, record, datefmt=None):
"""
Return the creation time of the LogRecord using the RFC 3339
format if datefmt is not specified.
"""
ct = self.converter(record.created)
if datefmt:
s = strftime(datefmt, ct)
else:
t = strftime('%Y-%m-%dT%H:%M:%S', ct)
s = '%s.%03dZ' % (t, record.msecs)
return s

Please use the root aiohttp logger: logger = logging.getLogger('aiohttp').
Particularly access logger uses 'aiohttp.access' name but maybe you want to see other log messages like errors and warnings as well.

Gunicorn not picking stderr

I have a Flask app with the following route:
#app.route('/')
def index():
console = logging.StreamHandler()
log = logging.getLogger("asdasd")
log.addHandler(console)
log.setLevel(logging.DEBUG)
log.error("Something")
print >> sys.stderr, "Another thing"
return 'ok'
I run this using
python gunicorn --access-logfile /mnt/log/test.log --error-logfile /mnt/log/test.log --bind 0.0.0.0:8080 --workers 2 --worker-class gevent --log-level debug server:app
The Logs are as below:
2014-06-26 00:13:55 [21621] [INFO] Using worker: gevent
2014-06-26 00:13:55 [21626] [INFO] Booting worker with pid: 21626
2014-06-26 00:13:55 [21627] [INFO] Booting worker with pid: 21627
2014-06-26 00:14:05 [21626] [DEBUG] GET /
10.224.67.41 - - [26/Jun/2014:00:14:14 +0000] "GET / HTTP/1.1" 200 525 "-" "python-requests/2.2.1 CPython/2.7.5 Darwin/13.2.0"
2014-06-26 00:14:14 [21626] [DEBUG] Closing connection.
What's happening to my logs in index method?

As of Gunicorn 19.0, gunicorn has stopped redirecting stderr to its logs.
Refer to https://github.com/benoitc/gunicorn/commit/41523188bc05fcbba840ba2e18ff67cd9df638e9