Develop Reference

Bot after deploing to server on debian doesn't work withou threaded (telebot, python)

i cache the problem, sense of this problem is bot, which doesn't work without arg threaded, anyone have a solution for this problem?
Server Linux Ubuntu 18.04 / centos 8 (both doesn't work) on first clear uwsgi on second uwsgi + nginx, and bot work with threaded, but without not.
I want achieve that my bot can work in threaded mode, i don't have errors and others, if i start my bot without parametr threaded (default it true)
bot = telebot.TeleBot('my_token')
The string which i demonstrate under (i mark it <----):
#app.route(WEBHOOK_URL_PATH, methods=['POST'])
def webhook():
if flask.request.headers.get('content-type') == 'application/json':
json_string = flask.request.get_data().decode('utf-8')
update = telebot.types.Update.de_json(json_string)
bot.process_new_updates([update,]) # <---- THIS STRING
bot.send_message(update.message.from_user.id, update.message.text)
return ''
else:
flask.abort(403)
Doesn't work, and from this all of my other code doesn't work, because it's method let start the query to handle of all my others handlers, but this not happening because threaded is True, if i set this parametr in false it's work, but it's work not so good and I want so help to solve this problem.
I finded description(github forum) why it's not work, but my host VDSINA and i same have one thread, maybe it's a problem?
My uwsgi configuration:
[uwsgi]
socket = 127.0.0.1:9090
#shared-socket = :9090
#https = =0,webhook_cert.pem,webhook_pkey.pem
wsgi-file = foobar.py
callable = app
master = true
processes = 4
threads = 2
enable-threads = true
single-interpreter = true
But how I change this and how many time, I don't get the solve.
All code of my bot:
from flask import Flask
import flask
from time import sleep
import logging
import telebot
app = Flask(__name__)
#app.route('/')
def index():
print("simple output")
return "<span style='color:red'>I am app 1</span>"
bot = telebot.TeleBot('my_tok')
logger = telebot.logger
telebot.logger.setLevel(logging.INFO)
API_TOKEN = bot.token
WEBHOOK_HOST = 'ip_my_serve'
# nginx # WEBHOOK_PORT = 8443 # 443, 80, 88 or 8443 (port need to be 'open')
WEBHOOK_LISTEN = '0.0.0.0' # In some VPS you may need to put here the IP addr
WEBHOOK_SSL_CERT = '/etc/ssl/certs/nginx-selfsigned.crt' # Path to the ssl certificate
WEBHOOK_SSL_PRIV = '/etc/ssl/private/nginx-selfsigned.key' # Path to the ssl private key
WEBHOOK_URL_BASE = "https://%s" % (WEBHOOK_HOST)
WEBHOOK_URL_PATH = "/%s/" % (API_TOKEN)
bot.remove_webhook()
sleep(1)
bot.set_webhook(url=WEBHOOK_URL_BASE + WEBHOOK_URL_PATH,
certificate=open(WEBHOOK_SSL_CERT, 'r'))
#app.route(WEBHOOK_URL_PATH, methods=['POST'])
def webhook():
if flask.request.headers.get('content-type') == 'application/json':
json_string = flask.request.get_data().decode('utf-8')
update = telebot.types.Update.de_json(json_string)
bot.process_new_updates([update,])
bot.send_message(update.message.from_user.id, update.message.text)
return ''
else:
flask.abort(403)
#bot.message_handler(func=lambda message: True, content_types=['text'])
def echo_message(message):
bot.reply_to(message, message.text)

A solve for this answer it's change uwsgi on cherrypy, and it's my solution.

Error creating volume Openstack Python SDK

I am a beginner in programming and I am creating an application to manage machines with OpenStack. However, when trying to create a postman machine, I get an error.
Could you help me?
My requirements.txt:
Click==7.0
Flask==1.0.3
itsdangerous==1.1.0
Jinja2==2.10.1
MarkupSafe==1.1.1
Werkzeug==0.15.4
openstacksdk
python code from Github Tutorial:
from flask import Flask, request
from openstack import connection
from pprint import pprint
import json
import os
app = Flask(__name__)
# Source the openstack project rc file before running this app, to create the
# environment variables needed
conn = connection.Connection(auth_url=os.environ['OS_AUTH_URL'],
project_name=os.environ['OS_PROJECT_NAME'],
username=os.environ['OS_USERNAME'],
password=os.environ['OS_PASSWORD'],
user_domain_id="default",
project_domain_id=os.environ['OS_PROJECT_DOMAIN_ID'])
#app.route("/")
def hello():
return "Hello from Flask!"
#app.route("/list")
def list_openstack_resources():
# Check if connection is established
print("conn: ", conn)
# Print list of servers, images, flavors, endpoints, projects, users
server_list = list(conn.compute.servers())
image_list = list(conn.compute.images())
flavor_list = list(conn.compute.flavors())
project_list = list(conn.identity.projects())
user_list = list(conn.identity.users())
pprint(server_list)
pprint(image_list)
pprint(flavor_list)
pprint(project_list)
pprint(user_list)
return "List printed to stdout"
#app.route("/create_server")
def create_server():
# Check if connection is established
print("conn: ", conn)
# Create the volume first
volume_size = request.args.get('volume_size')
print("Starting to create volume with size (in GiB): ", volume_size)
volume = conn.create_volume(size=volume_size,
image="cirros-0.4.0-x86_64-disk",
wait=True,
bootable=False, #Or True
)
print("Created volume: ", json.dumps(volume))
# Create the server using the server_name parameter in the GET request
server_name = request.args.get('server_name')
print("Starting to create the server with name: ", server_name)
server = conn.create_server(name=server_name,
flavor="m1.micro",
terminate_volume=True,
timeout=180,
boot_volume=volume.id,
key_name="mayank-public-key",
)
print("Created server: ", json.dumps(server))
return "Server create request sent!"
if __name__ == "__main__":
app.run(debug=True, host='0.0.0.0', port=8080)
When I try this:
GET http://localhost:8080/create_server?server_name=server1&volume_size=2
I receive this error:

Flask timeout when serve using gunicorn

I have an app that will convert audio file to text. Using flask and flask-socketio. It works perfectly when I run it using: "python run.py", but when I run it using: "gunicorn -k eventlet -b 0.0.0.0:5000 run:app" it will stop on the part where it calls the google speech to text api in audio.py file.
These are the current codes right now.
run.py:
from ats import socketio, app, db
if __name__ == '__main__':
db.create_all()
socketio.run(app, host='0.0.0.0', port=5001, debug=True)
init.py
import logging, json
from flask import Flask, jsonify, render_template, request
from flask_socketio import SocketIO, emit, send
from flask_cors import CORS
from flask_sqlalchemy import SQLAlchemy
from flask_marshmallow import Marshmall
app = Flask(__name__, instance_relative_config=True, static_folder="templates/static", template_folder="templates")
# Create db instance
db = SQLAlchemy(app)
ma = Marshmallow(app)
#app.route('/')
def index():
return render_template('index.html');
# import models
from ats import models
# set up CORS
CORS(app)
socketio = SocketIO(app, cors_allowed_origins='*', async_mode='eventlet')
# import blueprints
from ats.product.product import product_blueprint
# register blueprints
app.register_blueprint(product_blueprint, url_prefix='/api/product')
from ats import error_handlers
product.py
import os
import math
import eventlet
from os.path import join
from flask import Blueprint, request, jsonify, abort
from ats.utils import audio as AUDIO
product_blueprint = Blueprint('product', __name__)
#product_blueprint.route('/add', methods=['post'])
def addProduct():
try:
data = request.form
foldername = data['name']
scriptFile = request.files['script']
audioFile = request.files['audio']
# save the script and audio file to uploads folder
FILE.createFolder(foldername)
FILE.save(foldername, scriptFile)
FILE.save(foldername, audioFile)
# list the files in the uploads
audioFiles = FILE.getAudioFileList(foldername)
fileCount = len(audioFiles)
currentFile = 1
# ============ speech to text =============
for file in audioFiles:
recognizedText = AUDIO.convert(foldername, file)
# save to database
newAudio = {
'name': file,
'recognizedText': recognizedText,
'length': duration,
}
Audio.add(newAudio)
# emit event to update the client about the progress
percent = math.floor((currentFile / float(fileCount) ) * 100)
emit('upload_progress', {'data': percent}, room=data['sid'], namespace='/')
eventlet.sleep()
currentFile += 1
# Delete the files in uploads folder
FILE.delete(foldername)
return jsonify({'data': None, 'message': 'Product was added.', 'success': True}), 200
except Exception as e:
abort(500, str(e))
audio.py
import os
from ats import app
# Imports the Google Cloud client library
from google.cloud import speech
from google.cloud.speech import enums
from google.cloud.speech import types
# Instantiates a client
client = speech.SpeechClient()
def convert(foldername, filename):
try:
file = os.path.join(app.config['UPLOAD_FOLDER'], foldername, filename)
# Loads the audio into memory
with io.open(file, 'rb') as audio_file:
content = audio_file.read()
audio = types.RecognitionAudio(content=content)
config = types.RecognitionConfig(
encoding=enums.RecognitionConfig.AudioEncoding.LINEAR16,
sample_rate_hertz=48000,
language_code='ja-JP')
# Call speech in the audio file
response = client.recognize(config, audio) # The code will stop here, that results to worker timeout in gunicorn
return response
except Exception as e:
raise e
I've been searching solution for almost a week but I still couldn't find one. THank you for you're help guys.

When you run your application directly using python run.py there is no timeout applied the application takes whatever time it needs to process, however when you run your application using Gunicorn, the default timeout is 30 seconds which means that you will get a timeout error incase your application does not respond within 30 seconds. To avoid this you can increase the default timeout set by Gunicorn by adding --timeout <timeinterval-in-seconds>
The following command sets the timeout to 10 mins
gunicorn -k eventlet -b 0.0.0.0:5000 --timeout 600 run:app

It's working now, by running it using uwsgi instead of gunicorn. Here's the config, service and nginx
ats.ini
[uwsgi]
module = wsgi:app
master = true
processes = 1
socket = ats.sock
chmod-socket = 660
vacuum = true
die-on-term = true
/etc/systemd/system/ats.service
[Unit]
Description=uWSGI instance to serve ats
After=network.target
[Service]
User=ubuntu
Group=www-data
WorkingDirectory=/home/user/ats
Environment="PATH=/home/user/ats/env/bin"
ExecStart=/home/user/ats/env/bin/uwsgi --ini ats.ini --gevent 100
[Install]
WantedBy=multi-user.target
nginx
server {
listen 80;
server_name <ip_address or domain>;
access_log /var/log/nginx/access.log;
location / {
include uwsgi_params;
uwsgi_pass unix:/home/user/ats/ats.sock;
proxy_set_header Connection "Upgrade";
client_max_body_size 200M;
}
location /socket.io {
include uwsgi_params;
uwsgi_pass unix:/home/user/ats/ats.sock;
proxy_set_header Connection "Upgrade";
}
}
Thank you guys

Google cloud python had some conflict with gevent. I found out from this thread that in order for them to work, you need to add the following in the beginning of init.py:
from gevent import monkey
monkey.patch_all()
import grpc.experimental.gevent as grpc_gevent
grpc_gevent.init_gevent()

I met this problem too today, finally I found that the bug caused by proxy setting. at first, I set my proxy is "",
os.environ['http_proxy'] = ""
os.environ['https_proxy'] = ""
and I get the error about time out in request, after I comment the code and it works
# os.environ['http_proxy'] = ""
# os.environ['https_proxy'] = ""
I think it is not an error about gunicore timeout default setting, it is about system proxy setting.

python socketio communication with ssl related problem

I am using python socketio for communication and it works well for http. Have a problem when upgraded it to work with SSL.
I made a self-signed root certificate (CA), and issued server.cert and server.key. I told the computer to trust the CA. After that, I added the server.cert and server.key on the flask-socketio server side. And the code looks like this:
from flask import Flask, render_template
from flask_socketio import SocketIO
from flask_socketio import Namespace
app = Flask(__name__, template_folder="templates")
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app)
# Create a URL route in our application for "/"
#app.route('/')
def home():
"""
This function loads the homepage
"""
return render_template('index.html')
class MyCustomNamespace(Namespace):
def on_connect(self):
print("Client just connected")
def on_disconnect(self):
print("Client just left")
def on_messages(self, data):
print(f"\nReceived data from client: \n {data}\n")
return data
socketio.on_namespace(MyCustomNamespace('/channel_A'))
if __name__ == "__main__":
socketio.run(app, host="192.168.0.28", port=2000, certfile="server.crt", keyfile="server.key", server_side=True, debug=True)
#socketio.run(app, host="192.168.0.28", port=2000, debug=True)
The code for client connection is simply:
import socketio
sio = socketio.Client()
def message_received(data):
print(f"Message {data} received")
#sio.on('connect', namespace="/channel_A")
def on_connect():
print("Connect...")
#sio.on('disconnect', namespace="/channel_A")
def on_disconnect():
print(f"Disconnected from server")
if __name__ == '__main__':
sio.connect('https://192.168.0.28:2000', namespaces="/channel_A")
emit_times = 0
is_emit = True
data = '{"data": "foobar"}'
while is_emit:
sio.emit("messages", data, namespace="/channel_A", callback=message_received)
emit_times += 1
if emit_times > 1:
is_emit = False
sio.disconnect()
I am using python-socketio (https://python-socketio.readthedocs.io/en/latest/client.html#disconnecting-from-the-server) for client end.
When the server gets started, the website works well and the connection is secure. The command line looks like this:
* Restarting with stat
* Debugger is active!
* Debugger PIN: 142-782-563
(3484) wsgi starting up on https://192.168.0.28:2000
When SocketIO client tries to connect with the server, the connection is refused and on the server end, the error is finally thrown like this:
ssl.SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:2488)
I think I probably miss something. Any ideas? Thanks in advance!

It seems that this is a bug in a newer SocketIO version. You can try to downgrade to 2.x.x:
pip install python-socketio<3.0.0 --force-reinstall

Server not responding although the webhook to Telegram Bot API is set up

I run this script on Pythonanywhere. Webhook gets set up, but when I try using the webhook_handler it gives me "Bad Request The browser (or proxy) sent a request that this server could not understand". What am I missing?
import sys
import os
import time
sys.path.append(os.path.join(os.path.abspath('.'), 'path/to/virtualenv/'))
from flask import Flask, request
import telegram
# CONFIG
TOKEN = '<token>'
HOST = 'username.pythonanywhere.com' # Same FQDN used when generating SSL Cert
PORT = 8443
CERT = "ssl_certs/cert.pem"
CERT_KEY = "ssl_certs/key.pem"
bot = telegram.Bot(TOKEN)
app = Flask(__name__)
#app.route('/')
def hello():
return '<h1> BITCONNECT!!! </h1>'
#app.route('/' + TOKEN, methods=['POST'])
def webhook_handler():
update = telegram.Update.de_json(request.get_json(force=True), bot)
bot.sendMessage(chat_id=update.message.chat.id, text='Hello, there')
return '<h1> OK </h1>'
def setwebhook():
bot.setWebhook(url = "https://%s/%s" % (HOST, TOKEN), certificate = open(CERT, 'rb'))
if __name__ == '__main__':
context = (CERT, CERT_KEY)
setwebhook()
time.sleep(5)
app.run(host = '0.0.0.0', port = PORT, debug = True)
And here's the WSGI configuration file:
import sys
# add your project directory to the sys.path
project_home = u'/home/username/project'
if project_home not in sys.path:
sys.path = [project_home] + sys.path
# import flask app but need to call it "application" for WSGI to work
from main import app as application