Apscheduler running jobs concurrently in Flask - python

In Flask I'm attempting to run several jobs concurrently. But I'm faced with this issue:
This typically means that you attempted to use functionality that needed
to interface with the current application object in some way. To solve
this, set up an application context with app.app_context(). See the
documentation for more information.
Job "chron (trigger: interval[0:00:05], next run at: 2020-05-19 16:03:46 IST)" raised an exception
Traceback (most recent call last):
File "C:\Users\mithi\AppData\Local\Programs\Python\Python37\lib\site-packages\apscheduler\executors\base.py", line 125, in run_job
retval = job.func(*job.args, **job.kwargs)
File "c:\Users\mithi\Desktop\appengineering\server.py", line 128, in chron
return jsonify({})
File "C:\Users\mithi\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\json\__init__.py", line 358, in jsonify
if current_app.config["JSONIFY_PRETTYPRINT_REGULAR"] or current_app.debug:
File "C:\Users\mithi\AppData\Local\Programs\Python\Python37\lib\site-packages\werkzeug\local.py", line 348, in __getattr__
return getattr(self._get_current_object(), name)
File "C:\Users\mithi\AppData\Local\Programs\Python\Python37\lib\site-packages\werkzeug\local.py", line 307, in _get_current_object
return self.__local()
File "C:\Users\mithi\AppData\Local\Programs\Python\Python37\lib\site-packages\flask\globals.py", line 52, in _find_app
raise RuntimeError(_app_ctx_err_msg)
RuntimeError: Working outside of application context.
Here is my schedular object:
job_defaults = {
'coalesce': False,
'max_instances': 100
}
sched = BackgroundScheduler(daemon=True, job_defaults=job_defaults)
sched.start()
I'm creating multiple jobs by defining the following function:
def etl():
# read app config
with open('appConfig.json', encoding="utf-8") as json_file:
configData = json.load(json_file)
for obj in configData:
sched.add_job(chron, 'interval', seconds=5, args=[obj], id=obj)
Basically the function is adding jobs as a call to the function "chron" but args and id are different which creates unique jobs, running at intervals of five seconds. I get the app_context issue in the chron function. I have read about app_context but still unable to grasp the concept.

Your "chron" function is calling Flask current_app and jsonify() but it seems you haven't pushed a Flask context yet thus having this outside app context runtime error.
Before you can call a "contexted" func, you need to push your Flask app context.
Two ways of doing so,
app = Flask(__name__)
# do some stuff on your app if you need
app.app_context().push()
# do some other stuff
or in your chron func,
with app.app_context():
# do something
You can refer to manually pushing a context for more details

Related

RuntimeError: Working outside of application context. in my flask app

I'm trying to schedule sending email from my flask application with this function :
from apscheduler.scheduler import Scheduler
scheduler = Scheduler()
scheduler.start()
def email_job_scheduling():
to="abdellah.ala#gmail.com"
subject="summary projects"
message="your summary projects"
send_email(to,subject,message)
scheduler.add_cron_job(email_job_scheduling, day_of_week='tue', hour=12, minute=55)
this is how i declare app in my file init.py, is there any relationship or must i add schedule function in this file.
login_manager = LoginManager()
db = SQLAlchemy()
mail = Mail()
def create_app(config_name):
app = Flask(__name__, instance_relative_config=True)
app.config.from_object(app_config[config_name])
app.config.from_pyfile('config.py')
app.permanent_session_lifetime = timedelta(minutes=10)
db.init_app(app)
mail.init_app(app)
login_manager.init_app(app)
return app
but I receive this error,
Debug mode: off * Running on http://127.0.0.1:5000/ (Press CTRL+C to
quit) Job "email_job_scheduling (trigger: cron[day_of_week='wed',
hour='9', minute='57'], next run at: 2019-12-11 09:57:00)" raised an
exception Traceback (most recent call last): File
"/home/abdellah/Documents/venv/lib64/python3.6/site-packages/apscheduler/scheduler.py",
line 512, in _run_job
retval = job.func(*job.args, **job.kwargs) File "/home/abdellah/Documents/SUPPORT-STS/project/app/admin/views.py",
line 29, in email_job_scheduling
send_email(to,subject,message) File "/home/abdellah/Documents/SUPPORT-STS/project/app/emails.py",
line 11, in send_email
mail.send(msg) File "/home/abdellah/Documents/venv/lib64/python3.6/site-packages/flask_mail.py",
line 491, in send
with self.connect() as connection: File "/home/abdellah/Documents/venv/lib64/python3.6/site-packages/flask_mail.py",
line 508, in connect
return Connection(app.extensions['mail']) File "/home/abdellah/Documents/venv/lib64/python3.6/site-packages/werkzeug/local.py",
line 348, in getattr
return getattr(self._get_current_object(), name) File "/home/abdellah/Documents/venv/lib64/python3.6/site-packages/werkzeug/local.py",
line 307, in _get_current_object
return self.__local() File "/home/abdellah/Documents/venv/lib64/python3.6/site-packages/flask/globals.py",
line 52, in _find_app
raise RuntimeError(_app_ctx_err_msg) RuntimeError: Working outside of application context.
This typically means that you attempted to use functionality that
needed to interface with the current application object in some way.
To solve this, set up an application context with app.app_context().
See the documentation for more information.
Yes if you have not specified an application context you should create it.
This is because it is necessary to define all the necessary resources within the Flask application.
The documentation explains perfectly in which cases and how you should use a context application in Flask.
https://flask.palletsprojects.com/en/1.0.x/appcontext/
If you post more code I could be more helpful.
I will try to find a solution by the way:
from flask import g
def email_job_scheduling():
a = "abdellah.ala#gmail.com"
subject = "summary projects"
message = "your summary projects"
send_email (to, subject, message)
def get_scheduler():
if "scheduler" is not in g:
g.scheduler = Scheduler()
g.scheduler.start()
returns g.scheduler
with app.app_context(): #add the necessary resources
scheduler = get_scheduler()
#add another code if you need to set another resource
#This piece of code I think is in the main
scheduler.add_cron_job (email_job_scheduling, day_of_week = 'tue', now = 12, minute = 55)

scheduling lambda function in zappa to stop instances

Here I am I created a flask application and deploying using zappa .
While deployment I am facing no module found exception but same python code it working offline below is my stop.app application
import boto3
from flask import Flask, request,Response, jsonify
app = Flask(__name__)
logging.basicConfig()
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
region = 'ap-south-1'
myins = ['i-043ae2fbfc26d423f','i-0df3f5ead69c6428c','i-0bac8502574c0cf1d','i-02e866c4c922f1e27']
#app.route('/', methods=['GET'])
def lambda_handler(event=None, context=None):
logger.info('Lambda function invoked index()')
ec2 = boto3.resource('ec2')
ec2client = boto3.client('ec2',region_name=region)
ec2client.stop_instances(InstanceIds=myins)
return 'Instances are stopped!!'
# if __name__ == '__main__':
# app.run(debug=True)```
Below is the error
alling tail for stage dev..
[1568791437587] No module named stop: ImportError
Traceback (most recent call last):
File "/var/task/handler.py", line 602, in lambda_handler
return LambdaHandler.lambda_handler(event, context)
File "/var/task/handler.py", line 245, in lambda_handler
handler = cls()
File "/var/task/handler.py", line 139, in __init__
self.app_module = importlib.import_module(self.settings.APP_MODULE)
File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named stop
Worth checking the way the zip file is created, Also check the uploaded zip if its in the correct format.
The lambda_hanlder should be at the base of the root of the zip.
User pip install -r requirements.txt -t .
Also just a small improvement move the ec2 connection outside of lambda_handler.
To start and stop the instance boto3 docs start and stop instance

How to add a value to session using Celery in Flask?

I've created a task that adds a value to session so then i can access it through browser , my celery is running perfectly, here are my codes:
from flask import session, g
from ..extensions import celery
CELERYBEAT_SCHEDULE = {
'add-every-10-seconds': {
'task': 'tasks.ruble',
'schedule': timedelta(seconds=20)
}
}
#celery.task(name='tasks.ruble')
def ruble():
url = 'https://finance.google.com/finance/converter?a=1&from=KGS&to=RUB&meta=ei%3DmSr0WeHCCYvBsAH8n6OIBA'
urlHandler = urllib2.urlopen(url)
html = urlHandler.read()
bsoup = BeautifulSoup(html, 'lxml')
num = bsoup.find('span').text.split()[0]
g.get('ruble', float(num))
session['get_ruble'] = float(num)
but when the times come to execute the task, it ends up with this error:
[2017-10-30 19:19:29,372: ERROR/PoolWorker-4] Task tasks.ruble[ff7617ad-764b-455c-b541-96f3ba78a87b] raised unexpected: RuntimeError('Working outside of request context.\n\nThis typically means that you attempted to use functionality that needed\nan active HTTP request. Consult the documentation on testing for\ninformation about how to avoid this problem.',)
Traceback (most recent call last):
File "/home/xakep/Desktop/work/optomvse/venv/local/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
R = retval = fun(*args, **kwargs)
File "/home/xakep/Desktop/work/optomvse/celery_worker.py", line 21, in __call__
return TaskBase.__call__(self, *args, **kwargs)
File "/home/xakep/Desktop/work/optomvse/venv/local/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
return self.run(*args, **kwargs)
File "/home/xakep/Desktop/work/optomvse/develop/tasks.py", line 34, in ruble
session['get_ruble'] = float(num)
File "/home/xakep/Desktop/work/optomvse/venv/local/lib/python2.7/site-packages/werkzeug/local.py", line 350, in __setitem__
self._get_current_object()[key] = value
File "/home/xakep/Desktop/work/optomvse/venv/local/lib/python2.7/site-packages/werkzeug/local.py", line 306, in _get_current_object
return self.__local()
File "/home/xakep/Desktop/work/optomvse/venv/local/lib/python2.7/site-packages/flask/globals.py", line 37, in _lookup_req_object
raise RuntimeError(_request_ctx_err_msg)
RuntimeError: Working outside of request context.
This typically means that you attempted to use functionality that needed
an active HTTP request. Consult the documentation on testing for
information about how to avoid this problem.
Another thing i forgot to mention is that i also tried to use g instead of session , by just assigning the float(num) value to it so g.ruble = float(num), but inside my template if i typed {{g.ruble}} nothing get printed out .
If the value keeps changing store it somewhere (Redis/ any database) and when the request from the user comes just update the session data in the API call.
session['get_ruble'] = get_stored_value()
You can not update the session data asynchronously since you don't have the request context in the async task.
Just a suggestion:
If the value is so dynamic instead of storing it in the session, you can provide a separate API to fetch the latest data.

python flask, keeping session in scope using another class

So my basic question. I have a flask application. Inside of this application I have built a simple login system that uses flask sessions. I want to break my code
up into different classes to help separate my logic. But i am receiving the following error.
Traceback (most recent call last):
File "server.py", line 22, in <module>
securty = Security()
File "/Users/chrisburgin/Development/pigarage/security.py", line 6, in __init__
print(session.get('logged_in'))
NameError: global name 'session' is not defined
This error is receiving using the following.
My External Class
class Security:
def __init__(self):
print('Security Created')
print(session.get('logged_in'))
def loggedIn(self):
if session.get('logged_in') != True:
return False
And I am calling this from inside of my server file
securty = Security()
Just as summery. I understand that the 'Security' class doesn't have access to session because its not in scope but I could very much use assistance in understanding how to provide it access.
Please correct me on anything I have done wrong. Thanks!
editing using from flask import session at the top of my external class file
File "server.py", line 22, in <module>
securty = Security()
File "/Users/chrisburgin/Development/pigarage/security.py", line 5, in __init__
print(session.get('logged_in'))
File "/Library/Python/2.7/site-packages/werkzeug/local.py", line 338, in __getattr__
return getattr(self._get_current_object(), name)
File "/Library/Python/2.7/site-packages/werkzeug/local.py", line 297, in _get_current_object
return self.__local()
File "/Library/Python/2.7/site-packages/flask/globals.py", line 20, in _lookup_req_object
raise RuntimeError('working outside of request context')
RuntimeError: working outside of request context
The session variable from Flask is a context local. Context locals are a somewhat special type of global variable, in that they are only accessible at certain times.
Specifically for session, you can only use it when the application is responding to a request. Flask sets the value of this variable right before invoking a view function.
In your stack trace, it appears you are creating an instance of the Security class during initialization. The constructor for this class accesses session, but at this time, there is really no client, no request, and no session, so this gives you an error. It's okay to create an object of this class, as long as you don't access the session in the constructor. Then you must make sure that you invoke the loggedIn method only inside a view function or a function called from a view function, to ensure a request context exists.

RuntimeError when running a simple flask_restful app on Google App Engine

I have the following very simple app:
from lib.flask import Flask
from lib.flask_restful import Resource, Api
app = Flask(__name__)
api = Api(app)
class TestResource(Resource):
def get(self):
return {"a":"b","c":"d"}
api.add_resource(TestResource, "/")
When I run this, I get the following Exception:
Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 266, in Handle
result = handler(dict(self._environ), self._StartResponse)
File "/Users/me/dev/project/src/main.py", line 22, in do_in_request
app(*args, **kwargs)
File "/Users/me/dev/project/src/lib/flask/app.py", line 1836, in __call__
return self.wsgi_app(environ, start_response)
File "/Users/me/dev/project/src/lib/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/Users/me/dev/project/src/lib/flask_restful/__init__.py", line 249, in error_router
if self._has_fr_route():
File "/Users/me/dev/project/src/lib/flask_restful/__init__.py", line 230, in _has_fr_route
if self._should_use_fr_error_handler():
File "/Users/me/dev/project/src/lib/flask_restful/__init__.py", line 211, in _should_use_fr_error_handler
adapter = self.app.create_url_adapter(request)
File "/Users/me/dev/project/src/lib/flask/app.py", line 1601, in create_url_adapter
return self.url_map.bind_to_environ(request.environ,
File "/Users/me/dev/project/src/lib/werkzeug/local.py", line 338, in __getattr__
return getattr(self._get_current_object(), name)
File "/Users/me/dev/project/src/lib/werkzeug/local.py", line 297, in _get_current_object
return self.__local()
File "/Users/me/dev/project/src/lib/flask/globals.py", line 20, in _lookup_req_object
raise RuntimeError('working outside of request context')
RuntimeError: working outside of request context
So I tried to put my entire app into what I thought would be the request context. In the code above, both the Flask object and the Api object are being instantiated once, and can be called by the server multiple times. From the traceback, it looks like the instantiation should happen within a request context, so I wrapped it like this:
def do_in_request(*args, **kwargs):
from lib.flask import Flask
from lib.flask_restful import Resource, Api
app = Flask(__name__)
api = Api(app)
class TestResource(Resource):
def get(self):
return {"a":"b","c":"d"}
api.add_resource(TestResource, "/")
app(*args, **kwargs)
app = do_in_request
That still raises the same error. What's going on, what do they mean by 'request context', and how do I fix this?
I started from scratch using the App Engine Flask Skeleton and added flask-restful as a dependency in requirements.txt and then just added more or less the code you have above and it worked without an issue. I added the repository to my github here if you want to use it - you can see the changes I made to the skeleton in this commit.
I'm not sure why your example isn't working, but one thing I noticed that could be causing issues is that you're doing from lib.flask .... It seems your third-party packages live in lib and you haven't necessarily "lifted" lib into your sys.path. The skeleton includes a vendoring library that makes sure that this is handled properly. It might be a potential source of the problem.
Either way, I hope the forked skeleton can help you get up and running.

Categories

Resources