I inherited an NGINX server, which hosts an API, as part of a research project I'm involved in. I am responsible for the iOS portion, but I've been getting a lot of 500 Internal Server Error which is obviously a problem for the app.
Unfortunately, the grad students who built it are long gone and there are no READMEs to help get me acquainted.
I've never worked on an API before and am struggling with figuring out how this one works/how to debug it.
I see in the code some commented out web.debug messages. For example:
def imgpath2url(path):
"""Given a path for an image (e.g., from the 'leaf' table of the database ), returns a valid API url"""
els = path.split('/')
#web.debug('Getting imgurl for %s' % (path))
if path.startswith('findingspecies'):
return '/species/' + fixspeciesname(els[1]) + '/images/' + els[-1]
elif path.startswith('uploads'):
id = os.path.basename(path).rsplit('.',1)[0]
return '/%s/original.jpg' % (id)
return ''
Where are these debug messages printed out to? Are they printed out to a browser? Can I view these printed debu statements as I make calls to the api within the iOS app?
The errors in this case are usually written to the web server error logs which in case of NGINX can be found at /var/log/nginx/error.log check that file using any text editor i.e nano or use cat, tail.
Also the web debug functions seems to belong to a framework like Odoo or so (A non-native python function). Which can be isolated if you can share more code or name the specific framework the API is built on.
Related
My use case is I have a flaskapp running
The idea is it'll just display a static page (currently using PySimpleGUI) and upon receiving the information sent from the other computer (such as the name for e.g.) then it'll show the name as a QR code. (so perhaps a constant refresh on the GUI and changing upon receiving the data)
The code setup is this way currently
#app.route('/qrcode', methods=['GET'])
def displayQrCode():
args = request.args
name = args.get('name') #
try:
img = qrcode.make(name)
img.save('checkin-qrcode.png')
# Display QR Code
qrWindowThread = threading.Thread(target=qrCodeWindowShow)
qrWindowThread.start()
print(name)
return 'Success'
except RuntimeError:
return 'Error in GET'
However, being new to Flask i am not sure how to send a string from a separate computer running jupyter notebook to this Flask app. Online tutorials show like having a form field page but i wish to not have an user input.
So for instance if the jupyter notebook code sends a post request to the ipaddress:5000 it should ideally show (correct me here: ipaddress:5000/qrcode?name=JohnDoe)
so the flask app would retrieve this JohnDoe and display it as a QR code. I can hardcode or manually access this page but how would i have it 'listen' to this /qrcode and then Get the value when it's sent. Or am i misunderstanding it all and I need to do it via another way?
EDIT:
Alright third time lucky, I think I understand what you want now. So your use case is a first computer that is displaying a GUI and running a flask server with an route that can take in a string.
There's a second computer that can send a http request to that route with a name and when that happens you want the GUI on the first computer to refresh to display a QR code of the name.
I'd solve this by having the qrcode route write that name to persistent storage. Could be anything sqlite db, an environment variable, a string in a file.
The first computer running the PySimpleGUI interface poles this persistent storage for changes, and if it sees a change then it renders a new QR code for that name and displays it.
There are multiple ways available to do that:
REST API integration
Realtime data transmission using tornado or ??
Through a base origin for data transmission
Hope you get help with this.
Python Django /w Microsoft Graphs -
I'm following this Microsoft Tutorial for building Django apps with Microsoft Graph (using it on my existing Django webapp), and I am having an issue with authentication: https://learn.microsoft.com/en-us/graph/tutorials/python
I'm on the step 'Add Azure AD authentication' and, after implementing,
I hit the sign in button and enter credentials...and I keep getting value error "state missing from auth_code_flow".
The "callback" method is only making it to result=get_token_from_code(request) and then fails.
Here is the get_token_from_code method:
def get_token_from_code(request):
cache = load_cache(request)
auth_app = get_msal_app(cache)
# Get the flow saved in session
flow = request.session.pop('auth_flow', {})
result = auth_app.acquire_token_by_auth_code_flow(flow, request.GET)
save_cache(request, cache)
return result
What I'm trying to do is eventually access excel online from my webapp.
Any help is greatly appreciated!
I just had this issue and resolved it. It is one of these two things:
You are starting out at 127.0.0.1:8000 and then when you're redirected you're at localhost:8000, which is a different domain. The sessions aren't remembered from one domain to the other. The solution is to start out on localhost:8000 so that the session persists across login.
Your browser is using super-strict cookie settings. Microsoft Edge appears to default to this mode on localhost and 127.0.0.1. There is a lock or shield icon in or near your address bar that lets you relax the restrictions on your cookie settings.
Try one or both of these and you should succeed.
I'm a beginner coder, so i'm pretty sure im just circumventing around the error. But replacing the website URL with http://localhost:8000/# and re running it somehow got around the error. maybe that could be of some use.
If you are running on chrome, rather than running application on http://127.0.0.1:8000 run it on http://localhost:8000, because chrome isn't saving the cookies when the ip address is being used.
I have created a Django app with the usual components: applications, models views, templates, etc.
The architecture of a Django app is such that it basically just sits there and does nothing until you call one of the views by hitting a REST endpoint. Then it serves a page (or in my case some JSON) and waits for the next REST request.
I would like to insert into this app some automated tweeting. For this purpose I will be using the python-twitter library. My tweets will contain a URL. Twitter's website says that any URLs inserted into tweets get shortened to 23 characters by Twitter itself. So the remaining characters are available for the non-URL portion of the tweet. But the 23-character size may change. So Twitter recommends checking the current size of shortened URLs when the application is loaded but no more than once daily. This is how I can check the current shortened-URL size using python-twitter:
>>> import twitter
>>> twitter_keys = {
"CONSUMER_KEY": "BlahBlahBlah1",
"CONSUMER_SECRET": "BlahBlahBlah2",
"ACCESS_TOKEN_KEY": "BlahBlahBlah3",
"ACCESS_TOKEN_SECRET": "BlahBlahBlah4",
}
>>> api = twitter.Api(
consumer_key=twitter_keys['CONSUMER_KEY'],
consumer_secret=twitter_keys['CONSUMER_SECRET'],
access_token_key=twitter_keys['ACCESS_TOKEN_KEY'],
access_token_secret=twitter_keys['ACCESS_TOKEN_SECRET'],
)
>>> api.GetShortUrlLength()
23
Where and how should I save this value 23 such that it is retrieved from Twitter only once at the start of the application, but available to my Django models all the time during the execution of my application? Should I put it in the settings.py file? Or somewhere else? Please include a code sample if necessary to make it absolutely clear and explicit.
Lot's of different ways and it's primarily a matter of opinion. The simplest of course would be to keep that data in the source file for the module that connects to twitter. Which looks like your existing system. Which works fine as long as this is not an app that's being commited to a public VCS repository.
If the code get's into a public repository you have two choices. Use an 'app_settings' file or save it in the db. Both approaches are described here: https://stackoverflow.com/a/37266007/267540
The App Engine Dev Server documentation says the following:
The development server simulates the production App Engine service. One way in which it does this is to prepend a string (dev~) to the APPLICATION_IDenvironment variable. Google recommends always getting the application ID using get_application_id
In my application, I use different resources locally than I do on production. As such, I have the following for when I startup the App Engine instance:
import logging
from google.appengine.api.app_identity import app_identity
# ...
# other imports
# ...
DEV_IDENTIFIER = 'dev~'
application_id = app_identity.get_application_id()
is_development = DEV_IDENTIFIER in application_id
logging.info("The application ID is '%s'")
if is_development:
logging.warning("Using development configuration")
# ...
# set up application for development
# ...
# ...
Nevertheless, when I start my local dev server via the command line with dev_appserver.py app.yaml, I get the following output in my console:
INFO: The application ID is 'development-application'
WARNING: Using development configuration
Evidently, the dev~ identifier that the documentation claims will be preprended to my application ID is absent. I have also tried to use the App Engine Launcher UI to see if that changed anything, but it did not.
Note that 'development-application' is the name of my actual application, but I expected it to be 'dev~development-application'.
Google recommends always getting the application ID using get_application_id
But, that's if you cared about the application ID -- you don't: you care about the partition. Check out the source -- it's published at https://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/api/app_identity/app_identity.py .
get_app_identity uses os.getenv('APPLICATION_ID') then passes that to internal function _ParseFullAppId -- which splits it by _PARTITION_SEPARATOR = '~' (thus removing again the dev~ prefix that dev_appserver.py prepended to the environment variable). That's returned as the "partition" to get_app_identity (which ignores it, only returning the application ID in the strict sense).
Unfortunately, there is no architected way to get just the partition (which is in fact all you care about).
I would recommend that, to distinguish whether you're running locally or "in production" (i.e, on Google's servers at appspot.com), in order to access different resources in each case, you take inspiration from the way Google's own example does it -- specifically, check out the app.py example at https://cloud.google.com/appengine/docs/python/cloud-sql/#Python_Using_a_local_MySQL_instance_during_development .
In that example, the point is to access a Cloud SQL instance if you're running in production, but a local MySQL instance instead if you're running locally. But that's secondary -- let's focus instead on, how does Google's own example tell which is the case? The relevant code is...:
if (os.getenv('SERVER_SOFTWARE') and
os.getenv('SERVER_SOFTWARE').startswith('Google App Engine/')):
...snipped: what to do if you're in production!...
else:
...snipped: what to do if you're in the local server!...
So, this is the test I'd recommend you use.
Well, as a Python guru, I'm actually slightly embarassed that my colleagues are using this slightly-inferior Python code (with two calls to os.getenv) -- me, I'd code it as follows...:
in_prod = os.getenv('SERVER_SOFTWARE', '').startswith('Google App Engine/')
if in_prod:
...whatever you want to do if we're in production...
else:
...whatever you want to do if we're in the local server...
but, this is exactly the same semantics, just expressed in more elegant Python (exploiting the second optional argument to os.getenv to supply a default value).
I'll be trying to get this small Python improvement into that example and to also place it in the doc page you were using (there's no reason anybody just needing to find out if their app is being run in prod or locally should ever have looked at the docs about Cloud SQL use -- so, this is a documentation goof on our part, and, I apologize). But, while I'm working to get our docs improved, I hope this SO answer is enough to let you proceed confidently.
That documentation seems wrong, when I run the commands locally it just spits out the name from app.yaml.
That being said, we use
import os
os.getenv('SERVER_SOFTWARE', '').startswith('Dev')
to check if it is the dev appserver.
I have a little system built for learning purposes with Flask-SQLAlchemy and deployed on Heroku(python/postgresql:
http://monkey-me.herokuapp.com/
https://github.com/coolcatDev/monkey-me
My app works fine locally and I have unit tested its functionality and use cases through 14 successfully passed tests.
When I deploy everything seems to go perfectly. Its deployed, I define environment variable APP_SETTINGS>>>config.BaseConfig, I run the db_create.py script to initialize the db. I create some users:
username-userpassword:
Alex-passwordAlex
Jack-passwordJack
Sara-passwordSara
But one thing is missing...I go to the users page from the main navigation bar and I get a 5oo internal server error saying:
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
My app code on app.py has debug mode on:
if __name__ == '__main__':
app.run(debug=True)
But no further error is displayed.
Notice that if I access the system on heroku and I am logged out, if I go to Users I am as I should redirected to the login page so thats how far I have gone with the debug...
If I set the environment variable LAUNCHY_DEBUG on heroku to either true or false everything goes crazy and new problems appear...like I cant delete a user, the profile images wont load....
If I remove the LAUNCHY_DEBUG var from heroku the new problems(images wont load, can't remove user..) persist among the original 500 error of the users page.
Thanks in advance for any suggestion with the debugging
Use the following to get more feedback written in logs:
import logging
app.logger.addHandler(logging.StreamHandler(sys.stdout))
app.logger.setLevel(logging.ERROR)
That reveals the problem:
ProgrammingError: (ProgrammingError) column "user_bf.id" must appear in the GROUP BY clause or be used in an aggregate function
Modify query:
userList = (
users.query
.add_column(db.func.count(friendships.user_id).label("total"))
.add_column(user_bf.id.label("best_friend"))
.add_column(user_bf.userName.label("best_friend_name"))
.outerjoin(friendships, users.id == friendships.user_id)
.outerjoin(user_bf, users.best_friend)
.group_by(users.id, user_bf.id)
.order_by(test)
.paginate(page, 6, False)
)
Regarding the images disappearing:
Any file written on a filesystem hosted on heroku will be deleted on dynos end of lifecycle.