Develop Reference

Python Django /w Microsoft Graphs - I keep getting value error "state missing from auth_code_flow"

Python Django /w Microsoft Graphs -
I'm following this Microsoft Tutorial for building Django apps with Microsoft Graph (using it on my existing Django webapp), and I am having an issue with authentication: https://learn.microsoft.com/en-us/graph/tutorials/python
I'm on the step 'Add Azure AD authentication' and, after implementing,
I hit the sign in button and enter credentials...and I keep getting value error "state missing from auth_code_flow".
The "callback" method is only making it to result=get_token_from_code(request) and then fails.
Here is the get_token_from_code method:
def get_token_from_code(request):
cache = load_cache(request)
auth_app = get_msal_app(cache)
# Get the flow saved in session
flow = request.session.pop('auth_flow', {})
result = auth_app.acquire_token_by_auth_code_flow(flow, request.GET)
save_cache(request, cache)
return result
What I'm trying to do is eventually access excel online from my webapp.
Any help is greatly appreciated!

I just had this issue and resolved it. It is one of these two things:
You are starting out at 127.0.0.1:8000 and then when you're redirected you're at localhost:8000, which is a different domain. The sessions aren't remembered from one domain to the other. The solution is to start out on localhost:8000 so that the session persists across login.
Your browser is using super-strict cookie settings. Microsoft Edge appears to default to this mode on localhost and 127.0.0.1. There is a lock or shield icon in or near your address bar that lets you relax the restrictions on your cookie settings.
Try one or both of these and you should succeed.

I'm a beginner coder, so i'm pretty sure im just circumventing around the error. But replacing the website URL with http://localhost:8000/# and re running it somehow got around the error. maybe that could be of some use.

If you are running on chrome, rather than running application on http://127.0.0.1:8000 run it on http://localhost:8000, because chrome isn't saving the cookies when the ip address is being used.

Why is App Engine Returning the Wrong Application ID?

The App Engine Dev Server documentation says the following:
The development server simulates the production App Engine service. One way in which it does this is to prepend a string (dev~) to the APPLICATION_IDenvironment variable. Google recommends always getting the application ID using get_application_id
In my application, I use different resources locally than I do on production. As such, I have the following for when I startup the App Engine instance:
import logging
from google.appengine.api.app_identity import app_identity
# ...
# other imports
# ...
DEV_IDENTIFIER = 'dev~'
application_id = app_identity.get_application_id()
is_development = DEV_IDENTIFIER in application_id
logging.info("The application ID is '%s'")
if is_development:
logging.warning("Using development configuration")
# ...
# set up application for development
# ...
# ...
Nevertheless, when I start my local dev server via the command line with dev_appserver.py app.yaml, I get the following output in my console:
INFO: The application ID is 'development-application'
WARNING: Using development configuration
Evidently, the dev~ identifier that the documentation claims will be preprended to my application ID is absent. I have also tried to use the App Engine Launcher UI to see if that changed anything, but it did not.
Note that 'development-application' is the name of my actual application, but I expected it to be 'dev~development-application'.

Google recommends always getting the application ID using get_application_id
But, that's if you cared about the application ID -- you don't: you care about the partition. Check out the source -- it's published at https://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/api/app_identity/app_identity.py .
get_app_identity uses os.getenv('APPLICATION_ID') then passes that to internal function _ParseFullAppId -- which splits it by _PARTITION_SEPARATOR = '~' (thus removing again the dev~ prefix that dev_appserver.py prepended to the environment variable). That's returned as the "partition" to get_app_identity (which ignores it, only returning the application ID in the strict sense).
Unfortunately, there is no architected way to get just the partition (which is in fact all you care about).
I would recommend that, to distinguish whether you're running locally or "in production" (i.e, on Google's servers at appspot.com), in order to access different resources in each case, you take inspiration from the way Google's own example does it -- specifically, check out the app.py example at https://cloud.google.com/appengine/docs/python/cloud-sql/#Python_Using_a_local_MySQL_instance_during_development .
In that example, the point is to access a Cloud SQL instance if you're running in production, but a local MySQL instance instead if you're running locally. But that's secondary -- let's focus instead on, how does Google's own example tell which is the case? The relevant code is...:
if (os.getenv('SERVER_SOFTWARE') and
os.getenv('SERVER_SOFTWARE').startswith('Google App Engine/')):
...snipped: what to do if you're in production!...
else:
...snipped: what to do if you're in the local server!...
So, this is the test I'd recommend you use.
Well, as a Python guru, I'm actually slightly embarassed that my colleagues are using this slightly-inferior Python code (with two calls to os.getenv) -- me, I'd code it as follows...:
in_prod = os.getenv('SERVER_SOFTWARE', '').startswith('Google App Engine/')
if in_prod:
...whatever you want to do if we're in production...
else:
...whatever you want to do if we're in the local server...
but, this is exactly the same semantics, just expressed in more elegant Python (exploiting the second optional argument to os.getenv to supply a default value).
I'll be trying to get this small Python improvement into that example and to also place it in the doc page you were using (there's no reason anybody just needing to find out if their app is being run in prod or locally should ever have looked at the docs about Cloud SQL use -- so, this is a documentation goof on our part, and, I apologize). But, while I'm working to get our docs improved, I hope this SO answer is enough to let you proceed confidently.

That documentation seems wrong, when I run the commands locally it just spits out the name from app.yaml.
That being said, we use
import os
os.getenv('SERVER_SOFTWARE', '').startswith('Dev')
to check if it is the dev appserver.

NeedIndexError at Google App Engine forever

I deployed and ran my app at GAE a few hours ago. It still fails because it needs to order certain datastore item, and the index needed for that is still not generated by GAE. So at the point of .order() it throws me a NeedIndexError. How long is this gonna take?
I've been doing this same procedure for 10~ GAE apps in the past and it has never, to my memory, been this slow. (Ok, it has been slow...)
The page "Datastore Indexes" in the old console just says "You have not created indexes for this application.".
The new console says nothing. It just displays a "blue alert" as if I'm not waiting myself to death already. The message in the alert is:
Cloud Datastore queries are powered by indexes, scalable data
structures that are updated in real time as property values change.
Your project's datastore index configuration specifies the indexes it
needs to support its queries. Cloud Datastore builds new indexes as
needed when you deploy index configuration. You can inspect the ready
state of your app's indexes using this console.
(ie a joke)
What am I supposed to do?
Update:
here's the index.yaml:
indexes:
# AUTOGENERATED
- kind: Mjquizinfo
ancestor: yes
properties:
- name: version
direction: desc

FWIW, in some cases (multi-module apps for example) the plain appcfg.py update used to deploy the app code might not update the index.yaml file.
Try specifically updating the index using appcfg.py update_indexes - you should be able to see the index info in the Developer Console right away (it may still take a while for indexing to be performed and become effective).

SOLR mysolr pysolr Python 401 reply

If there is someone out there who has already worked with SOLR and a python library to index/query solr, would you be able to try and answer the following question.
I am using the mySolr python library but there are others out (like pysolr) there and I don't think the problem is related to the library itself.
I have a default multicore SOLR setup, so no authentication required normally. Don't need it to access the admin page at http://localhost:8080/solr/testcore/admin/ either
from mysolr import Solr
solr = Solr('http://localhost:8080/solr/testcore/')
response = solr.search(q='*:*')
print("response")
print(response)
This code used to work but now I get a 401 reply from SOLR ... just like that, no changes have been made to the python virtual env containing mysolr or the SOLR setup. Still...something must have changed somewhere but I'm out of clues.
What could be the causes of a SOLR 401 reponse?
Additional info: This script and mor advanced script do work on another PC, just not on the one I am working on. Also, adding "/select?q=:" behind the url in the browser does return the correct results. So the SOLR is setup correctly, it has probably something to do with my computer itself. Could windows settings (of any kind) have an impact on how SOLR responds to requests from python? The python env itself has been reinstalled several times to no avail.
Thanks in advance!

The problem was: proxy.
If this exact situation was ever to occur to someone and you are behind a proxy, check if your HTTP and HTTPS environmental variables are not set. If they are... this might cause the python session to try using the proxy while it shouldn't (connecting to localhost via proxy).
It didn't cause any trouble for months but out of the blue it did so whether you encounter this or not might be dependent on how your IT setup your proxy or made some other changes...somewhere.
thank you everyone!

Python App Engine debug/dev mode

I'm working on an App Engine project (Python) where we'd like to make certain changes to the app's behavior when debugging/developing (most often locally). For example, when debugging, we'd like to disable our rate-limiting decorators, turn on the debug param in the WSGIApplication, maybe add some asserts.
As far as I can tell, App Engine doesn't naturally have any concept of a global dev-mode or debug-mode, so I'm wondering how best to implement such a mode. The options I've been able to come up with so far:
Use google.appengine.api.app_identity.get_default_version_hostname() to get the hostname and check if it begins with localhost. This seems... unreliable, and doesn't allow for using the debug mode in a deployed app instance.
Use os.environ.get('APPLICATION_ID') to get the application id, which according to this page is automatically prepended with dev~ by the development server. Worryingly, the very source of this information is in a box warning:
Do not get the App ID from the environment variable. The development
server simulates the production App Engine service. One way in which
it does this is to prepend a string (dev~) to the APPLICATION_ID
environment variable, which is similar to the string prepended in
production for applications using the High Replication Datastore. You
can modify this behavior with the --default_partition flag, choosing a
value of "" to match the master-slave option in production. Google
recommends always getting the application ID using get_application_id,
as described above.
Not sure if this is an acceptable use of the environment variable. Either way it's probably equally hacky, and suffers the same problem of only working with a locally running instance.
Use a custom app-id for development (locally and deployed), use the -A flag in dev_appserver.py, and use google.appengine.api.app_identity.get_application_id() in the code. I don't like this for a number of reasons (namely having to have two separate app engine projects).
Use a dev app engine version for development and detect with os.environ.get('CURRENT_VERSION_ID').split('.')[0] in code. When deployed this is easy, but I'm not sure how to make dev_appserver.py use a custom version without modifying app.yaml. I suppose I could sed app.yaml to a temp file in /tmp/ with the version replaced and the relative paths resolved (or just create a persistent dev-app.yaml), then pass that into dev_appserver.py. But that seems also kinda dirty and prone to error/sync issues.
Am I missing any other approaches? Any considerations I failed to acknowledge? Any other advice?

In regards to "detecting" localhost development we use the following in our applications settings / config file.
IS_DEV_APPSERVER = 'development' in os.environ.get('SERVER_SOFTWARE', '').lower()
That used in conjunction with the debug flag should do the trick for you.