Develop Reference

Flask Session Cookies Expire Almost instantly, Can't Set Samesite Attribte

I a making a web application with a session cookie log in system. When using the cookies they expire within seconds, logging the user out of any service they were in. When I open my app I occasionaly get a warning in the terminal that states UserWarning: The session cookie domain is an IP address. This may not work as intended in some browsers. Add an entry to your hosts file, for example "localhost.localdomain", and use that instead. I'm hosting this app on Heroku so I don't think editing my local file would help, but if theres a way to get this to be solved on Heroku that would be great. Another error message I get comes from the console in the website itself, which reads:
Cookie “session” will be soon rejected because it has the “SameSite” attribute set to “None”
or an invalid value, without the “secure” attribute. To know more about the “SameSite“
attribute, read https://developer.mozilla.org/docs/Web/HTTP/Headers/Set-Cookie/SameSite
I set the Session cookie in my web application to:
app.config["SESSION_FILE_DIR"] = tempfile.mkdtemp()
app.config["SESSION_PERMANENT"] = False
app.config["SESSION_TYPE"] = "filesystem"
app.config["SESSION_COOKIE_SECURE"] = True
app.config["SESSION_COOKIE_SAMESITE"] = "None"
Session(app)
But this didn't solve my problem and both errors keep coming up. If there's any way to manually set SameSite and Secure that would be fantastic. Getting a https connection on Heroku did not work, I don't know why this is happening and it breaks the site, if there's any advice anyone has that would be greatly appreciated!

You need to use a domain name to access the service (https://domain.xxx/) and not the IP-address (https://123.123.123.213).
To avoid a lot of pain and errors, you should aim to use HTTPS, especially if you want cookies to work properly. Both the Secure and SameSite attributes requires HTTPS to work properly in most cases. And to get HTTPS to work you need a domain name and a proper certificate.

Python Django session returns "None"

tl;dr: Python newbie, Django's session not propagated correctly while using HTTPS
I'm building a basic web service which rely on session/cookies to authentication an user.
During the first authentication, I configure a specific session like this:
request.session['userSecureId'] = "blabla"
return HttpResponseRedirect('http://localhost/secure',context)
At this point, a new session key has been added to django_session table. A basic b64 decode on the session_data field confirm the presence of 'userSecureId'
On my view, I check if this session exist like this:
if request.session.get('userSecureId'):
# do something
If I try this on my local system (plain HTTP), it works great. So my next step was to run it on my remote server with SSL enabled. I've configured
SESSION_COOKIE_SECURE = True on my settings.py but now, the value returned by 'userSecureId' is always None.
This is probably a newbie question, so any pointer will be appreciated =)
Additionally, If I print request.session.session_key I'm able to successfully retrieve the session key, meaning Django correctly detect my sessionid cookie, but can't decode the content of session_value
EDIT: I just tried accessing Django on my remote system (same configuration) and I'm facing the same issue. I have no idea why I can't run the session value. Code works using 127.0.0.1 w/o problem though

According to here and here
To share a session between HTTP and HTTPS (and cross domain also), you should set SESSION_COOKIE_DOMAIN in your settings.
SESSION_COOKIE_DOMAIN = '.example.com'

last_accessed time in beaker session always None, but _accessed_time is changing

I'm using beakers WSGI SessionMiddleware to manage a session between browser and application.
I am trying to differentiate between when a session is first accessed against any further requests.
Fom the docs it appears there are two useful values made available in the WSGI environment,
["beaker.session"].last_accessed and ["beaker.session"]["_accessed_time"]
However, on repeated requests ["beaker.session"].last_accessed is always returning None, while the timestamp value in ["beaker.session"]["_accessed_time"] can be seen to be increasing with each request.
Each request performs a ["beaker.session"].save() - I have tried various combinations of setting auto=True in the session, and using .save() / .persist(), but no joy : .last_accessed is always None.
I am not using the session to actually persist any data, only to manage the creation of and pass through the session.id. ( I am using a session type of 'cookie' )

It turns out this behaviour is down to multiprocessing via apache.
It was resolved by using an external store to manage tracking when the session ID is first seen, and maintaining my own 'last_accessed_time' etc.

Does urllib2.urlopen() cache stuff?

They didn't mention this in python documentation. And recently I'm testing a website simply refreshing the site using urllib2.urlopen() to extract certain content, I notice sometimes when I update the site urllib2.urlopen() seems not get the newly added content. So I wonder it does cache stuff somewhere, right?

So I wonder it does cache stuff somewhere, right?
It doesn't.
If you don't see new data, this could have many reasons. Most bigger web services use server-side caching for performance reasons, for example using caching proxies like Varnish and Squid or application-level caching.
If the problem is caused by server-side caching, usally there's no way to force the server to give you the latest data.
For caching proxies like squid, things are different. Usually, squid adds some additional headers to the HTTP response (response().info().headers).
If you see a header field called X-Cache or X-Cache-Lookup, this means that you aren't connected to the remote server directly, but through a transparent proxy.
If you have something like: X-Cache: HIT from proxy.domain.tld, this means that the response you got is cached. The opposite is X-Cache MISS from proxy.domain.tld, which means that the response is fresh.

Very old question, but I had a similar problem which this solution did not resolve.
In my case I had to spoof the User-Agent like this:
request = urllib2.Request(url)
request.add_header('User-Agent', 'Mozilla/5.0')
content = urllib2.build_opener().open(request)
Hope this helps anyone...

Your web server or an HTTP proxy may be caching content. You can try to disable caching by adding a Pragma: no-cache request header:
request = urllib2.Request(url)
request.add_header('Pragma', 'no-cache')
content = urllib2.build_opener().open(request)

If you make changes and test the behaviour from browser and from urllib, it is easy to make a stupid mistake.
In browser you are logged in, but in urllib.urlopen your app can redirect you always to the same login page, so if you just see the page size or the top of your common layout, you could think that your changes have no effect.

I find it hard to believe that urllib2 does not do caching, because in my case, upon restart of the program the data is refreshed. If the program is not restarted, the data appears to be cached forever. Also retrieving the same data from Firefox never returns stale data.

Web/Screen Scraping with Google App Engine - Code works in python interpreter but not GAE

I want to do some web scraping with GAE. (Infinite Campus Student Information Portal, fyi). This service requires you to login to get in the website.
I had some code that worked using mechanize in normal python. When I learned that I couldn't use mechanize in Google App Engine I ended up using urllib2 + ClientForm. I couldn't get it to login to the server, so after a few hours of fiddling with cookie handling I ran the exact same code in a normal python interpreter, and it worked. I found the log file and saw a ton of messages about stripping out the 'host' header in my request... I found the source file on Google Code and the host header was in an 'untrusted' list and removed from all requests by user code.
Apparently GAE strips out the host header, which is required by I.C. to determine which school system to log you in, which is why it appeared like I couldn't login.
How would I get around this problem? I can't specify anything else in my fake form submission to the target site. Why would this be a "security hole" in the first place?

App Engine does not strip out the Host header: it forces it to be an accurate value based on the URI you are requesting. Assuming that URI's absolute, the server isn't even allowed to consider the Host header anyway, per RFC2616:
If Request-URI is an absoluteURI, the host is part of the Request-URI.
Any Host header field value in the
request MUST be ignored.
...so I suspect you're misdiagnosing the cause of your problem. Try directing the request to a "dummy" server that you control (e.g. another very simple app engine app of yours) so you can look at all the headers and body of the request as it comes from your GAE app, vs, how it comes from your "normal python interpreter". What do you observe this way?