I'm trying to fetch fresh content from a wordpress blog, and it uses varnish on server side. Is there a way to bypass varnish cache control so that I can get fresh content each time I request that site?
Thanks
The documentation claims that Varnish will not cache any requests with a Cookie header, so a quick work around might be to include a Cookie: … header. Alternatively, you can include an un-used GET parameter like ?cachebuster=1234, which should bypass caching.
Usually, Everybody install Varnish on port 80 and change the default web service (Apache, Nginx or etc...) to listen on 8080.
So, Add the port 8080 to bypass Varnish:
Example: www.example.com:8080
Related
I am using HAProxy(open to alternatives) as a Rotating proxy server for my python crawler. I want a persistence session in HAProxy.
But not able to do so, because HAProxy cant change the headers of an HTTPS request.
Am using this
backend bk_web
balance round-robin
cookie SERVERID insert indirect nocache
my HAProxy server is not adding this cookie to the HTTPS request.
Thanks
Session persistence is usually when you want to persist your client to a specific backend server. However, you seem to indicate that you want to use it as a rotating proxy server, so I'm not quite sure you really want session persistence.
Are you terminating HTTPS in HAProxy? You need to terminate HTTPS in HAProxy (or any proxy for that matter) in order to modify headers within an HTTPS session. Alternatively, you can use HAProxy's stick-tables for persisting TCP connections where header modification is not possible or balance source
Using this
request.url_root+ attributes["footer"]["district_link"]
in jinja on a page that is https:// gives me a url that has http:// which is causing problems. Is there a way to use this request that preserves this so it will be http:// on a page that uses http and https on a page that uses https?
The backend is using flask if that is relevant here
Is your app proxied behind something like nginx?
If so you will need to use something like ProxyFix to add headers that include this information.
You can read more about proxy configuration in the flask docs on proxy setups.
Once you have ProxyFix working you can make use of the X-Forwarded-Proto header (for example). There's some more helpful info in this answer.
I am using http proxy in my python webbrowser. In my php script on server side I am still able to detect requests go through proxy. How can I mask it so other servers are not able to find out I am using http proxy?
Thank you.
Modify your proxy so it does not add the X-Forwarded-For header identifying the request as coming from a proxy.
If you don't control the proxy, you are SOL.
You could also, conceivably, use a SOCKS proxy instead of an HTTP one.
I am working on a game I want to support over iOS/Android/Browser and thinking Websockets is what I want to use for the communication. I use python and so found that I should be using Tornado.
I am trying to understand websockets a little better and their integration in browsers.
Will the messages over the websocket connection also contain the HTTP cookies for the connection? If not can I send it?
How is the HTTP connection for the web page linked to the websocket connection? I mean how will I know they are coming from the same webapp on the server side?
The Tornado wiki page says in the performance section that Tornado can be set up with nginx as the front end. How does that work? I thought Tornado and nginx have to be running on separate machines since both listen on port 80 and also because nginx does not understand WS protocol. What am I missing?
Also it will be great if someone can point me to any resources I can read up on about either Tornado or websocket that could help me.
The websocket is setup by sending an ordinary http request to the server, this request will contain all the stored cookies for the domain. If you do a native implementation for e.g. Android you can use libraries like Autobahn|Android, the API allows you to set cookies for the websocket handshake.
You can set a cookie when first loading the page to maintain a session identifier.
In that scenario they would be running 4 Tornado instances (on different ports, but not port 80) and Nginx on port 80 as a load-balancer, spreading the incoming client requests to the Tornado instances, see running Tornado and Nginx on same server for a configuration example. Recent versions of Nginx does support websockets, see e.g nginx + python + websockets.
I am doing a POST request to my Tastypie api, which creates a resource.
It normally returns the resource uri, through the Location header in the response.
The problem I'm having is the Location header contains a non-ssl url, even though my initial request (and the whole of my application) is under https.
From my request headers:
URL: https://example.com/api/v1/resource/
From my response headers:
Location: http://example.com/api/v1/resource/80/
Because this is a reusable application that is not always running under ssl, I do not want to hardcode an ugly string replace. Also, there is already a 301 redirect in place, from http to https, but I do not want the redirect to happen.
All help appreciated!
Update:
This actually didn't have anything to do with Tastypie, it was because of the servers/proxy configuration. See answer below for resolution details.
The reason is simple: seemingly request.is_secure() returns False in your case, so the URL is constructed using http instead of https.
There are couple of solutions, but you should first find what caused request.is_secure() to return False. I bet you are running behind some proxy or load balancer. If you did not change the logic behind URL generation, then this is probably the cause of your issue.
To fix that, you can take a look at SECURE_PROXY_SSL_HEADER setting in Django, which defines headers that indicate the SSL connection established with the proxy or load balancer:
If your Django app is behind a proxy, though, the proxy may be "swallowing" the fact that a request is HTTPS, using a non-HTTPS connection between the proxy and Django. In this case, is_secure() would always return False -- even for requests that were made via HTTPS by the end user.
In this situation, you'll want to configure your proxy to set a custom HTTP header that tells Django whether the request came in via HTTPS, and you'll want to set SECURE_PROXY_SSL_HEADER so that Django knows what header to look for.
But if you are designing a reusable app and the above is correct in your case, just make sure it is not something different. If you are sure this is the case, then leave that to the user - the headers responsible for secure request indication should be set explicitly, only by the programmer who uses your app. Otherwise this could mean a security issue.