I am using http proxy in my python webbrowser. In my php script on server side I am still able to detect requests go through proxy. How can I mask it so other servers are not able to find out I am using http proxy?
Thank you.
Modify your proxy so it does not add the X-Forwarded-For header identifying the request as coming from a proxy.
If you don't control the proxy, you are SOL.
You could also, conceivably, use a SOCKS proxy instead of an HTTP one.
Related
I am using HAProxy(open to alternatives) as a Rotating proxy server for my python crawler. I want a persistence session in HAProxy.
But not able to do so, because HAProxy cant change the headers of an HTTPS request.
Am using this
backend bk_web
balance round-robin
cookie SERVERID insert indirect nocache
my HAProxy server is not adding this cookie to the HTTPS request.
Thanks
Session persistence is usually when you want to persist your client to a specific backend server. However, you seem to indicate that you want to use it as a rotating proxy server, so I'm not quite sure you really want session persistence.
Are you terminating HTTPS in HAProxy? You need to terminate HTTPS in HAProxy (or any proxy for that matter) in order to modify headers within an HTTPS session. Alternatively, you can use HAProxy's stick-tables for persisting TCP connections where header modification is not possible or balance source
I want to share my local WebSocket on the internet but ngrok only support HTTP but my ws.py address is ws://localhost:8000/
it is good working on localhost buy is not know how to use this on the internet?
You can use ngrok http 8000 to access it. It will work. Although, ws is altogether a different protocol than http but ngrok handles it internally.
I'm writing this application where the user can perform a web search to obtain some information from a particular website.
Everything works well except when I'm connected to the Internet via Proxy (it's a corporate proxy).
The thing is, it works sometimes.
By sometimes I mean that if it stops working, all I have to do is to use any web browser (Chrome, IE, etc.) to surf the internet and then python's requests start working as before.
The error I get is:
OSError('Tunnel connection failed: 407 Proxy Authentication Required',)
My guess is that some sort of credentials are validated and the proxy tunnel is up again.
I tried with the proxies handlers but it remains the same.
My doubts are:
How do I know if the proxy need authentication, and if so, how to do it without hardcoding the username and password since this application will be used by others?
Is there a way to use the Windows default proxy configuration so it will work like the browsers do?
What do you think that happens when I surf the internet and then the python requests start working again?
I tried with requests and urllib.request
Any help is appreciated.
Thank you!
Check if there is any proxy setting in chrome
I am working on a game I want to support over iOS/Android/Browser and thinking Websockets is what I want to use for the communication. I use python and so found that I should be using Tornado.
I am trying to understand websockets a little better and their integration in browsers.
Will the messages over the websocket connection also contain the HTTP cookies for the connection? If not can I send it?
How is the HTTP connection for the web page linked to the websocket connection? I mean how will I know they are coming from the same webapp on the server side?
The Tornado wiki page says in the performance section that Tornado can be set up with nginx as the front end. How does that work? I thought Tornado and nginx have to be running on separate machines since both listen on port 80 and also because nginx does not understand WS protocol. What am I missing?
Also it will be great if someone can point me to any resources I can read up on about either Tornado or websocket that could help me.
The websocket is setup by sending an ordinary http request to the server, this request will contain all the stored cookies for the domain. If you do a native implementation for e.g. Android you can use libraries like Autobahn|Android, the API allows you to set cookies for the websocket handshake.
You can set a cookie when first loading the page to maintain a session identifier.
In that scenario they would be running 4 Tornado instances (on different ports, but not port 80) and Nginx on port 80 as a load-balancer, spreading the incoming client requests to the Tornado instances, see running Tornado and Nginx on same server for a configuration example. Recent versions of Nginx does support websockets, see e.g nginx + python + websockets.
I am behind my college's ISA Proxy | Forefront Threat Management Gateway. The proxy uses NTLM Auth, so we are given credentials along with the proxy server ip and port. I have been trying a lot of Python modules like urllib,ullib2,urllib3, requests,requests-ntlm httlib and even cntlm and ntlm proxy. Nothing is working in my case. It's returning " 407 Proxy Authentication Required ( Forefront
TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. )" or any socket error. I even tried ntlmaps. It too didn't worked out. I know NTLM is already kind of depreciated. But still some of you guys may be working behind your corporate proxy with NTLM Auth. So Any work arounds? I want a pythonic code that works on both Windows and Linux that can communicate to internet through the intermediate upstream proxy.
check out this article http://www.codemiller.com/blog/2011/05/28/overcoming-auth-pop-ups/
Maybe it will help - maybe it wont, but worth trying some of the solutions - as one of them worked for me in working around the glorious ntlm auth.
have you tried cntlm? with http_proxy environment varible? if you wish i can post step by step guide to solve this problem permanently..