from os import system
system("ping www.twitter.com")
system("ping www.yahoo.com")
system("ping www.facebook.com")
I am in China, and Twitter and Facebook are banned here. I can open them in the browser using Clash for Windows software.
I have to download tweets from Twitter. So I need to ping the websites using Python to get tweets. I cannot ping the websites though.
How do I make my Python code use the Clash for Windows.
Output of the above code:
Pinging www.twitter.com [108.160.169.186] with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Ping statistics for 108.160.169.186:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
Pinging new-fp-shed.wg1.b.yahoo.com [180.222.102.201] with 32 bytes of data:
Reply from 180.222.102.201: bytes=32 time=258ms TTL=42
Reply from 180.222.102.201: bytes=32 time=229ms TTL=42
Reply from 180.222.102.201: bytes=32 time=230ms TTL=42
Request timed out.
Ping statistics for 180.222.102.201:
Packets: Sent = 4, Received = 3, Lost = 1 (25% loss),
Approximate round trip times in milli-seconds:
Minimum = 229ms, Maximum = 258ms, Average = 239ms
Pinging www.facebook.com [69.63.184.14] with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Ping statistics for 69.63.184.14:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
OS: Windows 10 (updated to latest edition). Using PyCharm as my IDE.
You said in comment that you are using Clash, however Clash is not a VPN:
Clash - A rule-based tunnel in Go.
Features:
Local HTTP/HTTPS/SOCKS server with authentication support
VMess, Shadowsocks, Trojan, Snell protocol support for remote connections
Built-in DNS server that aims to minimize DNS pollution attack impact, supports DoH/DoT upstream and fake IP.
Rules based off domains, GEOIP, IP CIDR or ports to forward packets to different nodes
Remote groups allow users to implement powerful rules. Supports automatic fallback, load balancing or auto select node based off
latency
Remote providers, allowing users to get node lists remotely instead of hardcoding in config
Netfilter TCP redirecting. Deploy Clash on your Internet gateway with iptables.
Comprehensive HTTP RESTful API controller
source: https://github.com/Dreamacro/clash
I'm not sure exactly how it works, my current understanding is that it allows you to use proxies.
Although it also has TUN mode as a "Premium Feature" (not sure what does it mean, I see no option to buy "Premium") which may work similarly to VPN, but I'm not sure:
Premium Features:
TUN mode on macOS, Linux and Windows. Doc
Match your tunnel by Script
Rule Provider
Clash for Windows which you try to use, is GUI for Clash.
Documentation is available, but it is only in Chinese:
https://github.com/Fndroid/clash-win-docs-new
screenshot:
I tried to use it, but I don't know how to configure it. It didn't work at all.
I see many possible solutions:
You can switch to proper VPN. Paid VPNs are more recommended than free ones. Cost is typically 3-10$/month depending on offer. Alternatively you can try to setup OpenVPN on your own VPS, it may be a bit cheaper, but not necessarily.
You can configure your script to use proxy. Libraries like requests support it: Proxies with Python 'Requests' module
You can try to read Clash for Windows documentation to see if it is possible what you try to achieve. Maybe it is enough to turn on System Proxy as visible on screenshot above?
You can try to configure Clash in TUN mode. In my opinion it may be more difficult than solutions 1 and 2. If you prefer this way, I suggest to read Clash documentation thoroughly: https://github.com/Dreamacro/clash/wiki/premium-core-features#tun-device
If you can afford it, I recommend solution 1 in most use-cases. I understand that you prefer free solution, however servers are not free, somebody needs to incur costs of running them (hardware, electricity etc.)
There is a variety of Python libraries that can help you. openpyn is one of them. Firstly, call this command in your terminal for setup:
sudo openpyn --init
After which all your Internet traffic can be redirected to a VPN server using the following command:
openpyn us
The command above is the default, to transfer all the traffic to the US. Other locations can be chosen, see the link above for more info.
As soon as your traffic is redirected, you are free to see banned sites as you did:
from os import system
system("ping www.twitter.com")
system("ping www.facebook.com")
When you have VPN running and active then it should redirect all traffic via VPN server. If it is not redirecting all traffic, then maybe it was configured to redirect only webrowser traffic. Also it is possible that you aren't using true VPN, but a proxy.
Please share which VPN are you using, it will help us to help you.
You can just start VPN manually and then try executing your Python code.
Alternatively you can control when your VPN is running from Python. It requires different library for each VPN provider. Please tell us which VPN do you use.
This is working example for NordVPN using nordvpn_switcher package:
import time
from nordvpn_switcher import initialize_VPN,rotate_VPN,terminate_VPN
initialize_VPN(save=1,area_input=['complete rotation'])
for i in range(1):
rotate_VPN()
from os import system
system("ping www.twitter.com")
system("ping www.yahoo.com")
system("ping www.facebook.com")
print('\nDo whatever you want here (e.g. pinging). Pausing for 10 seconds...\n')
time.sleep(10)
terminate_VPN()
Tor uses SOCKS 5 proxy server to give anonymity. In case you're using Tor, there are two ways to use SOCKS 5 proxy, one is to configure it at application level(browser, python code etc..), another to configure at Network Interface level. If you're using Tor, simply follow this answer, socks server is running at localhost:9050 by default - How to make python Requests work via socks proxy
Since you haven't done anything and it's already working, I guess you're using tunnel-based VPN. In this case, it should work automatically. In your case, ping could be blocked by the VPN provider.
Ping uses ICMP protocol while HTTP/HTTPS uses TCP protocol in the Transport layer (in OSI model). They are very different things, and one working doesn't guarantee other working. In many cases, ping is blocked, by the server or other middlewares that don't support ICMP protocols. Most cloud providers don't support ICMP protocols in their networking components, for ex. Azure doesn't, for their Load Balancers. So, instead of trying ping, you should try the real http request. Following is sample code
import requests
r = requests.get("www.google.com")
You can use this code:
from ping3 import ping
p=ping("example.com")
print(p)
Here is the solution:
go to https://www.wintun.net and download the latest release, copy the right wintun.dll into Clash home directory
restart Clash for windows
open clash dashboard, switch TUN Mode on
https://github.com/Dreamacro/clash/wiki/premium-core-features#windows
Here is an update for people facing the same problem.
There is an option in settings>System Proxy>Specify Protocol in Clash for Windows , I turned it on.
Before i was not able to run pip commands, after turning this on, i can do that. I hope it will be useful for someone. (This is not exact answer to the question but indeed it is useful for someone, i am sure)
Let's try to isolate this problem if it's Python or network-related.
Does it work if you run ping in the shell directly?
ping twitter.com
It's possible that your VPN has a setting that blocks Pings from the internet. It should be one of the configurations in the Firewall.
I have a bunch of Google alerts set up as rss feeds that update in real time. What I want is to be able to store the new data the rss feed is sending out in a database.
After looking around I found Google and Superfeedr both offer hubs that do most of the work for you; however they both require a callback url (obviously). I do have an Apache server running on the machine I'm working off, it already has python enabled so I can run python scripts on my server. However at the moment its only accessible from within my LAN.
What my real question is, what do I do next? I know that in php you would just have a call back file that handles requests but I'm lost as to what to do in python. Would I write a script and give the google/superfeedr services a url to that script? What would be in the script? Specific imports needed?
Also, I just read that if you use XMPP you don't need a callback url. How does that work?
For the local LAN problem, the most commonly used solution is to use tuneling solutions like Passageway. They will temporarily expose a local port of your machine to the "outer" web.
Now, as for implementation, it's fairly easy to set things up. Python is similar to PHP in the sense that you'll have to write a script that listen on networking connection and then handles the HTTP requests you're getting from Superfeedr or Google. (it looks like you're not familiar with Python, why not stick to PHP then?)
Finally XMPP is a feature that only us (Superfeedr) offer. It solves the problem of exposing local ports because it works behind the firewall.
I'm developing an application in Python for the Raspberry Pi that requires some level of configuration for use. This configuration can be done by hand, but given the eventual placement of the Pi's themselves, hooking up a monitor and keyboard will be unfeasible. Thus, some level of auto-configuration will be required.
The configuration pretty much just sets some properties in the program before it begins being used, so ideally this configuration information could be delivered via a web response (the Raspberry Pi will always be connected to a local intranet via the eth0 interface).
However, the issue is that the webserver which will deliver the configuration data to the Pi has a hostname which is unpredictable. The server may reside at 10.0.0.2, 10.0.0.3, or 192.168.1.2 for that matter, there's no way of knowing.
I would like to try to use the broadcast address to essentially send a request to every host on the subnet and wait for a response that sounded right, eg with a 200 status code and some data that made sense. Then, knowing the host of the server, it could request the initial configuration data and continue.
I'm currently using the urllib Python module for my web requests but I quickly discovered that using the broadcast address to achieve this goal would be confusing, since I'm essentially sending out a single request but preparing for multiple (mostly bad) responses.
Has anyone out there done anything like this using Python?
I'm trying to isolate a bug that exists either in Python's httplib2 HTTP client or an API. (First guess is the API.) While using httplib2 to POST data to a RESTful API, I'm getting a 401 response status (no authorization) and saving data to the API.
I'd like to examine the HTTP request and response to the client, the very strings put onto and received from the network. The httplib2 code seems too involved to easily capture the values from within it, and might possibly miss the bug.
It seems quicker to look at the network communications with the client. Is there some tool I can use to monitor the client's communications with the local network socket?
I use http://www.charlesproxy.com for all my network debugging.
http://www.wireshark.org/ enables you to monitor local sockets too.
I was able to monitor local loopback even on windows using trick whit adding route.
http://wiki.wireshark.org/CaptureSetup/Loopback check Other Alternatives
Or you can just write raw socket server that listen on client side on one port and send data to server on other port and vice versa and prints out all data. It should not take more than dozen of lines of code
I want to write a Python script that will check the users local network for other instances of the script currently running.
For the purposes of this question, let's say that I'm writing an application that runs solely via the command line, and will just update the screen when another instance of the application is "found" on the local network. Sample output below:
$ python question.py
Thanks for running ThisApp! You are 192.168.1.101.
Found 192.168.1.102 running this application.
Found 192.168.1.104 running this application.
What libraries/projects exist to help facilitate something like this?
One of the ways to do this would be the Application under question is broadcasting UDP packets and your application is receiving that from different nodes and then displaying it. Twisted Networking Framework provides facilities for doing such a job. The documentation provides some simple examples too.
Well, you could write something using the socket module. You would have to have two programs though, a server on the users local computer, and then a client program that would interface with the server. The server would also use the select module to listen for multiple connections. You would then have a client program that sends something to the server when it is run, or whenever you want it to. The server could then print out which connections it is maintaining, including the details such as IP address.
This is documented extremely well at this link, more so than you need but it will explain it to you as it did to me. http://ilab.cs.byu.edu/python/
You can try broadcast UDP, I found some example here: http://vizible.wordpress.com/2009/01/31/python-broadcast-udp/
You can have a server-based solution: a central server where clients register themselves, and query for other clients being registered. A server framework like Twisted can help here.
In a peer-to-peer setting, push technologies like UDP broadcasts can be used, where each client is putting out a heartbeat packet ever so often on the network, for others to receive. Basic modules like socket would help with that.
Alternatively, you could go for a pull approach, where the interesting peer would need to discover the others actively. This is probably the least straight-forward. For one, you need to scan the network, i.e. find out which IPs belong to the local network and go through them. Then you would need to contact each IP in turn. If your program opens a TCP port, you could try to connect to this and find out your program is running there. If you want your program to be completely ignorant of these queries, you might need to open an ssh connection to the remote IP and scan the process list for your program. All this might involve various modules and libraries. One you might want to look at is execnet.