urllib2.URLError when using Quandl for Python behind a proxy - python

I'm posting this because I tried searching for the answer myself and I was not able to find a solution. I was eventually able to figure out a way to get this to work & I hope this helps someone else in the future.
Scenario:
In Windows XP, I'm using Python with Pandas & Quandl to get data for a US Equity security using the following line of code:
bars = Quandl.get("GOOG/NYSE_SPY", collapse="daily")
Unfortunately, I was getting the following error:
urllib2.URLError: <urlopen error [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond>
#user3150079: kindly Ctrl+X / Ctrl+V your solution as an [Answer]. Such MOV is perfectly within StackOverflow
Solution:
I recognized that this was an issue with trying to contact a server without properly targeting my network's proxy server. Since I was not able to set the system variable for HTTP_PROXY, I added the following line which corrected the issue:
import os
os.environ['HTTP_PROXY']="10.11.123.456:8080"
Thanks - I'm interested to hear about any improvements to this solution or other suggestions.

You can set your user environment variable HTTP_PROXY if you can't or won't set the system environment variable:
set HTTP_PROXY "10.11.123.456:8080"
python yourscript.py
and to permanently set it (using setx from Windows XP Service Pack 2 Support Tools):
setx HTTP_PROXY "10.11.123.456:8080"
python yourscript.py
Other ways to get this environment variable set include: registry entries, putting os.environ["HTTP_PROXY"] = ..." insitecustomize.py`.

More control using requests without using the Quandl Package:
import requests
def main():
proxies = {'http': 'http://proxy.yourdomain.com:port',
'https': 'http://proxy.yourdomain.com:port',}
url = 'https://www.quandl.com/api/v3/datasets/GOOG/NYSE_SPY.json?collapse=daily'
response = requests.get(url, proxies=proxies)
status = response.status_code
html_text = response.text
repo_data = response.json()
print(repo_data)
print(status)
print('HTML TEXT')
print('=========')
print(html_text)
if __name__ == '__main__':
main()

Related

metatrader error initialize() failed, error code = (-10005, 'IPC timeout')

I got this error
initialize() failed, error code = (-10005, 'IPC timeout')
when execute this code:
import MetaTrader5 as mt5
# display data on the MetaTrader 5 package
print("MetaTrader5 package author: ",mt5.__author__)
print("MetaTrader5 package version: ",mt5.__version__)
# establish connection to the MetaTrader 5 terminal
if not mt5.initialize(login=999999, server="xyz-Demo",password="abcdef"):
print("initialize() failed, error code =",mt5.last_error())
mt5.shutdown()
Can anyone help me please? Thanks in advance
There might be some solutions to your problem:
Try initializing a connection to the MT5 terminal using mt5.initialize() then login into the trading account using mt5.login(account, server, password).
Try closing all previous connections to the mt5 terminal using mt5.shutdown() on all previous scripts
This is how I solved it.
I divided the process into two:
I did the initialization and then,
I logged in.
Importantly, I'm using a Windows PC and everything started working when I changed the path from
"C:\Program Files\MetaTrader 5\terminal64.exe"
to
"C:/Program Files/MetaTrader 5/terminal64.exe"
The code:
def account_login(login = name,password=key, server= serv,):
if mt5.login(login,password,server):
print("logged in succesffully")
else:
print("login failed, error code: {}".format(mt5.last_error()))
def initialize(login = name, server=serv, password=key, path=path):
if not mt5.initialize(path):
print("initialize() failed, error code {}", mt5.last_error())
else:
account_login(login, password, server)
Maybe, We must launch the app.
In actually, I solved same issue by launch the app.

Rasa core agent.handle_channel

I am trying to do a slack integration for my bot. this is my python script that will run the bot on slack:
from rasa_core.channels import HttpInputChannel
from rasa_core.agent import Agent
from rasa_core.interpreter import RasaNLUInterpreter
from rasa_slack_connector import SlackInput
nlu_interpreter = RasaNLUInterpreter('./model/nlu/default/weathernlu')
agent = Agent.load('./model/dialogue', interpreter = nlu_interpreter)
input_channel = SlackInput('*******', #app verification token
'*******', # bot verification token
'********', # slack verification token
True)
agent.handle_channel(HttpInputChannel(5006, '/', input_channel))
My problem is everytime I close the app and try to run it, i can't use the same port. I started with 5000 and you can see I reached 5006 because I had to change it everytime. If I try to run it using the same port I get this error:
OSError: [WinError 10048] Only one usage of each socket address
(protocol/networ k address/port) is normally permitted
Can anyone explain what's going on?
You should check which port are binded using the cmd netstat and also check the process still running on your machine.
Closing your app might not kill the process therefore your previous instance of your app may still use the ports.

Python 3 Read data from URL [duplicate]

I have this simple minimal 'working' example below that opens a connection to google every two seconds. When I run this script when I have a working internet connection, I get the Success message, and when I then disconnect, I get the Fail message and when I reconnect again I get the Success again. So far, so good.
However, when I start the script when the internet is disconnected, I get the Fail messages, and when I connect later, I never get the Success message. I keep getting the error:
urlopen error [Errno -2] Name or service not known
What is going on?
import urllib2, time
while True:
try:
print('Trying')
response = urllib2.urlopen('http://www.google.com')
print('Success')
time.sleep(2)
except Exception, e:
print('Fail ' + str(e))
time.sleep(2)
This happens because the DNS name "www.google.com" cannot be resolved. If there is no internet connection the DNS server is probably not reachable to resolve this entry.
It seems I misread your question the first time. The behaviour you describe is, on Linux, a peculiarity of glibc. It only reads "/etc/resolv.conf" once, when loading. glibc can be forced to re-read "/etc/resolv.conf" via the res_init() function.
One solution would be to wrap the res_init() function and call it before calling getaddrinfo() (which is indirectly used by urllib2.urlopen().
You might try the following (still assuming you're using Linux):
import ctypes
libc = ctypes.cdll.LoadLibrary('libc.so.6')
res_init = libc.__res_init
# ...
res_init()
response = urllib2.urlopen('http://www.google.com')
This might of course be optimized by waiting until "/etc/resolv.conf" is modified before calling res_init().
Another solution would be to install e.g. nscd (name service cache daemon).
For me, it was a proxy problem.
Running the following before import urllib.request helped
import os
os.environ['http_proxy']=''
response = urllib.request.urlopen('http://www.google.com')

Python - Controlling Tor

I'm attempting to control Tor with Python. I've read a couple of the other questions asked about this subject on stackoverflow but none of them answer this question.
I'm looking for a method to have tor give you a 'new identity', a new IP address, when the command is run. I've googled around and found the TorCtl module as a method for controlling tor, but can't find a way to get a new identity. Here's what I have so far for atleast connecting to tor, but can't get any farther.
from TorCtl import TorCtl
conn = TorCtl.connect(controlAddr="127.0.0.1", controlPort=9051, passphrase="123")
Any help on this is appreciated, if there are other modules better then TorCtl that'd be great too! Thank you!
Well, out of luck I managed to find a PHP script that did the exact same thing I wanted, and with the help of that I converted it to work in TorCtl. This is what it looks like for anyone else needing it in the future!
from TorCtl import TorCtl
conn = TorCtl.connect(controlAddr="127.0.0.1", controlPort=9051, passphrase="123")
TorCtl.Connection.send_signal(conn, "NEWNYM")
You can use a similar code in python:
def renewTorIdentity(self, passAuth):
try:
s = socket.socket()
s.connect(('localhost', 9051))
s.send('AUTHENTICATE "{0}"\r\n'.format(passAuth))
resp = s.recv(1024)
if resp.startswith('250'):
s.send("signal NEWNYM\r\n")
resp = s.recv(1024)
if resp.startswith('250'):
print "Identity renewed"
else:
print "response 2:", resp
else:
print "response 1:", resp
except Exception as e:
print "Can't renew identity: ", e
You can check this post for a mini-tutorial
Apparently the stem package works better. You can install tor on your computer and keep it running in terminal. Then run the following program:
from stem import Signal
from stem.control import Controller
with Controller.from_port(port = 9051) as controller:
controller.authenticate()
controller.signal(Signal.NEWNYM)
stem is the official package developed by tor.org, and you can see their documentation

Difficulties doing a webpage search with Python through Tor

I am running python 2.5.1 and Tor 0.2.2.34 on OSX 10.5
I have checked the SOCKS question and the Trying to get Tor to work with Python question and the Tor with Python question and have tried them all, and a combination of the above, while running Tor in the background and none have worked really. If I try the "Tor with Python" way (just urllib2) the script works, but my IP goes unchanged when checked by reading and printing the source code on a whatsmyip page in the same way through Python.
this is the script I'm trying to run through Tor:
import socks
import socket
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 8118)
socket.socket = socks.socksocket
import urllib2
web_page = "http://www.cartage.org.lb/en/themes/arts/architec/architecturalstructure/LookingforLiminality/LookingforLiminality.htm"
req = urllib2.Request(web_page)
response = urllib2.urlopen(req)
the_page = response.read()
matches = re.findall('Gianni Vattimo', the_page)
if len(matches) == 0:
print 'RESULTS!'
else:
print 'There were NO results!'
(the web page is just an example and not my actual target obv.)
When I run this script it just stalls in Terminal for an indefinite amount of time. As I said, I've tried different renditions, changing the port to other suggestions, etc, but nothing has worked. Any suggestions or tested fixes?
Thank you.

Categories

Resources