Develop Reference

gRPC client call gives "too many positional arguments" error -- method being called works fine with apart from gRPC

I am new to gRPC, I am trying to use it to create a microservice from a python app which I developed.
I of course understand the error message in and of itself; I just cannot see how it is true of the code. As far as I can tell everything is correct. I have tested the package in isolation and it is working as expected, so I believe the issue is somewhere in what I am doing with gRPC. (Advanced apologies if I am being a complete idiot somehow.)
I have included the code for my server.py, client.py and .proto file below. The method on the server takes two arguments (self, request) and the client makes the call only passing a request variable. However when I run my client script, I receive the following error:
❯ python3 my_client.py
Traceback (most recent call last):
File "my_client.py", line 16, in <module>
client.Do_a_thing(request=request)
File "/home/me/venvs/my_env/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/me/venvs/my_env/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNKNOWN
details = "Exception calling application: Do_a_thing() takes 2 positional arguments but 3 were given"
debug_error_string = "{"created":"#1631108483.863151653","description":"Error received from peer ipv6:[::1]:50051","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Exception calling application: Do_a_thing() takes 2 positional arguments but 3 were given","grpc_status":2}"
As the method takes one (+ self) argument, and I am passing one argument, I am completely lost as to what is going wrong.
Any and all help, even in the form of advice on how to meaningfully debug a gRPC service, would be massively appreciated.
#server code
import os
import sys
from concurrent import futures
import package # <-- the package I am converting to a microservice
import grpc
from package_pb2 import PackageResponse
import package_pb2_grpc
class PackageService (
package_pb2_grpc.PackageServicer
):
def Do_a_thing(self, request):
return_values = package.function(
var1=request.var1,
var2=request.var2,
var3=request.var3
)
return PackageResponse(response_values=list(return_values))
def serve():
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
package_pb2_grpc.add_PackageServicer_to_server(
PackageService(), server
)
server.add_insecure_port("[::]:50051")
server.start()
server.wait_for_termination()
if __name__ == "__main__":
serve()
#client code
import grpc
from package_pb2_grpc import PackageStub
from package_pb2 import PackageRequest
channel = grpc.insecure_channel("localhost:50051")
client = PackageStub(channel)
request = PackageRequest(
var1 = <some_value>,
var2 = <some_value>,
var3 = <some_value>
)
client.Do_a_thing(request)
#proto file
syntax = "proto3";
message PackageRequest {
string var1 = 1;
string var2 = 2;
string var3 = 3;
}
message PackageResponse {
repeated string response_values = 1; //repeated indicates there may be more than one
}
service Package {
rpc Do_a_thing(PackageRequest) returns (PackageResponse);
}

How to Run WebTest Example?

I'm trying to understand how to use WebTest to do integration testing and I'm stuck on the first example.
I've tried to follow the instructions. First I created a module that contains the code I want to test:
# functions.py
def application(environ, start_response):
"""docstring for application"""
# I added body, otherwise you get an undefined variable error
body = 'foobar'
headers = [('Content-Type', 'text/html; charset=utf8'),
('Content-Length', str(len(body)))]
start_response('200 OK', headers)
return [body]
Then I created a test runner file:
# test.py
from webtest import TestApp
from functions import application
app = TestApp(application)
resp = app.get('/')
assert resp.status == '200 OK'
assert resp.status_int == 200
When I execute test.py, I get the following error:
AssertionError: Iterator returned a non- object: 'foobar'.
What do I need to do to make this sample code from the WebTest documentation run?

In WSGI the body must be an iterable of bytes:
body = b'foobar'

Python restFUL web service - routing to a specific function in a file

I am implementing a simple API in python using werkzeug. I have created a simple application 'localhost'. I want to execute a function after a GET request. I am confused with URL routing. I have gone through this tutorial and implemented routing but still can't figure out how to send a request to another file. Here is my code:
url_map = Map([
Rule('/spell', endpoint='spell_checker.py'),
Rule('/he', endpoint='hello/test')
])
#Request.application
def application(request):
urls = url_map.bind_to_environ(environ)
try:
endpoint, args = urls.match()
print(endpoint + " " + args)
#urls.dispatch('test')
except httplib.HTTPException, e:
return e(environ, start_response)
start_response('200 OK', [('Content-Type', 'text/plain')])
return ['Rul1e points to %r with arguments %r' % (endpoint, args)]
if __name__ == '__main__':
from werkzeug.serving import run_simple
run_simple('127.0.0.1', 4000, application)
I have another another in a file named hello.py in the same directory
def index():
print 'This is the test'
I want to call index function from URL like localhost:4000/hello which will call the index function of the file hello.py
Kindly assist me.

If you want to use a function from an external library first of all you have to import the external library
import foo #your library
And then for calling a function "foo_function" you have to call this function using:
foo.foo_function(args) #where args are declared in the hello.py file

The endpoint is typically a string and can be used to uniquely
identify the URL
So it doesn't bind a function to a url, you have to do it yourself.
After this line
endpoint, args = urls.match()
you can put some kind of control statements to run specific functions, in your case:
if endpoint == "hello/test":
hello.index(request, **args)
assuming you have imported hello module when you point your browser to \he the program will call hello.index function.
http://werkzeug.pocoo.org/docs/0.11/tutorial/#step-4-the-routing

How to catch all exceptions with CherryPy?

I use CherryPy to run a very simple web server. It is intended to process the GET parameters and, if they are correct, do something with them.
import cherrypy
class MainServer(object):
def index(self, **params):
# do things with correct parameters
if 'a' in params:
print params['a']
index.exposed = True
cherrypy.quickstart(MainServer())
For example,
http://127.0.0.1:8080/abcde:
404 Not Found
The path '/abcde' was not found.
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\cherrypy\_cprequest.py", line 656, in respond
response.body = self.handler()
File "C:\Python27\lib\site-packages\cherrypy\lib\encoding.py", line 188, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "C:\Python27\lib\site-packages\cherrypy\_cperror.py", line 386, in __call__
raise self
NotFound: (404, "The path '/abcde' was not found.")
Powered by CherryPy 3.2.4
I am trying to catch this exception and show a blank page because the clients do not care about it. Specifically, the result would be an empty body, no matter the url or query string that resulted in an exception.
I had a look at documentation on error handling cherrypy._cperror, but I did not find a way to actually use it.
Note: I gave up using CherryPy and found a simple solution using BaseHTTPServer (see my answer below)

Docs somehow seem to miss this section. This is what I found while looking for detailed explanation for custom error handling from the source code.
Custom Error Handling
Anticipated HTTP responses
The 'error_page' config namespace can be used to provide custom HTML output for
expected responses (like 404 Not Found). Supply a filename from which the
output will be read. The contents will be interpolated with the values
%(status)s, %(message)s, %(traceback)s, and %(version)s using plain old Python
string formatting.
_cp_config = {
'error_page.404': os.path.join(localDir, "static/index.html")
}
Beginning in version 3.1, you may also provide a function or other callable as
an error_page entry. It will be passed the same status, message, traceback and
version arguments that are interpolated into templates
def error_page_402(status, message, traceback, version):
return "Error %s - Well, I'm very sorry but you haven't paid!" % status
cherrypy.config.update({'error_page.402': error_page_402})
Also in 3.1, in addition to the numbered error codes, you may also supply
error_page.default to handle all codes which do not have their own error_page
entry.
Unanticipated errors
CherryPy also has a generic error handling mechanism: whenever an unanticipated
error occurs in your code, it will call
Request.error_response to
set the response status, headers, and body. By default, this is the same
output as
HTTPError(500). If you want to provide
some other behavior, you generally replace "request.error_response".
Here is some sample code that shows how to display a custom error message and
send an e-mail containing the error
from cherrypy import _cperror
def handle_error():
cherrypy.response.status = 500
cherrypy.response.body = [
"<html><body>Sorry, an error occurred</body></html>"
]
sendMail('error#domain.com',
'Error in your web app',
_cperror.format_exc())
#cherrypy.config(**{'request.error_response': handle_error})
class Root:
pass
Note that you have to explicitly set
response.body
and not simply return an error message as a result.

Choose what's most suitable for you: Default Methods, Custom Error Handling.
I don't think you should use BaseHTTPServer. If your app is that simple, just get a lightweight framework (e. g. Flask), even though it might be a bit overkill, OR stay low level but still within the WSGI standard and use a WSGI-compliant server.

CherryPy IS catching your exception. That's how it returns a valid page to the browser with the caught exception.
I suggest you read through all the documentation. I realize it isn't the best documentation or organized well, but if you at least skim through it the framework will make more sense. It is a small framework, but does almost everything you'd expect from a application server.
import cherrypy
def show_blank_page_on_error():
"""Instead of showing something useful to developers but
disturbing to clients we will show a blank page.
"""
cherrypy.response.status = 500
cherrypy.response.body = ''
class Root():
"""Root of the application"""
_cp_config = {'request.error_response': show_blank_page_on_error}
#cherrypy.expose
def index(self):
"""Root url handler"""
raise Exception
See this for the example in the documentation on the page mentioned above for further reference.

You can simply use a try/except clause:
try:
cherrypy.quickstart(MainServer())
except: #catches all errors, including basic python errors
print("Error!")
This will catch every single error. But if you want to catch only cherrypy._cperror:
from cherrypy import _cperror
try:
cherrypy.quickstart(MainServer())
except _cperror.CherryPyException: #catches only CherryPy errors.
print("CherryPy error!")
Hope this helps!

import cherrypy
from cherrypy import HTTPError
def handle_an_exception():
cherrypy.response.status = 500
cherrypy.response.headers['content-type'] = 'text/plain;charset=UTF-8'
cherrypy.response.body = b'Internal Server Error'
def handle_a_404(status=None, message=None, version=None, traceback=None):
cherrypy.response.headers['content-type'] = 'text/plain;charset=UTF-8'
return f'Error page for 404'.encode('UTF-8')
def handle_default(status=None, message=None, version=None, traceback=None):
cherrypy.response.headers['content-type'] = 'text/plain;charset=UTF-8'
return f'Default error page: {status}'.encode('UTF-8')
class Root:
"""Root of the application"""
_cp_config = {
# handler for an unhandled exception
'request.error_response': handle_an_exception,
# specific handler for HTTP 404 error
'error_page.404': handle_a_404,
# default handler for any other HTTP error
'error_page.default': handle_default
}
#cherrypy.expose
def index(self):
"""Root url handler"""
raise Exception("an exception")
#cherrypy.expose
def simulate400(self):
raise HTTPError(status=400, message="Bad Things Happened")
cherrypy.quickstart(Root())
Test with:
http://127.0.0.1:8080/
http://127.0.0.1:8080/simulate400
http://127.0.0.1:8080/missing

Though this was the one of the top results when I searched for cherrypy exception handling, accepted answer did not fully answered the question. Following is a working code against cherrypy 14.0.0
# Implement handler method
def exception_handler(status, message, traceback, version)
# Your logic goes here
class MyClass()
# Update configurations
_cp_config = {"error_page.default": exception_handler}
Note the method signature. Without this signature your method will not get invoked.Following are the contents of method parameters,
status : HTTP status and a description
message : Message attached to the exception
traceback : Formatted stack trace
version : Cherrypy version

Maybe you could use a 'before_error_response' handler from cherrypy.tools
#cherrypy.tools.register('before_error_response', priority=90)
def handleexception():
cherrypy.response.status = 500
cherrypy.response.body = ''
And don't forget to enable it:
tools.handleexception.on = True

I gave up using CherryPy and ended up using the follwing code, which solves the issue in a few lines with the standard BaseHTTPServer:
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
from urlparse import urlparse, parse_qs
class GetHandler(BaseHTTPRequestHandler):
def do_GET(self):
url = urlparse(self.path)
d = parse_qs(url[4])
if 'c' in d:
print d['c'][0]
self.send_response(200)
self.end_headers()
return
server = HTTPServer(('localhost', 8080), GetHandler)
server.serve_forever()

Python WSGI: Reading env['wsgi.input'] more than once

I am building a simple web service that requires all requests to be signed. The signature hash is generated using request data including the request body. My desire is to have a middleware component that validates the request signature, responding with an error if the signature is invalid. The problem is the middleware needs to read the request body using env['wsgi.input'].read(). This advances the pointer for the request body string to the end, which makes the data inaccessible to other components further down in the chain of execution.
Is there any way to make it so env['wsgi.input'] can be read twice?
Ex:
from myapp.lib.helpers import sign_request
from urlparse import parse_qs
import json
class ValidateSignedRequestMiddleware(object):
def __init__(self, app, secret):
self._app = app
self._secret = secret
def __call__(self, environ, start_response):
auth_params = environ['HTTP_AUTHORIZATION'].split(',', 1)
timestamp = auth_params[0].split('=', 1)[1]
signature = auth_params[1].split('=', 1)[1]
expected_signature = sign_request(
environ['REQUEST_METHOD'],
environ['HTTP_HOST'],
environ['PATH_INFO'],
parse_qs(environ['QUERY_STRING']),
environ['wsgi.input'].read(),
timestamp,
self._secret
)
if signature != expected_signature:
start_response('400 Bad Request', [('Content-Type', 'application/json')])
return [json.dumps({'error': ('Invalid request signature',)})]
return self._app(environ, start_response)

You can try seeking back to the beginning, but you may find that you'll have to replace it with a StringIO containing what you just read out.

The following specification deals with that exact problem, providing explanation of the problem as well as the solution including source code and special cases to take into account:
http://wsgi.readthedocs.org/en/latest/specifications/handling_post_forms.html