I'm using python-raven to automatically capture all 500 in my django project, and it works great. I'm also forwarding some exceptions that I handle and append them with a special tag to be able to filter them out. The problem is that I can not filter for messages that is missing a particular tag so I'd like to set a default tag for all messages, but can't get it to work.
I've tried the following but it's just ignored.
RAVEN_CONFIG = {
'dsn': 'udp://x:y#z:q/w',
'tags': {'testtag': 'value'},
}
Does anyone know how to send a default tag to sentry?
The following works for me in the settings module.
It adds the environment variable DJANGO_ENVDIR to every message being sent.
from raven.contrib.django.client import DjangoClient as RavenDjangoClient
class SentryDjangoClient(RavenDjangoClient):
def build_msg(self, *args, **kwargs):
data = super(RavenDjangoClient, self).build_msg(*args, **kwargs)
data['tags']['ENVDIR'] = os.environ.get('DJANGO_ENVDIR', 'unset')
return data
if get_env_var('SENTRY_DSN', False):
RAVEN_CONFIG = {
'dsn': get_env_var('SENTRY_DSN'),
# NOTE: timeout set via DSN
}
SENTRY_CLIENT = 'project.settings.base.SentryDjangoClient'
You need to adjust the SENTRY_CLIENT setting, according to where you put the SentryDjangoClient class, which extends the build_msg method.
You could assign a different name to the logger, before logging the message. That would allow you to filter by the "logger" in the Sentry server.
After looking into it a bit more it seems like this is not at all possible ATM.
Related
I'm using the Open Tracing Python library for GRPC and am trying to build off of the example script here: https://github.com/opentracing-contrib/python-grpc/blob/master/examples/trivial/trivial_client.py.
Once I have sent a request through the intercepted channel, how do I find the trace-id value for the request? I want to use this to look at the traced data in the Jaeger UI.
I had missed a key piece of documentation. In order to get a trace ID, you have to create a span on the client side. This span will have the trace ID that can be used to examine data in the Jaeger UI. The span has to be added into the GRPC messages via an ActiveSpanSource instance.
# opentracing-related imports
from grpc_opentracing import open_tracing_client_interceptor, ActiveSpanSource
from grpc_opentracing.grpcext import intercept_channel
from jaeger_client import Config
# dummy class to hold span data for passing into GRPC channel
class FixedActiveSpanSource(ActiveSpanSource):
def __init__(self):
self.active_span = None
def get_active_span(self):
return self.active_span
config = Config(
config={
'sampler': {
'type': 'const',
'param': 1,
},
'logging': True,
},
service_name='foo')
tracer = config.initialize_tracer()
# ...
# In the method where GRPC requests are sent
# ...
active_span_source = FixedActiveSpanSource()
tracer_interceptor = open_tracing_client_interceptor(
tracer,
log_payloads=True,
active_span_source=active_span_source)
with tracer.start_span('span-foo') as span:
print(f"Created span: trace_id:{span.trace_id:x}, span_id:{span.span_id:x}, parent_id:{span.parent_id}, flags:{span.flags:x}")
# provide the span to the GRPC interceptor here
active_span_source.active_span = span
with grpc.insecure_channel(...) as channel:
channel = intercept_channel(channel, tracer_interceptor)
Of course, you could switch the ordering of the with statements so that the span is created after the GRPC channel. That part doesn't make any difference.
Correct me, if I'm wrong. If you mean how to find the trace-id on the server side, you can try to access the OpenTracing span by get_active_span. The trace-id, I suppose, should be one of the tags in it.
I am learning how web apps work and after successfully creating connection between front and back end I managed to perform get request with axiom:
Route in my Flask
#app.route('/api/random')
def random_number():
k = kokos()
print(k)
response = {'randomNumber': k}
return jsonify(response)
my kokos() function
def kokos():
return (890)
Function that I call to get data from backend:
getRandomFromBackend () {
const path = `http://localhost:5000/api/random`
axios.get(path)
.then(response => {this.randomNumber = response.data.randomNumber})
.catch(error => {
console.log(error)
})
}
Now suppose I have an input field in my App with value that I want to use in the function kokos() to affect the result and what is going to be displayed in my app.. Can someone explain me how to do that?
Is this what POST requests are for and I have to post first and then get? Or can I use still GET and somehow pass "arguments"? Is this even GET and POST are for or am I making it too complicated for myself?
Is this the proper way to do these kind of thing? I just have a lot of code in python already written and want to simply exchange data between server and client.
Thank you, Jakub
You can add second argument
axios.get(path, {
params: {
id: 122
}
})
.then ...
You can pass id like this or anything it will be available in get params at python side like we pass in URL.
python side [Flask] (http://flask.pocoo.org/docs/1.0/quickstart/#accessing-request-data)
To access parameters submitted in the URL (?key=value) you can use the args attribute:
def random_number():
id = request.args.get('id', '')
k = kokos(id)
id will be passed kokos function if no id is provided it will be blank ''
you can read axios docu to make complex requests.
https://github.com/axios/axios
if any doubt please comment.
In django-ipware version 2.1 ; old get_real_ip function is deprecated. When I use the new get_client_ip ; my test units are not showing the same results. Means that the two functions do not behave the same.
The following is an original test from django-ipware test unit (not mine)
def test_http_x_forwarded_for_multiple(self):
request = HttpRequest()
request.META = {
'HTTP_X_FORWARDED_FOR': '192.168.255.182, 10.0.0.0, 127.0.0.1, 198.84.193.157, 177.139.233.139',
'HTTP_X_REAL_IP': '177.139.233.132',
'REMOTE_ADDR': '177.139.233.133',
}
ip = get_real_ip(request)
self.assertEqual(ip, "198.84.193.157")
The above works fine of course, but I want to ensure that using the new get_client_ip will give the same results (for a system upgrade purposes). But the test is actually failing the assertion:
def test_http_x_forwarded_for_multiple(self):
request = HttpRequest()
request.META = {
'HTTP_X_FORWARDED_FOR': '192.168.255.182, 10.0.0.0, 127.0.0.1, 198.84.193.157, 177.139.233.139',
'HTTP_X_REAL_IP': '177.139.233.132',
'REMOTE_ADDR': '177.139.233.133',
}
ip, is_routable = get_client_ip(request)
self.assertEqual(ip, "198.84.193.157")
resulting:
AssertionError: '177.139.233.132' != '198.84.193.157'
After digging into the code, I found that the new get_client_ip is not iterating inside the meta like get_real_ip . It checks out the left-most ip (or right-most depending on the settings) and skips to the next meta if a Public IP is not found
My question(s) now are:
How can I call get_client_ip in a way that returns the same ip returned by get_real_ip ? What is the logic behind changing the behavior of the function ? Should I trust the new get_client_ip and forget about get_real_ip, or keep using the deprecated get_real_ip and forget about the new get_client_ip ?????
We should forget about the old get_real_ip function and its behavior. See author's reply here: https://github.com/un33k/django-ipware/issues/45#issuecomment-421572304
Lets say I have a very simple web app which is presented as blue if the current president is a democrat and red if they are a republican. A REST API is used to get the current president, via the endpoint:
/presidents/current
which currently returns the json object:
{name: "Donald Trump", party: "Republican"}
So when my page loads I call the endpoint and I show red or blue depending on who is returned.
I wish to test this HTML/javascript page and I wish to mock the back-end so that I can control from within the test environment the API responses. For example:
def test_republican():
# configure the response for this test that the web app will receive when it connects to this endpoint
configure_endpoint(
"/presidents/current",
jsonify(
name="Donald Trump",
party="Republican"
)
)
# start the web app in the browser using selenium
load_web_app(driver, "http://localhost:8080")
e = driver.find_element_by_name("background")
assert(e.getCssValue("background-color") == "red")
def test_democrat():
# configure the response for this test that the web app will receive when it connects to this endpoint
configure_endpoint(
"/presidents/current",
jsonify(
name="Barack Obama",
party="Democrat"
)
)
# start the web app in the browser using selenium
load_web_app(driver, "http://localhost:8080")
e = driver.find_element_by_name("background")
assert(e.getCssValue("background-color") == "blue")
So the question is how should I implement the function configure_endpoint() and what libraries can you recommend me?
As #Kie mentioned, configure_endpoint implementation won't be enough, if you're going to stub the whole server-side within Selenium Python code. You would need a web server or whatever that will response via HTTP to requests from within testing environment.
It looks like the question is partially about testing of client-side code. What I see is that you're trying to make unit-test for client-side logic, but use integration testing suite in order to check this logic (it's strange).
The main idea is as follows.
You're trying to test client-side code. So, let's make mocks client-side too! Because this part of code is completely client-side related stuff.
If you actually want to have mocks, not stubs (watch the difference here: https://stackoverflow.com/a/3459491/882187) it is a better way to mock out HTTP requests inside your Javascript code. Just because you're testing a client-side piece of code, not some parts of server-side logic.
Having it isolated from whatever server-side is - is a great idea that you would love when your project become grow, while more and more endpoints will be appearing.
For example, you can use the following approach:
var restResponder = function() { // the original responder your client-side app will use
this.getCurrentPresident = function(successCallback) {
$.get('/presidents/current', callback);
}
};
var createMockResponder = function(president, party){ // factory that creates mocks
var myPresident = president;
var myParty = party;
return function() {
this.getCurrentPresident = function (successCallback) {
successCallback({"name": myPresident, "party": myParty});
}
};
}
// somewhere swap the original restResponder with new mockResponder created by 'createMockResponder'
// then use it in your app:
function drawColor(restResponder, backgroundEl) {
restResponder.getCurrentPresident(function(data){
if (data.party == "Democrat") $(backgroundEl).style('background-color', 'blue')
else if (data.party == "Republican") $(backgroundEl).style('background-color', 'red')
else console.info('Some strange response from server... Nevermind...');
});
}
Practically, this implementation depends on what do you have at the client-side as a framework. If jQuery, then my example is enough, but it looks very wordy. In case you have something more advanced, like AngularJS, you can do the same in 2-3 lines of code:
// Set up the mock http service responses
$httpBackend = $injector.get('$httpBackend');
// backend definition common for all tests
authRequestHandler = $httpBackend.when('GET', '/auth.py')
.respond({userId: 'userX'}, {'A-Token': 'xxx'});
Check out the docs: https://docs.angularjs.org/api/ngMock/service/$httpBackend
If you're still stick to the idea, that you need mocks inside Selenium tests, please
try this project: https://turq.readthedocs.io/en/latest/
It serves with Python DSL for describing REST responders.
Using turq your mocks will look as follows:
path('/presidents/current').json({'name':'Barack Obama', 'party': 'Democrat'}, jsonp=False)
Also, I would recommend to try stubs instead of mocks and use this Python module: mock-server https://pypi.python.org/pypi/mock-server/0.3.7
You are required to create the directory layout containing corresponding pre-populated JSON responses and to add some boilerplate code in order to make the mock-server respond on 'localhost:8080'. The directory layout for your example will look like this:
stub_obama/
presidents/
current/
GET_200.json # will contain {"name": "Barack Obama", "party": "Democrat"}
stub_trump/
presidents/
current/
GET_200.json # will contain {"name": "Donald Trump", "party": "Republican"}
But the mock_server is based on Tornado, it is very heavy solution for using in tests I think.
I hope, my answer is helpful and informative. Welcome to discuss it! I made tons of projects with Selenium, big and small tests, tested client-side and server-side.
I would use tornado web framework.
import json
import functools
import operator
from tornado import ioloop, web, gen
from tornado.options import define, options
define("data_file", default='default/mock.json', type=str)
class Handler(web.RequestHandler):
def data_received(self, chunk):
pass
def initialize(self, data):
self.data = data
#gen.coroutine
def get(self, *args, **kwargs):
path = self.request.path.split("/")[1:]
path = functools.reduce(
operator.add,
[[k, v[0].decode("utf-8")] for k, v in self.request.query_arguments.items()],
path
)
try:
self.write(functools.reduce(operator.getitem, path, self.data))
except KeyError:
self.set_status(404)
class Application(web.Application):
def __init__(self):
data = {}
with open(options.data_file) as data_file:
data = json.load(data_file)
handlers = [
('(.*)', Handler, {"data": data})
]
settings = dict(
gzip=True,
static_hash_cache=True,
)
web.Application.__init__(self, handlers, **settings)
def main():
io_loop = ioloop.IOLoop.instance()
backend_application = Application()
backend_application.listen(8001)
io_loop.start()
if __name__ == "__main__":
main()
This is a code I used for mocking a REST-API which is a standalone script, but it can be embedded into your test environment as well.
I defined a JSON file which defines the different path components and what should be returned. Like this:
{
"presidents": {
"current": {
"name": "Donald Trump",
"party": "Republican"
}
}
}
I saved this to a mock.json and called the script with a parameter mock_rest.py --data-file="./mock.json".
I hope that gives you a starting point and a good example.
If your load_web_app function uses the requests library to access the REST API, using requests-mock is a convenient way to fake that library's functionality for test purposes.
For those who stumble upon this question, and do not want to end up writing the code to create their own mock server implementations of the API, you can use Mocktastic, which is a downloadable desktop application for Windows, MacOS and Linux, which provides an easy to use GUI to setup your mock API servers.
I have written my own custom dispatcher that uses regular expressions to map routes, however, I can no longer host static files in /static. Here is the dispatcher and the config:
class Dispatcher(object):
def __init__(self):
self.urls = {}
def __call__(self, path_info):
print('Dispatcher called: ' + path_info)
func = self.find_handler(path_info)
cherrypy.serving.request.handler = func
def find_handler(self, path_info):
request = cherrypy.serving.request
request.config = cherrypy.config.copy()
for url in self.urls:
args = re.findall(url, path_info)
if len(args) > 0:
# in the case that the route is just a URL, we don't want
# an extra argument in the method function
try:
args.remove(path_info)
except ValueError:
pass
controller = self.urls[url]
method = request.method.lower()
return cherrypy._cpdispatch.LateParamPageHandler(getattr(controller, method), *args)
return cherrypy.NotFound()
def connect(self, url, controller):
if not url.endswith("$"):
url += "$"
self.urls[url] = controller
And the config:
config = {
'global': {
'server.socket_host': '0.0.0.0',
'server.socket_port': port,
},
'/static': {
'tools.staticdir.on': True,
'tools.staticdir.dir': os.path.join(os.getcwd(), 'static'),
},
'/': {
'request.dispatch': self.dispatcher,
}
}
If I use the standard dispatcher, static files work as they should, however if I use my own, they no longer work. Having done debugging in the dispatcher, static files go through the dispatcher, even though I have specific that only in '/' does the dispatcher get used.
I'm not familiar with cherrypy, but it seems obvious: everything in /static is also in /, so it is anyone's guess which config entry it will use. I would hope that "more specific has priority", but according to your description, this is not the case. Looking at the documentation also doesn't help, there's no mention of ambiguous path handling.
You would think that changing the order might help, but since this is a dictionary, the order isn't preserved.
It seems that cherrypy is unable to do this. If it has a default dispatcher which is overloaded with others, that could solve the problem. Another option is that your custom dispatcher can call the static one if it detects the path.
Finally, the documentation talks about "mounting an application to a path". If you do this, you may want to change the order. If you don't do this, it might be done automatically, and doing it manually may solve your problem.
Not all of this may make sense, since as I wrote I'm not familiar with cherrypy, but I hope it helps you a bit anyway.