I use django and apache with mod_wsgi.
I'm trying this module: https://github.com/opiate/SimpleWebSocketServer
Basically I'm trying to integrate websocket server with my django app, so I can share variables and do db queries with both servers.
this is my code using this library:
myserver.py:
from SimpleWebSocketServer import WebSocket, SimpleWebSocketServer
import thread
class socket(WebSocket):
def handleMessage(self):
if self.data is None:
self.data = ''
print self.data
# echo message back to client
self.sendMessage(str(self.data))
def handleConnected(self):
print self.address, 'connected'
#print self.request
#print self.server.connections
def handleClose(self):
print self.address, 'closed'
server = SimpleWebSocketServer('',8001,socket)
#server.serveforever()
thread.start_new_thread(server.serveforever,())
print "done"
I use thread so it wont block the rest of the code.
If I run this code alone, and create a webSocket in the browser:
javascript:
var socket;
function startSocket(){
socket= new WebSocket("ws://localhost:8001");
socket.onopen= function(evt) {test.innerHTML+="connected\n";};
socket.onclose = function(evt) {test.innerHTML+="disconnected\n"};
socket.onmessage = function(evt) { alert(evt.data);};
socket.onerror = function(evt) {alert("error");};
}
startSocket();
everything is working fine. the problem is how to integrate it in my django code.
So I put myserver.py code in the __init__.py file of my project. I've created a setting WEB_SOCKET, to which I assign the SimpleWebSocket instance. it does create the object (I've checked in debug mode), but still- no events, no connection, no nothing. it does not work. why?
And perhaps there's another solution to this problem? I need something easy and simple, like this module, and it must be able to integrate with django.
You just need to add the location of your project to the python path so you can setup in the os.environ the location of your project settings and that would do it.
import sys,os
sys.path.append('path_to_my_dango_project')
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')
from app.models import *
...
obj=Model()
..
obj.save()
Related
I am trying to build a skeleton-code setup for a new project.
In that case I need to use jsonify() which is imported from "flask", in order to make a str -> json.
However when I try to do this, I get a RunTimeError: "Working outside of application context".
When I google, I see a lot of people actually creating an app instance with Flask, but I do not need a app instance. I have seen people get this to work without making the app initialization (app = Flask(name)).
Anybody can explain what I am doing wrong?
This is my Controller:
# Standard library imports
import requests
import json
import logging
# Third party imports
from flask import Flask, request, jsonify
# Internal codebase
from testing import TestObjects as testObj
from core import RequestHandler as rh
from intent import Heartbeat as hb
from intent import CenterCapacity as cc
class Controller():
def __init__(self):
# Key is the naming for action internally, value is the actual syntax for the action, mathcing the DialogFlow configuration.
# If any actions change in DialogFlow, they should be corrected in the dictionary aswell.
self.actionValues = {
'getCenterCapacity': 'get.center.capacity', 'heartbeat': 'heartbeat'}
def main(self, request):
self.action = rh.getActionFromRequest(request)
print(self.action)
if self.action == self.actionValues.get('getCenterCapacity'):
print(cc.getCenterCapacity())
elif self.action == self.actionValues.get('heartbeat'):
print(hb.emptyHeartbeat())
if __name__ == '__main__':
c = Controller()
c.main(testObj.getCapacityObj())
c.main(testObj.getHeartbeatObj())
This is my intent "heartbeat" which causes the issue.
# Third party imports
from flask import jsonify
def emptyHeartbeat():
print("You entered the heartbeat method!")
msg = {
"fulfillmentMessages": [
{
"text": {
"text": [""]
}
}
]
}
return jsonify(msg)
Try replacing the jsonify function with json.dumps https://docs.python.org/3/library/json.html
I have a Django app created using Django rest framework. Below is the configuration that my setup is using:
Django 1.9
mongoDB as backend
gunicorn
nginx
Now I have created an API to enter data in DB and retrieve it using REST. I have test it via postman and it is working fine. We have a firmware which is consuming those APIs and that team want to use SSL socket connection instead of REST API.
I am new in SSL Socket connection and I am not able to find anything on internet that can help me on this. I know it is possible to create socket in Python but I am not able to understand how to use it in Django app to Read/Write data in mongoDB.
Any guidance will be very helpful.
TO READER : I understand you may want to close this question but please put up a remark on how to get guidance on this as SO is the biggest portal for putting up questions.
EDIT 1 : I am adding the code of my serializer.py API.
from rest_framework import serializers
class SaveUserLogs(serializers.Serializer):
token = serializers.CharField(label=_("Token"))
device_ip = serializers.CharField(label=_('Device IP'))
device_name = serializers.CharField(label=_('Device Name'))
device_os = serializers.CharField(label=_('Device OS'))
device_macid = serializers.CharField(label=_('Device MAC ID'))
dest_port = serializers.CharField(label=_('Device Port'))
conn_type = serializers.CharField(label=_('Connection Type'))
date = serializers.CharField(label=_('modified date'))
def validate(self, attrs):
device_macid = attrs.get('device_macid')
token = attrs.get('token')
if device_macid:
tokendetails = validate_token(token)
if not tokendetails:
msg = _('Invalid token.')
raise serializers.ValidationError(msg)
else:
userdetails = tokendetails.user
if userdetails.check_mac_id(device_macid):
all_logs = UserLog.objects.all().order_by("-id")
log_id = 1
if all_logs:
getid = all_logs[0].id
log_id = getid + 1
new_log = UserLog(
id=log_id,
device_ip=attrs.get('device_ip'),
device_name=attrs.get('device_name'),
device_os=attrs.get('device_os'),
device_macid=attrs.get('device_macid').upper(),
dest_port=attrs.get('dest_port'),
conn_type=attrs.get('conn_type'),
)
new_log.save()
print("saving log", log_id)
else:
msg = _('Invalid MAC ID')
raise serializers.ValidationError(msg)
else:
msg = _('Must include "MAC ID".')
raise serializers.ValidationError(msg)
attrs['mac_id'] = userdetails.mac_id
return attrs
It looks like you require socket library provided with python. But using this is not recommended since you will have to take care of lot of low level networking stuff and your program will get complicated.
You should instead keep your REST api and run your nginx server on https. You can then write the code on firmware to send and receive data from the server using https.
If for some reason you don't want to use https then you should use requests library to write your server.
We are using grpc as the RPC protocol for all our internal system. Most of the system is written in Java.
In Java, we can use InprocessServerBuilder for unittest. However, I haven't find a similar class in Python.
Can any one provide a sample code for how to do GRPC unittest in python?
How serendipitous that you have asked this question today; our unit test framework just entered code review. So for the time being the way to test is to use the full production stack to connect your client-side and server-side code (or to violate the API and mock a lot of internal stuff) but hopefully in days to weeks the much better solution will be available to you.
Some example code to get started:
proto
syntax = "proto3";
service MyLibrary {
rpc Search (Request) returns (Response);
}
message Request {
string id = 1;
}
message Response {
string status = 1;
}
python unit test
#!/usr/bin/env python
# coding=utf-8
import unittest
from grpc import StatusCode
from grpc_testing import server_from_dictionary, strict_real_time
import mylibrary_pb2
class TestCase(unittest.TestCase):
def __init__(self, methodName) -> None:
super().__init__(methodName)
myServicer = MyLibraryServicer()
servicers = {
mylibrary_pb2.DESCRIPTOR.services_by_name['MyLibrary']: myServicer
}
self.test_server = server_from_dictionary(
servicers, strict_real_time())
def test_search(self):
request = mylibrary_pb2.Request(
id=2,
)
method = self.test_server.invoke_unary_unary(
method_descriptor=(mylibrary_pb2.DESCRIPTOR
.services_by_name['Library']
.methods_by_name['Search']),
invocation_metadata={},
request=request, timeout=1)
response, metadata, code, details = method.termination()
self.assertTrue(bool(response.status))
self.assertEqual(code, StatusCode.OK)
if __name__ == '__main__':
unittest.main()
I find pytest-grpc is easy to follow and get it works in few minutes.
src: https://pypi.org/project/pytest-grpc/
Lets say I have a very simple web app which is presented as blue if the current president is a democrat and red if they are a republican. A REST API is used to get the current president, via the endpoint:
/presidents/current
which currently returns the json object:
{name: "Donald Trump", party: "Republican"}
So when my page loads I call the endpoint and I show red or blue depending on who is returned.
I wish to test this HTML/javascript page and I wish to mock the back-end so that I can control from within the test environment the API responses. For example:
def test_republican():
# configure the response for this test that the web app will receive when it connects to this endpoint
configure_endpoint(
"/presidents/current",
jsonify(
name="Donald Trump",
party="Republican"
)
)
# start the web app in the browser using selenium
load_web_app(driver, "http://localhost:8080")
e = driver.find_element_by_name("background")
assert(e.getCssValue("background-color") == "red")
def test_democrat():
# configure the response for this test that the web app will receive when it connects to this endpoint
configure_endpoint(
"/presidents/current",
jsonify(
name="Barack Obama",
party="Democrat"
)
)
# start the web app in the browser using selenium
load_web_app(driver, "http://localhost:8080")
e = driver.find_element_by_name("background")
assert(e.getCssValue("background-color") == "blue")
So the question is how should I implement the function configure_endpoint() and what libraries can you recommend me?
As #Kie mentioned, configure_endpoint implementation won't be enough, if you're going to stub the whole server-side within Selenium Python code. You would need a web server or whatever that will response via HTTP to requests from within testing environment.
It looks like the question is partially about testing of client-side code. What I see is that you're trying to make unit-test for client-side logic, but use integration testing suite in order to check this logic (it's strange).
The main idea is as follows.
You're trying to test client-side code. So, let's make mocks client-side too! Because this part of code is completely client-side related stuff.
If you actually want to have mocks, not stubs (watch the difference here: https://stackoverflow.com/a/3459491/882187) it is a better way to mock out HTTP requests inside your Javascript code. Just because you're testing a client-side piece of code, not some parts of server-side logic.
Having it isolated from whatever server-side is - is a great idea that you would love when your project become grow, while more and more endpoints will be appearing.
For example, you can use the following approach:
var restResponder = function() { // the original responder your client-side app will use
this.getCurrentPresident = function(successCallback) {
$.get('/presidents/current', callback);
}
};
var createMockResponder = function(president, party){ // factory that creates mocks
var myPresident = president;
var myParty = party;
return function() {
this.getCurrentPresident = function (successCallback) {
successCallback({"name": myPresident, "party": myParty});
}
};
}
// somewhere swap the original restResponder with new mockResponder created by 'createMockResponder'
// then use it in your app:
function drawColor(restResponder, backgroundEl) {
restResponder.getCurrentPresident(function(data){
if (data.party == "Democrat") $(backgroundEl).style('background-color', 'blue')
else if (data.party == "Republican") $(backgroundEl).style('background-color', 'red')
else console.info('Some strange response from server... Nevermind...');
});
}
Practically, this implementation depends on what do you have at the client-side as a framework. If jQuery, then my example is enough, but it looks very wordy. In case you have something more advanced, like AngularJS, you can do the same in 2-3 lines of code:
// Set up the mock http service responses
$httpBackend = $injector.get('$httpBackend');
// backend definition common for all tests
authRequestHandler = $httpBackend.when('GET', '/auth.py')
.respond({userId: 'userX'}, {'A-Token': 'xxx'});
Check out the docs: https://docs.angularjs.org/api/ngMock/service/$httpBackend
If you're still stick to the idea, that you need mocks inside Selenium tests, please
try this project: https://turq.readthedocs.io/en/latest/
It serves with Python DSL for describing REST responders.
Using turq your mocks will look as follows:
path('/presidents/current').json({'name':'Barack Obama', 'party': 'Democrat'}, jsonp=False)
Also, I would recommend to try stubs instead of mocks and use this Python module: mock-server https://pypi.python.org/pypi/mock-server/0.3.7
You are required to create the directory layout containing corresponding pre-populated JSON responses and to add some boilerplate code in order to make the mock-server respond on 'localhost:8080'. The directory layout for your example will look like this:
stub_obama/
presidents/
current/
GET_200.json # will contain {"name": "Barack Obama", "party": "Democrat"}
stub_trump/
presidents/
current/
GET_200.json # will contain {"name": "Donald Trump", "party": "Republican"}
But the mock_server is based on Tornado, it is very heavy solution for using in tests I think.
I hope, my answer is helpful and informative. Welcome to discuss it! I made tons of projects with Selenium, big and small tests, tested client-side and server-side.
I would use tornado web framework.
import json
import functools
import operator
from tornado import ioloop, web, gen
from tornado.options import define, options
define("data_file", default='default/mock.json', type=str)
class Handler(web.RequestHandler):
def data_received(self, chunk):
pass
def initialize(self, data):
self.data = data
#gen.coroutine
def get(self, *args, **kwargs):
path = self.request.path.split("/")[1:]
path = functools.reduce(
operator.add,
[[k, v[0].decode("utf-8")] for k, v in self.request.query_arguments.items()],
path
)
try:
self.write(functools.reduce(operator.getitem, path, self.data))
except KeyError:
self.set_status(404)
class Application(web.Application):
def __init__(self):
data = {}
with open(options.data_file) as data_file:
data = json.load(data_file)
handlers = [
('(.*)', Handler, {"data": data})
]
settings = dict(
gzip=True,
static_hash_cache=True,
)
web.Application.__init__(self, handlers, **settings)
def main():
io_loop = ioloop.IOLoop.instance()
backend_application = Application()
backend_application.listen(8001)
io_loop.start()
if __name__ == "__main__":
main()
This is a code I used for mocking a REST-API which is a standalone script, but it can be embedded into your test environment as well.
I defined a JSON file which defines the different path components and what should be returned. Like this:
{
"presidents": {
"current": {
"name": "Donald Trump",
"party": "Republican"
}
}
}
I saved this to a mock.json and called the script with a parameter mock_rest.py --data-file="./mock.json".
I hope that gives you a starting point and a good example.
If your load_web_app function uses the requests library to access the REST API, using requests-mock is a convenient way to fake that library's functionality for test purposes.
For those who stumble upon this question, and do not want to end up writing the code to create their own mock server implementations of the API, you can use Mocktastic, which is a downloadable desktop application for Windows, MacOS and Linux, which provides an easy to use GUI to setup your mock API servers.
The documentation of Bottle show how to use beaker for session management, like the following
import bottle
from beaker.middleware import SessionMiddleware
session_opts = {
'session.type': 'file',
'session.cookie_expires': 300,
'session.data_dir': './data',
'session.auto': True
}
app = SessionMiddleware(bottle.app(), session_opts)
#bottle.route('/test')
def test():
s = bottle.request.environ.get('beaker.session')
s['test'] = s.get('test',0) + 1
s.save()
return 'Test counter: %d' % s['test']
bottle.run(app=app)
My problem is, I have multiple bottle applications, and each of them serves a virtual host (that is powered by cherrypy). So I can not use decorate "#bottle.route", instead, I need to use decorate like "app1.route('/test')" , "app2.route('/test')" .
But if I warp app with Beaker middleware, like the following,
app1= Bottle()
app2= Bottle()
app1 = SessionMiddleware(app1, session_opts)
app2 = SessionMiddleware(app2, session_opts)
when python run to the following ,
#app1.route('/test')
def test():
return 'OK'
it will report error, AttributeError: 'SessionMiddleware' object has no attribute 'route'
That's for sure, because now app1 is actually a 'SessionMiddleware' not a Bottle app.
How to solve that issue?
After digging into the beaker source code a little, finally I found the way.
Use the decorator this way:
#app1.wrap_app.route('/test')
def test():
return 'OK'