Creating a web service best practices - python

I've published an iOS app using a local database stored directly on the user's device. What I'd like to do for my next app is have a central database on a remote server where users can asynchronously send/receive data. I'm relatively new to web service programming and I'm not sure where to start. I've purchased a server space on a web hosting site and have a MySql database/phpMyAdmin configuration on the server. I'm not sure how to handle the server side code. I know well enough that a database should never be exposed publicly over the internet for obvious security reasons. Therefore I need some kind of web service where my iOS/Android apps can query the service, the web service fetches data and sends it back to the clients in a XML or JSON format. I'd like to write the web service layer in python. I've done a little research and django seems to be recommended by some for these kinds of things. So, my questions are:
What are good resources for making a web service in python on a remote server.
What are the "best practices" for creating/debugging/testing the server side code. Should I try and make a local MySql database and write the server code locally and test and then push it to my remote server when it is finished?
My ultimate starting goal is some kind of proof of concept hello world app. Where from my iOS and android devices I can query the remote database going through the service layer and getting data back or inserting data.
Any tips or advise would be appreciated. I'm a noob in this area but ready to learn.

Too many general questions in one, so I'll just refer you.
I'd start with an awesome Designing Poetic APIs PyCon US 2014 talk.
Since you want to follow the REST principles with Django, take a look at the list of django packages available for writing RESTful apps: Applications that help you build a REST API.
Basically, the major "players" are:
django-tastypie
django-rest-framework
Also see:
What are the differences between django-tastypie and djangorestframework?
Choosing an API framework for Django
Hope that helps.

Related

How do I deploy this app for my job: EC2, Elastic Beanstalk, something else entirely?

I'm tasked with creating a web app (I think?) for my job that will tracker something in our system. It'll be an internal tool that staff uses to keep track of the status of one of the things we do. It should look like trello, with cards that drag from step to step. That frontend exists, but my job is to make the system update when the cards are dragged. This requires using an API in Python and isn't that complicated to grab from/update. I have no idea how to put all of this together. My job is almost completely nontechnical and there's no one internally who knows what I'm doing except for me. I'm in so over my head here and have no idea where to begin. Is this something I should deploy on Elastic Beanstalk? EC2? How do I tie this together and put it somewhere?
Are you trying to pull in live data from Trello or from your companies own internal project management tool?
An EC2 might be useful, but honestly, it may be completely unnecessary if your company has its own servers. An EC2 is basically just a collection of rental computers to help with scaling. I have never used beanstalk so my input would be useless there.
From what I can assume from the question, you could have a python script running to pull from the API and make the changes without an EC2.
First thing you should do is gather as much information about what the end product should look like. From your question, I have the feeling that you have only a vague idea of what the stakeholders want. Don't be afraid to ask more clarification about an unclear task. It's better to spend 30 minutes discussing and taking note than to show the end-product after a month and realizing that's not what your boss/team wanted.
Question I would Ask
Who is going to be using this app? (technical or non-technical person)
For what purpose is this being developed?
Does it need to be on the web or can it be used locally?
How many users need to have access to this application?
Are we handling sensitive information with this application?
Will this need to be augmented with other functionality at some point?
This is just a sample of what I would ask, during the conversation with the stakeholder a lot more will pop up for sure.
What I think you have to do
You need to make a monitoring system for the tasks that need to be done by your development team (like a Kanban)
What I think you already have
A frontend with the card that are draggable to each bin. I also assume that you can create a new card and delete one in the frontend. The frontend is most likely written in React, Angular or Vue.js. You might also have no frontend framework (a mix of jQuery and vanilla js), but usually frontend developper end up picking a framework of sort to help the development.
A backend API in Python (in Flask or with Django-rest-framework most likely) that is communicating with a SQL database like postgresql or a Document database like MongoDB.
I'm making a lot of assumption here, but your aim should be to understand the technology you will be working with in order to check which hosting would work best. For instance, if the database that is setup is a MySQL database you might have some trouble with some hosting provider.
What I think you are missing
Currently the frontend and the backend don't communicate to each other. When you drag a card it won't persist if you refresh the page. Also, all of this is sitting in your computer and cannot be used by any one from your staff. You need to first connect the frontend with the backend so that the application has persistance. Then you need to deploy this application somewhere so that it is reachable by your staff.
What I would do is first work locally to make sure that the layer of persistance is working. This imply having the API server, the frontend server and the database server running simultaneously on your computer to develop. You should then fetch data from the API to know which cards are there in the database and then create them visually in your frontend at the right spot.
When you drop a card to a new spot after having dragging it should trigger a POST request to your API server in order to update the status of this particular card (look at the documentation of your API to check what you need to send).
The server should be sending back an updated version of the cards status if the POST request was sucessful, so your application should then just redraw the card at the right spot (it won't make a difference for you since they are already at the right spot and your frontend framework will most likely won't act on this change since the state hasn't changed). That's all I would do for that part.
I would then move to the deployment phase to make sure that whatever you did locally can still work online. I would use Heroku to start instead of jumping directly to AWS. Heroku is a service built on top of AWS which manage a lot of the complexity of AWS for you. This is great for prototyping and it means that when your stuff is ready you can migrate to AWS easily and be confident that a setup exist to make your app work. You might also be tied up to your company servers, which is another thing I would ask to the stakeholder (i.e. where can I put this application and where I can't put it).
The flow for a frontend + api + database application on Heroku is usually as follow. You create a github repo for your frontend (make it private) and you create an app on Heroku that will watch this repository for changes. It will re-deploy the application for you when it sees a change at a specific subdomain of Heroku hosting. You will need to configure some procfiles that will tell Heroku what to do with a given application type. This is where you need to double check what frontend you are using since that might change the procfiles used. It's most likely a node.js based frontend (React, Angular or Vue) so head over here for the documentation of how to put that online.
You will need to make a repo for the backend also that is separate from the frontend, these two entities are distinct and they only communicate through HTTP request (frontend->backend) and JSON (backend->frontend). You will need to follow the same idea as with the frontend to deploy, head over here.
Once you have these two online, you need to create a database on Heroku. This is done by adding a datastore to your api, head over here. There are some framework specific configuration you need to do to make the API talk to an online database, but then you will need to find that configuration on the framework documentation. The database could also be already up and living on your server, if this is the case you just need to configure your online backend to talk to that particular database at a particular address.
Once all of the above is done, re-test your application to check if you get the same behavior as before. This is a usable MVP, however there are no layer of security. Anyone with the right URL could just fetch your frontend and start messing around with your data.
There is more engineering that need to be done to make this a viable end product. This leads us to my final remark: why you are not using a product like Trello, Jira, or even Github Project? If it is to save some money on not paying for a subscription I think you should factor in the cost of development, security and maintenance of this application.
Hope it helps!
One simple option is Heroku for deploy your API and your frontend application.

Capacity of Django Development server

I have a django powered website which allows uploading/downloading of some event, where event contain fields like geocode, some text, an image.
Hopefully the events would be around 1000-1500 at max but they might come simultaneously.
Can django development server handle the pilot load or should I shift to standard web server(will have to do some changes for that)?
you should definitely switch, django dev server is pretty much a toy, a simple single threaded server easily used for development, there are very few steps involved with serving your django application using apache –
If someone uploads a large image, the server will block all other requests during that time, I believe that alone is good enough reason to switch servers.
Additionally, advice from the first page tutorial in django documentation:
Now’s a good time to note: don’t use this server in anything
resembling a production environment. It’s intended only for use while
developing. (We’re in the business of making Web frameworks, not Web
servers.)

How to re-architect a portal for creating mobile app

Currently I am working on a portal which is exposed to end users. This portal is developed using Python 2.7, Django 1.6 and MySQL.
Now we want to expose this portal as a mobile app. But current design does not support that as templates, views and database are tightly coupled with each other. So we decided to re-architect the whole portal. After some research I found following:
Client side: AngularJS for all client side operations like show data and get data using ajax.
Server side: Rest API exposed to AngularJS. This Rest API can be developed using either Tastypie or Django Rest Framework (still not decided). Rest API will be exposed over Django.
I have few questions:
What you guys think about architecture? Is this is a good or bad design? How it can be improved?
Will performance of portal will go down after adding above layers in architecture?
In the above architecture whether 2 servers should be used to run this (like one for client and other for serving the API's) or one server will be enough. Currently Heroku is used for deployment.
Currently portal is getting 10K hits in a day and it is expected to go to 100K a day in 6 months. Will be happy to provide more information if needed.
If i got an opportunity to architect the portal which you mentioned than i would really love to design the architecture which i have already explained here.
What you guys think about architecture?
This is a common Service Oriented Architecture with decoupled Clients. You just have REST endpoints on your backend, and any Client can consume those endpoints.
You should also think about:
Do you need RESTful service (RESTful == stateless, will you store any state on the server?)
How to scale the service in the future? (this is a legit thing as you already aware of huge traffic increase and assume 2 servers)
How it can be improved?
Use scala instead of python :)
Will performance of portal will go down after adding above layers in architecture?
It depends.
It will get some performance penalty (any additional abtract layer has it's tax), but most probably you won't event notice it. But still, you should measure it using some stress tests.
In the above architecture whether 2 servers should be used to run this (like one for client and other for serving the API's) or one server will be enough. Currently Heroku is used for deployment.
Well, as usual, it depends.
It depends on the usage profile you have right now and on the resources available.
If you are interested in whether the new design will perform better than the old one? - there are a number of parameters.
Resume
This is a good overall approach for the system with different clients.
It will allow you:
Totally decouple mobile app and frontend development from backend development. (It could be different independent teams, outsourceable)
Standardize your API layer (as all clients will consume the same endpoints)
Make you service scalable easier (this includes the separate webserver for static assets and many more).

Desktop Python App with Online Storage

I am looking for a 'sanity check' before I start working on this, as I'm new to writing server-side code. I want to stick to Python if possible, since that's what I'm used to!
I have written a desktop app (wxPython) that allows offsite employees to record their working times, the results of which they currently email to the company. I want to be able to have them save data directly to an 'online' location, from which the company can get summary data.
From what I have read (mostly here on StackOverflow) leads me to think I should do the following:
Run a database on the server with local access only (I'm favouring RethinkDB...)
Write a Python server app that can access the database but only exposes the functionality needed per user. Probably with different ports for users, payroll, and admin (me). Secure the sockets with TLS.
Add code to the desktop app to access the server.
Is this a good approach, or am I reinventing wheels and should learn to use Django or some other web framework?
As Paulo Almeida suggested in the comments making a REST application you interface with your wxPython application is probably the way to go. For this django may be a solution but it is probably an overkill microframework such as web.py, flask or bottle is enough and more easier to grasp

Flask login together with client authentication methods for RESTful service

Here is the situation:
We use Flask for a website application development.Also on the website sever, we host a RESTful service. And we use Flask-login for as the authentication tool, for BOTH the web application access and the RESTful service (access the Restful service from browsers).
Later, we find that we need to, also, access the RESTful from client calls (python), so NO session and cookies etc. This gives us a headache regarding the current authentication of the RESTful service.
On the web, there exist whole bunch of ways to secure the RESTful service from client calls. But it seems no easy way for them to live together with our current Flask-login tool, such that we do not need to change our web application a lot.
So here are the question:
Is there a easy way(framework) so the RESTful services can support multiple authentication methods(protocols) at the same time. Is this even a good practice?
Many thanks!
So, you've officially bumped into one of the most difficult questions in modern web development (in my humble opinion): web authentication.
Here's the theory behind it (I'll answer your question in a moment).
When you're building complicated apps with more than a few users, particularly if you're building apps that have both a website AND an API service, you're always going to bump into authentication issues no matter what you're doing.
The ideal way to solve these problems is to have an independent auth service on your network. Some sort of internal API that EXCLUSIVELY handles user creation, editing, and deletion. There are a number of benefits to doing this:
You have a single authentication source that all of your application components can use: your website can use it to log people in behind the scenes, your API service can use it to authenticate API requests, etc.
You have a single service which can smartly managing user caching -- it's pretty dangerous to implement user caching all over the place (which is what typically happens when you're dealing with multiple authentication methods: you might cache users for the API service, but fail to cache them with the website, stuff like this causes problems).
You have a single service which can be scaled INDEPENDENTLY of your other components. Think about it this way: what piece of application data is accessed more than any other? In most applications, it's the user data. For every request user data will be needed, and this puts a strain on your database / cache / whatever you're doing. Having a single service which manages users makes it a lot nicer for you to scale this part of the application stack easily.
Overall, authentication is really hard.
For the past two years I've been the CTO at OpenCNAM, and we had the same issue (a website and API service). For us to handle authentication properly, we ended up building an internal authentication service like described above, then using Flask-Login to handle authenticating users via the website, and a custom method to authenticate users via the API (just an HTTP call to our auth service).
This worked really well for us, and allowed us to scale from thousands of requests to billions (by isolating each component in our stack, and focusing on user auth as a separate service).
Now, I wouldn't recommend this for apps that are very simple, or apps that don't have many users, because it's more hassle than it's worth.
If you're looking for a third party solution, Stormpath looks pretty promising (just google it).
Anyhow, hope that helps! Good luck.

Categories

Resources