I am trying to build a web-app that has both a Python part and a Node.js part. The Python part is a RESTful API server, and the Node.js will use sockets.io and act as a push server. Both will need to access the same DB instance (Heroku Postgres in my case). The Python part will need to talk to the Node.js part in order to send push messages to be delivered to clients.
I have the Python and DB parts built and deployed, running under a "web" dyno. I am not sure how to build the Node part -- and especially how the Python part can talk to the Node.js part.
I am assuming that the Node.js will need to be a new Heroku app, so that it too can run on a 'web' dyno, so that it benefits from the HTTP routing stack, and clients can connect to it. In such a case, will my Python dynos will be accessing it using just like regular clients?
What are the alternatives? How is this usually done?
After having played around a little, and also doing some reading, it seems like Heroku apps that need this have 2 main options:
1) Use some kind of back-end, that both apps can talk to. Examples would be a DB, Redis, 0mq, etc.
2) Use what I suggested above. I actually went ahead and implemented it, and it works.
Just thought I'd share what I've found.
Related
I'm getting my feet wet with GCP and GAE, also nodejs and python and networking (I know).
[+] What I have:
Basically I have some nodejs code that takes in some input and is supposed to then send that input to some python code that will do more stuff to it. My first idea was to deploy the nodejs code via GAE, then host the python code in a python server, then make post requests from the nodejs front-end to the python server backend.
[+] What I would like to be able to do:
just deploy both my nodejs code and my python code in the same project and instance of GAE so that the nodejs is the frontend that people see but so that the python server is also running in the same environment and can just communicate with the nodejs without sending anything online.
[+] What I have read
https://www.netguru.co/blog/use-node-js-backend
Google App Engine - Front and Backend Web Development
and countless other google searches for this type of setup but to no avail.
If anyone can point me in the right direction I would really appreciate it.
You can't have both python and nodejs running in the same instance, but they can run as separate services, each with their own instance(s) inside the same GAE app/project. See Service isolation and maybe Deploying different languages services to the same Application [Google App Engine]
Using post requests can work pretty well, but will likely take some effort to ensure no outside access.
Since you intend to use as frontend the nodejs service you're limited to using only the flexible environment for it, which limits the inter-service communication options - you can't use push queues (properly supported only in the standard environment) which IMHO would be a better/more secure solution than post requests.
Another secure communication option would be for the nodejs service to place the data into the datastore and have the python service pick it up from there - the datastore is shared by all instances/versions/services inside the same GAE app. Also more loosely coupled IMHO - each service can function (at least for a while) without the other being alive (not possible if using the post requests).
Maybe of interest: How to tell if a Google App Engine documentation page applies to the standard or the flexible environment
UPDATE:
Node.JS is currently available in the standard environment as well, so you can use those features, see:
Now, you can deploy your Node.js app to App Engine standard environment
Google App Engine Node.js Standard Environment Documentation
I'm a newbie in django and have a project that involves distributed remote storage and I'm suggested to use mod x-sendfile as part of the project procedure.
In have a django app that receives a file and transforms it into N segments each to be stored on a distinct server. Those servers having a django app receiving and storing the segments.
But since mod x-sendfile works need apache and I am just in developing and trying stage this question occurred to me.
I googled a lot but found nothing in that regard.
So my question being: Is it possible to use apache as django web server during the development of django apps? Does it make sense in development mode to replace apache with django built-in web server?
There should be nothing stopping you from installing a copy of Apache on your workstation and using it for developing, and since you're working on something that depends on some of that functionality it makes perfect sense for you to use that for your development server instead of ./manage.py runserver.
Most people use Djangos built-in server because they don't need more than that for what they're trying to do - it sounds like your solution does.
Heck, since you're testing distributed you may even want to consider grabbing a virtualization tool (qemu, virtualbox, et al) so you can have a faux-distributed setup to work with (I'd suggest doing a bit of scripting to make it easy to deploy / restart them all at once though - it'll save you from having to track down issues where the code that's running is older than you thought it was).
Your development environment can be what you need it to be for what you're doing.
I want to have simple program in python that can process different requests (POST, GET, MULTIPART-FORMDATA). I don't want to use a complete framework.
I basically need to be able to get GET and POST params - probably (but not necessarily) in a way similar to PHP. To get some other SERVER variables like REQUEST_URI, QUERY, etc.
I have installed nginx successfully, but I've failed to find a good example on how to do the rest. So a simple tutorial or any directions and ideas on how to setup nginx to run certain python process for certain virtual host would be most welcome!
Although you can make Python run a webserver by itself with wsgiref, I would recommend using one of the many Python webservers and/or web frameworks around. For pure and simple Python webhosting we have several options available:
gunicorn
tornado
twisted
uwsgi
cherrypy
If you're looking for more features you can look at some web frameworks:
werkzeug
flask
masonite
cherrypy (yes, cherrypy is both a webserver and a framework)
django (for completeness, I know that was not the purpose of the question)
You should look into using Flask -- it's an extremely lightweight interface to a WSGI server (werkzeug) which also includes a templating library, should you ever want to use one. But you can totally ignore it if you'd like.
You can use thttpd. It is a lightweight wsgi server for running cgi scripts. It works well with nginx. How to setup thttpd with Nginx is detailed here: http://nginxlibrary.com/running-cgi-scripts-using-thttpd/
All the same you must use wsgi server, as nginx does not support fully this protocol.
My Flask stops responding when uploading files or when collecting data from another server via GET. I assume, the problem is, that Flask is only running on one thread.
How can I change this, so multiple users can use the site?
Flask's development webserver (invoked when you use app.run) is not a production web server.
Quoting the docs:
You can use the builtin server during development, but you should use a full deployment option for production applications. (Do not use the builtin development server in production.)
If you want to use Flask in a production environment take a look at the deployment options suggested by the documentation.
For testing purposes with small applications that are doing slightly complicated things I deploy the code I'm developing behind CherryPy using this snippet. (The only disadvantage of this pattern is you loose access to Werkzeug's debugger.)
What I want to know are actually 2 things.
Part 1:
I have a developed source code for an application in python that uses Django framework.
What I want is to be able to run the code on a developer machine and see the result. What do I need for it?
(my guesses):
Python development enironment (Eclipse/PyDev/Aptana Studio 3 seem to be the better ones for windows not sure linux yet),
I also have a postgre database already setup (I know there's a file where I have to specify connection information)
- something installed from django or would this be already included in the code that I have?
Part II:
I also want to make a dev server accessible through internet.
- this is the major part of the question and the most important. How do I publish the app?
- I have a linux machine that I would do this on, but unsure of all the things I need. Apache server?
To answer your questions:
What you need: A list of requirements and instructions to get started with Django is available here: http://djangobook.com/en/2.0/chapter02/.
Database: that chapter also includes a section on configuring access to your database, with a specific section on postgreSQL.
Dev server: To start a basic development server, see this tutorial section
Deploying django (production): For instructions on how to deploy Django for production, see chapter on deploying Django.
Publishing on internet: as for making your dev server accessible through the internet, ask on https://serverfault.com/. Make sure you provide more information about your network setup, what you've tried, what isn't working, etc. (Briefly, you need to make sure that the host you are running your server on is on a publicly accessible IP, or has port 80 forwarded to it from such a host. If in doubt, speak to your sys/network admin if you have one. Or use a django hosting service such as those listed on http://djangohosting.com)
IDE : Regarding IDE, it's down to personal preference. What you mentioned are fine and can run on Linux too.
As a first step, I suggest you follow the tutorial which guides you through the process of starting a development server and developing a basic app.
Even if your goal is to deploy an existing app, the tutorial will give you an idea of how the different components work together (apps, models, urls, templates, etc) which will help with debugging when something goes wrong with your deployment.
Good luck.
You need Python, Django, a WSGI container (e.g. mod_wsgi, uWSGI, Paste Deploy), and a database server. You make the Django project available as a WSGI app, bound to the appropriate interface on the machine.