I'm a freelance editor and tutor as well as a fiction writer and artist looking to transition to the latter on a full-time basis. Naturally, part of that transition involves constructing a website; a dynamic site to which new content in various forms can be added with ease. Now, I've always intended to learn how to program, and I simply haven't the money to hire someone else to do it. So, having had a good experience with my brief dabblings in Python, I decided I'd go with Django for building my site.
I set up a Fedora Virtualbox for a development environment (as I didn't want to jump through hoops to make Windows work) and went to town on some Django tutorials. Everything went swimmingly until life intervened and I didn't touch the project for three weeks. I'm in a position to return to it now, but I've realized two things in the process. First, I'm having to do a fair bit of retracing of my steps just to find where certain files are, and second, I don't know how I'd go about deploying the site after I'm done building it. My intention is to get the cheapest Linode and host off that until some theoretical point in the future where I required more.
I suspect that re: the file organization issue, that's just something I'll become more familiar with over time, though if there are any tricks I should be aware of to simplify the structure of my overall Django development space, I'm eager to know them. However, what about deployment? How viable is it to, with sufficient knowledge, automate the process of pushing the whole file structure of a site with Git? And how can I do that in such a way that it doesn't tamper with the settings of my development environment?
As a Django developer i can assure you that it grows on you and becomes easier to understand the development environment.
You should remember that settings.py is probably going to be where your thoughts will be for quite a while in the start; the good part is that its only once, after you got it up and running you'll only touch settings.py to add new modules or change some configuration but its unlikely.
I believe there are hosts that integrate with git so that should not be a problem since you will probably just git clone your project's url into the host (and not forget to enable/configure wsgi)
To leave the settings.py out of the mess, you will tell git not to track the file with: git rm file; and then when you add your files for commit you do it with git add -u so it refers only to your tracked files.
I'm not sure if i was clear enough. (probably not) But, i hope i could help you in some way.
Related
We have few Zope&Plone projects in our company and until today I was only one single developer developing all changes throught ZMI or ZopeEdit. Our company is growing so I need to start cooperating with others developers which can help me with developing features and solving bugs in projects. This means that is no more possible to use ZMI but every developer needs to make and test own changes without affecting others work and paste own changes to production enviroment using git merge in git repo.
I need to move development to git - this means I need to start tracking all portals files and settings in git.
I think I need to move whole projects from ZODB/ZMI (including templates, scripts, sql methods, properties as portal_properties or portal_javascripts etc.) to filesystem and run git on this file system. In the next step every developer can install own pure Plone instance, pull source code and settings from git, create own branch, make changes, test, commit, push, code review ...
My question is: Is there any way to do this and start well-known rapid development process using git? Supports ZODB something like "live migration" of content/settings to/from filesystem? Is there any way to tel Zope to load some folder with content/settings from filesystem instead only from ZODB?
I know there is something called eggs, but is possible to move all types of files mentioned above to separated egg?
Thank you for your help.
The way your company was following until now was the "Old Way Way" of Plone development, but this was a deprecated and discouraged way to do.
Nowadays ZMI can still be used for "quick and dirty" fixes, but commonly this changes stored in DB must be removed (and moved to real code) as soon as possible. This was already possible on Plone 2.0!
More important: every new Plone release tend to reduce the ZMI powers (for example: until Plone 2.1 you were able to do lot of stuff from ZMI, starting from Plone 2.5 some UI elements where impossible to be modified TTW).
So: the answer to your question is "yes". Plone can (must) read code from filesystem, and this code can be stored on VCS (it can be git svn, ...).
All of those information can be found in the Plone Developer Manual.
Creating a new Plone package for modern Plone? Use mr.bob.
Automatically integrate VCS in your buildout? Use mr.developer
If you are starting today from a project you where developing through ZMI you must probably mode code from ZMI to filesystem.
This can be done manually; it's simpler as you are using Zope External Editor.
There is also a very old add-on (Plone Skin Dump) for flushing skin content to filesystem, but I fear it won't work on recent Plone, thus it was not supporting some stuff like SQL methods (if you are using them).
You can have a look at http://docs.plone.org/develop/
There you can find how to create a package (egg). Its source code can be added to git. You can checkout your git repository using mr.developer during buildout. https://pypi.python.org/pypi/mr.developer/
You can use mr.bob to create a PloneAddon. For MySQL you can use MySQL Python. Add this Packages to your Buildout in the eggs section.
Then you can write your own MySQL Statements in the Addon. I know reimplementation is expensive, but you have more control in the future.
I'm trying to setup nginx, gunicorn, supervisor and trac, it is all working fine, even the static files which I asked for help here some days before... but the authentification. How does it works??
I know about the basic/digest authentification with nginx... but is there any plugin or something to have authentification without to setup basic/digest authentification in nginx? I mean, is there some web-based authentification?
I've found an AccountManager plugin, but seems that it has been discontinued or it doesn't receive any update (at least for installation directions)
If it is not posible to setup trac with some kind of web/psql authentification, tell me about an alternative based in python, please.
I'm considering taiga because it's based on django at the back and looks very good, but I'm afraid it goes private software in the future, so i wanted to find a completely open source solution here.
Thanks
I'm considering taiga because it's based on django at the back and
looks very good, but I'm afraid it goes private software in the
future, so i wanted to find a completely open source solution here.
Taiga developers here! Taiga is not going open source in the future. We will charge for private projects, but public projects will always be free and Open Source self-hosted version will always be available. So please, don't worry about this or write us to our mailing list (some people asked this already)
https://groups.google.com/forum/#!forum/taigaio
We are happy that you considered Taiga.
I have several small Python libraries that I wrote with stuff that I find myself wanting over and over again. I think most programmers have something similar. I want to use these libraries from a variety of different machines so I've started keeping this stuff in my DropBox. However, I'd like to be able to use my code on machines on which I can't install DropBox or other cloud storage applications, even in portable form. I can just download the files every time one of them changes (DropBox can provide me a URL for each file in my Public folder), which is only a moderate nuisance. But--and I admit this is a longshot--is there a solution out there that will let me tell Python to load a library from my DropBox via http?
BTW, I'd like to add the whole remove folder to my sys.path, but getting a URL for a folder is complicated, so I'm going to try to walk before I run by starting with individual files.
Yes, it's possible. I think you want the combination of two previous questions:
How to download a file in python over HTTP
How to dynamically load a library in python
So your task basically breaks down into writing a little bit of glue code: download the URL via the first bullet, write it to a local file, and then import that file using the second bullet.
So that's how you'd do that.
BUT - please keep in mind that dynamically downloading and executing code has many potential security downfalls. Will you be doing this over a secure connection? Who else has the ability to manipulate that URL? There are a bunch of security issues inherent in downloading and executing code on the fly. I would ask you to consider going about your solution in a different way, but I'm giving you the answer you're asking for.
As a simple security check, you can establish a known-good hash for your file, and then refuse to import any file other than one that's on the list of known-good hashes. This makes it a pain to update your modules, but gives you a little bit of extra safety.
Don't use DropBox as a Revision control
Pick a real solution like Git
Setup access to the Git repository on one of your servers
Clone the repository to your worker machines and checkout master
Create a develop branch where you put every change you make
Test the changes and when you consider any of them stable, merge it to master
On your worker machines set up a cron job which periodically pulls from master branch of repository (and possibly restarts some Python processes as importing the same module again won't make Python interpreter aware of changes since imported modules are cached)
Enjoy your automatically updated workers :)
Don't feel shame - it happens that even experienced software developers come up with XY problem
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
This may be a little clean and green, however when I try to explain my setup to a rubber duck, many times I awake to find myself in a pile of my own hair and the duck still there, staring at me emotionlessly. thats when I knew it was time to escalate the matter to SO.
What frustrated me was, initially when I follow a guide that I think looks credible, install virtualenv etc etc, and following which, started my hello world project. And while researching on a webserver I was faced with the apache2 vs gunicorn dilema, I figured ok lets use gunicorn. Didn't think too much and got started with another Hello World app, then I realised that I missed out version control and created another hello world project and I am here biting on my nails and eating my hair as I found that I may have likely messed up the file system folder structure and convention, and here I am creating my forth django hello world app.
Someone save me from bootstrap hell.
OS: ubuntu 12.04 server lts
db: MySQl
Framework: Django
Background: To create a reporting app with utilises d3.js
facts: This project will not use PaaS
how I went about doing it,
installed easyinstall
installed pip
installed django ( pip install gunicorn django )
installed gunicorn
setup nginx
started django project (cmd: startproject, runproject)
----- here is the part where things get fuzzy -----
installing git???
installing django-south?
adding h5bp (to-do)
Would appreciate to know if I am doing this in the right order OR any other clear guides for setting up django
This is a difficult question to answer because its more about opinions than anything else; and everyone works a different way. However, if you understand the reason behind the recommendations, then you will be better prepared to judge other bootstrapping scripts and - even better - build your own that does what you want.
Here are things you need to ensure are taken care of.
Isolate your development packages from your system-wide Python install. Use virtual environments to accomplish this. Further, you should make sure you pin your packages so that you don't get surprises when versions are upgraded.
Use any form of version control. It doesn't have to be git. You don't need to use github.com. Use mercurial, cvs, bazaar or whatever else. The important part is you use something. So find out what works for you and get going.
For development, do not even bother with a web server. Django comes with a webserver that you should use during development. Run it with python manage.py runserver. When you are finished with your development, you should deploy any web server that supports wsgi. The documentation provides deployment guides that can help you with this.
Everyone and their dog has a recommendation for folder layout and structure. The problem is, each have their own reasons for recommending it. If you do not understand why you will only get frustrated with the layout. So, my suggestion is start with the standard layout that comes with django. The only thing I would add is that you create a docs/ directory where you stick any/all documentation for your project; and a requirements/ directory where you save all your requirements files and keep both of those updated.
Finally, some of my suggestions.
Use postgresql unless you have a compelling reason to go with mysql. postgresql works best with django and some ORM features (like using distinct on columns) only work with postgresql.
Keep your templates and views simple. Instead of cramming a lot of logic in the views, create other python modules that you can import. This will make your project easy to extend (for example, if you want to create an API). Your templates should have absolutely the minimal amount of logic. If you use custom template tags, make them as simple as possible because they are difficult to debug.
Don't be afraid of creating custom model managers and other tricks in the ORM. There is a lot of stuff you can do there that will make your application easier to develop.
django-south is recommended, but make sure you understand how it impacts your projects before you install it; otherwise you'll just end up after a few iterations with a migration that won't run - then you'll either spend a good amount of your time trying to fix the migrations, or simply dropping the database and starting over.
Found this great resource which was almost exactly what I am looking for.
epicserv updated 2013
https://gist.github.com/epicserve/1332256 (updated 2013)
senko updated 2011
http://senko.net/en/django-nginx-gunicorn/
I have picked up python/django just barely a year. Deployment of django site is still a subject that I have many questions about, though I have successfully manual deployed my site. One of my biggest questions around deployment is what measures can I take to safeguard the source code of my apps, including passwords in django's setting.py, from others, especially when my site runs on a virtual hosting provided by some 3rd party. Call me paranoid but the fact that my source code is running on a third-party server, which someone has the privileges to access anything/anywhere on the server, makes me feel uneasy.
There is almost no scenario where your hosting provider would be interested in your source code. The source code of most websites just isn't worth very much.
If you really feel it is necessary to protect your source code, the best thing to do is serve it from a system that you own and control physically and have exclusive access to.
Failing that, there are a few techniques for obfuscating python, the most straightforward of which is to only push .pyc files and not .py files to your production server. However, this is not standard practice with Django because theft of web site source code by hosting providers is not really an extant problem. I do not know whether or not this technique would work with Django specifically.
If someone has the privileges to access anything/anywhere on the server you can't do much, because what you can do others can do too, you can try some way of obfuscation but that will not work. Only solution is NOT to use such shared repository.
Edit: options
Keep working with shared repository if your data is not very sensitive
Use dedicated hosting from companies like rack-space etc
Use AWS to run your own instance
Use google-app-engine server but that may require a DB change
Run your own server (most secure)
While your source code's probably fine where it is, I'd recommend not storing your configuration passwords in plaintext, whether the code file is compiled or not. Rather, have a hash of the appropriate password on the server, have the server generate a hash of the password submitted during login and compare those instead. Standard security practice.
Then again I could just be talking out my rear end since I haven't fussed about with Django yet.
Protecting source code is not that important IMHO. I would just deploy compiled files and not worry too much about it.
Protecting your config (specially passwords) is indeed important. Temia's point is good.