I have been going through Flask and it seems that now I have a decent understanding to go ahead and build a web application. However I want to know how would I approach the problem. For eg:
I decide to build a blogging application. The first thing I do is write down all the things that come to my mind, from user registrations to posting data on the blog to publishing it on the site
But after that I just get into coding. I know that is not the right approach. I have heard of UML diagrams, workflow diagrams and stuff. It would be great if someone could just outline the steps they would follow for building a large application and then I could go ahead and learn more about them.For eg,it could be something like the below:-
Collect requirements
Draw UML diagrams
Draw workflow diagrams
start coding
Please suggest an exhaustive list so that I can make my approach much more systematic.
Thanks
That all sounds like a lot of work. Try web2py. You get a lot of stuff, such as user registrations and updating data for a blog, for free. Then just write the code. web2py is not as freeform as flask, there is a "one right way to do things". But for beginners, that can be helpful until you have enough experience to go it on your own.
Related
Firstly, apologies for the very basic question. I have looked into other answers but they haven't quite answered what I'm after. I'm confident designing a site in HTML/CSS and have very very basic knowledge of Python.
I want to run a very basic Python script on my website. It analyses tweets about a specific topic, and then posts a sentiment analysis score. I want it to run this sentiment analysis every hour and cache the score.
I have a working Python script which does this in Jupyter Notebook. Could you give me an overview of how I would make this script function online and cache the results? I've read into using Python web frameworks, but from my limited understanding, they seem like overkill?
Thank you for your help!
Could you give me an overview of how I would make this script function online
The key thing would be to uncouple the two parts of your system:
Producing the data
Showing it in a website.
So the first thing to do is have your sentiment-analysis script push its value to a database. The database could be something as simple as a csv file, or it could be a key/value store, or something like MySQL or CouchDB (or hundreds of other choices).
Over on the website you have to make a decision between:
Server-side
Client-side
If the former, you could program in Python if that is what you are most familiar with. Whatever language/framework combination you go for, there will an example tutorial of how to read a value from a database and display it: it is just about the most fundamental thing.
If client-side you will usually be programming in JavaScript. Again you need to choose a framework, but again you should easily be able to find a tutorial to follow.
(Unless you have a good reason to prefer server-side, such as familiarity with an existing framework, or security issues with accessing your database, I'd go with a client-side approach.)
I've read into using Python web frameworks... overkill?
Yes and no. You are going to need some kind of database, and some kind of framework. It would be good to understand the basics of web security, too. If the sentiment analysis is your major goal, all that is going to be a distraction, and it might be better to find a friend who already knows web programming to work with. Or just find a tutorial that is very close to what you want to do, and adapt that.
(P.S. I was going to flag your question as "too broad", but you did ask for an overview, so I hope this helps.)
I am trying to build a web application that requires intensive mathematical calculations. Can I use Django to populate python charts and pandas dataframe?
You can use any framework to do so. If you worked with Python before I can recommend using Django since you have the same (clear Python) syntax along your project. This is good because you keep the same logic everywhere but should not be your major concern when it comes to choosing the right framework for your needs. So for example if you are a top Ruby-On-Rails developer I would not suggest to learn Django just because of Pandas.
In general: A lot of packages/libraries are written in other languages but you will still be able to use them in Django/Python. So for example the famous "Elasticsearch" Searchbackend has its roots in JAVA but is still used in a lot of Django apps.
But it also goes the other way around "Celery" is written in Python but can be used in Node.js or PHP. There are hundreds of examples but I think you get the Point.
I hope that I brought some light into the darkness. If you have questions please leave them in the comments.
I'm tasked with creating our Google Maps website store locator and so far all I've been able to find is old php tutorials and some new appEngine apps.
The apps look great. They seem to function as designed and it looks like this is the way I need to proceed. I even found a demos here and here and both are perfect.
Problem is, I'm not at the level yet to understand them in order to learn from them and start implementing my own app for our stores. I do plan on using them to learn, but at the moment I'm not at that level yet so I'm not even really learning anything by examining the code.
Is there anything I can use at the moment that is a plugin option while I learn this? Perhaps any python tutorials out there hiding somewhere? I can learn these demos but I really need something for the time being while I'm figuring it all out.
This demo from 2008 might be a bit old but will put you on the right tracks.
There is also locator in geodatastore. Demo
I've been meaning to learn another language than java. So I started to poke around with python. I've gone over 'dive into python' so I have a decent knowledge about python now.
where do you suggest I go from here? I dont want to go through another advanced book again and would like to use the python knowledge towards building 'something'.
I've heard that python is good for web crawling, however, I did not see that in dive into python. Can the community suggest how to use my pythong knowledge towards web crawlers or spiders?
That really kind of depends on what you enjoy, or would like to build. Since you haven't said, I'll recommend something I enjoyed instead. Programming Collective Intelligence by Toby Segaran is a fun book, and the examples are all in Python. It might be more interesting to you -- if nothing else, it would give your web crawler something to do with the pages it gathers.
Edit: Fusspawn's suggestion of PyGame is very good, if don't want any more books and just want to "dive in" to something.
You can try my Building Skills in OO Design.
http://homepage.mac.com/s_lott/books/oodesign.html
If you like math try learning Python by solving Project Euler problems using python. Each problem is not too much code and it helped me increase my python skills.
I always find making a small game is a nice way to learn a language
PyGame makes it simple and could help learn more about python. I suggest giving it ago if your that way inclined.
To get started with web crawling, consider the Scrapy framework.
http://scrapy.org/
"Scrapy is a high level scraping and web crawling framework for writing spiders to crawl and parse web pages for all kinds of purposes, from information retrieval to monitoring or testing web sites."
It's still edging towards a first release, but is usable and has decent documentation.
For very basic web scraping, check out Mechanize (for basic web "browsing") and BeautifulSoup (for parsing "html soup"):
http://wwwsearch.sourceforge.net/mechanize/
http://www.crummy.com/software/BeautifulSoup/
One fun thing to do would be to combine these interests with some natural language processing projects. The NLTK book recently published by O'Reilly is available online as well:
http://www.nltk.org/book
Lots of fun to be had combining these interests. :-)
If you want to expand beyond web crawling and don't want to start a your own project (or don't know what to do), check out The Python Challenge. It's a game where you have to solve puzzles with a bit of python code. I really enjoyed it.
Is web crawling something you want to do or just something you think you can accomplish? Python is a good tool for web crawling(see here and here), but if you really just want ANY project to work on to get more familiar to the language/APIs I'd suggest you pick a project that you have a general interest in regardless. That way it'll be easier to stick with to fruition as you already have an interest in the project in addition to an interest in the language.
Find an interesting open source project to participate in. You could start looking on pythonsource or sourceforge.
The Tools/webchecker/ directory, which should be in your Python distribution (otherwise you can get it via the link I gave), is a start -- with lots of limitations (no threading except in wsgui.py, no async operation, ...), but removing some of them would be a great learning experience!
A vastly superior spidering system could be built on top of Twisted, e.g. starting with the snippet at the bottom of this mail (which only gets one page, but in the proper asynchronous way!) and adding the other functionality you see exemplified in webchecker (parse and respect robots.txt, get links from pages, etc, etc).
If you wanna "advanced book", I recommend Alex's Python in a Nutshell, Second Edition, learn quite a lot from the book, and Tarek's Expert Python Programming,we all know it's a advanced book for it's title:) .
For read some open source project, recommend SQLAlchemy and Django.
Maybe try to start you own project is the best way.
Others have said it but I'll repeat: work on something you are interested in or it won't be fun.
If you do decide that a crawler would be fun, take a look at google-kongulo, web spider plugin for Google desktop search. The code is quite short and well-written, so this might make a good base for when you decide what you want to crawl.
If you're specifically interested in crawling the Web, check out the three-part talk called "Scrape the Web" given at PyCon 2009. It's part of this RSS feed.
Read Dive Into Python again, it discusses HTML processing and HTTP web services in chapters 8 and 11.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I'm a programmer with some experience working on various languages and platforms, but I lack any web development experience (apart of some very basic HTML produced by PHP).
I'm looking for a tutorial about the basics of web programming under Linux. I am less interested with apache configuration and server maintenance which I know quite well, but with the actual building of a website using modern techniques. I am familiar with python, but I'll handle any scripting language quite well.
Do you have any recommendations? Can you tell anything about the W3Schhools tutorials?
Bunch of thanks,
Udi
This is a fairly broad question you are asking. You have to be aware that there are a lot of potential answers, the ones already given here being decent ones. And you have to be aware that it is very much a platform decision that you make, whatever tutorial you choose. And that's because web (application) development is a complex thing that can be addressed on various levels (particularly outside the MS world).
I have no close knowledge about the W3Schools you mention, but on first glance it looks they will be teaching you a lot of basic frontend technology: HTML, XHTML, Javascript, CSS and the like. This is not bad and will give you a solid foundation in these things. But web development is usually not done on this level, as it is too tedious and inflexible for larger applications. And you would be missing out on backend/database technology altogether.
Then there are platforms (and I would guess this is the majority) which have a templating approach. You implement page and business logic in a mix of HTML and programming code in some language (Python, Perl, PHP, ...) within an HTML file that is then processed by an engine to generate the final HTML for the user interface and transaction code for the database. Django and TurboGears are the prominent Python representatives of this, Ruby on Rails probably the biggest name currently. But there are a lot others (how about Scala/Lift?), so it's worth taking the time to see which one you like best. They usually do a good job for the database handling. On the UI side you still have page changes.
In that vein there are platforms that try to move away from HTML with embedded code to a pure programmatical approach. You just write code and use specific APIs of the given platform. "Compiling" your project in one way or the other will then generate all the necessary stuff which you have to deploy in a runtime environment. I think Google's GWT and Eclipse RAP are such approaches, and if you think, dream and breath in Java, this is probably for you.
Yet another approach is interesting when page changes in the browser (the most disruptive part of the web experience) is not good enough anymore, when you want desktop-like user interfaces. The way to attack this is to create "fat web clients", with lots of interaction logic built in, usually in Javascript, and have them interact with a server backend only for essential data transfer using Ajax, REST or RPC protocols. Candidates for the client technology are e.g. qooxdoo or Dojo. On the server side you can still use whatever technology you are comfortable with (from RoR to Servlets and beyond). If I had my pick, I would choose qooxdoo for the frontend, and Erlang/CouchDb on the backend.
You have specifically asked about tutorials, and I haven't mentioned a lot. The point I was trying to make was whatever you choose, it is most likely that you will invest quite a bit of time and effort in that technology since they are all quite deep, and will stick with it for some time. During your evaluation you will also check the instructional material for the given platform (don't forget online videos - they're big these days), but this will inevitably be specific. AFAICS, there is no such thing as a "general introduction" to web programming.
With your Python knowledge, you'll might find tutorials like Django useful. It is modern enough to be used in Google App Engine.
Also try the TurboGears tutorial, another Python web framework. This will give you a different angle on (modern) web programming.
Find an introduction and many pointers to other frameworks on Wikipedia.
Ruby on Rails is really interesting for rapid development. It's clean, it's neat, and it lets you focus on the important things like your database and front end.
There are a plethora of RoR tutorials. There are almost two hundred Railscast tutorial videos on loads of subjects. They get pretty in depth too.
There are also plenty of places to look for help on your current app. APIDock is pretty good for finding method uses and how different parts of Rails work. You might also consider going on freenode IRC and getting in the Ruby chatroom: #ruby.
Hope that's helpful!
ok ... the most important thing is to completely abstract your output mechanism (this may even seem trivial to you, but the truth is, too many people disobey that rule and too few tutorials emphasize this point), so that behind a concise API you have some rendering engine (bet it for HTML, XML, JSON or what so ever), most probably using templates ... this is one of the fundamental aspects of request based web applications (this is the actual difference to desktop apps to me) and covered by any better framework ... using MVC architectures is the next step ... there are tons of MVC frameworks for nearly any server language that do A LOT of work for you ... and MVC is perfect for request based apps ... the seperation between business logic and output generation works just about PERFECT ... the key point to a scalable web application is the implementation of your business logic, which in general always involves databases ... this is also a thing you'll have to work with a lot ... creating good HTML templates is a hell of a work, but i'd claim it is relatively easy once you get the hang of it ... no need to come up with super creative solutions and new approaches here ... plus, to me, styling and skinning is replacable ... it is far more difficult to design a good UI that exposes your functionality in the most efficient way, than to implement it, or even make it fancy ...
in your place, i wouldn't delve too much into CSS unless you really want to DESIGN pages (find someone else to do it. maybe even the HTML templates. seriously, you will learn to hate that VERY quickly, especially if you try to get it work in IE7 or lower). rather try to produce rocksolid semantically well structured HTML (good for SEO and accessibility (look at progressive enhancement for that matter)) and learn JavaScript. look at some good frameworks ... jQuery, Ext ... whatever ... don't reinvent the wheel here ...
apart from that, haxe might be of interest for you ... many helpful libraries on
haxelib ...
well, hope that helps ... ;)
greetz
back2dos
There are a few sources to learn HTML, javascript and CSS which is what you were asking for. w3schools is a little company from Norway with not always very good article, but can be used as a quick reference.
I would recommend the following two
WASP InterAct Curriculum
Web Standards Curriculum
There is also HTTP which most people do not really grok. A good way of understanding HTTP is going through REST as an architecture style. Joe Gregorio has created a wonderful series of articles to implement a Web service step by step.
The RESTful Web
Hope it helps.
If you think testing is important, you might be interested in following a TDD (test-driven-development) approach - so, learning how to test Python web apps is as important as learning how to code Python web apps...
I've written a tutorial that starts from zero, aiming to teach Python web development and TDD at the same time. It covers browser-based testing with Selenium as well as unit testing.
http://www.tdd-django-tutorial.com/
comments and suggestions welcomed!