Develop Reference

AIML for Intelligent Answering Engine

I have heard about a programming language called AIML which can be used for programming Intelligent Robots.
I am a web developer and have a web crawler build using Python 2.7 and have indexed Wikipedia ...
So I wanted to build a answering engine using python which would use a string variable
(It is a HUGE variable containing the whole of Wikipedia) as a source of information and use AI to answer...
Finally, I wanted to put this up on my school website...
So can I do that in AIML?
Later on I also want to modify it so as to give my live scores answers to questions like:
"What is the age of ~someperson~?" etc.
For that I'll send my web crawler to index some score pages etc..
Can I program this sort of answering agent in AIML?
If yes please provide links to tutorials which tell me how to do that? (using string variables as a source of information to parse queries and answer like a human)
moreover, AIML uses syntax like:
<category>
<pattern>WHAT ARE YOU</pattern>
<template>
<think><set name="topic">Me</set></think>
I am the latest result in artificial intelligence,
which can reproduce the capabilities of the human brain
with greater speed and accuracy.
</template>
</category>
Where pattern is the query and template is answer, so does that mean I have to sit and write these tags for all possible queries?
Or can I make it use its brains to figure out what the person wants and give them answers
using the string variable as its source of information.
Thank you.

AIML
It looks like AIML is a form of pattern matching. Moreover, it looks like this is mainly meant for dialog based agents. Therefore, to use AIML, you would likely need to manually generate every question and the correct response (answer).
Question answering
What it seems like you are really after is what we call a question answering system. Very briefly, a QA system generally has these components:
Question analysis.
Extract keywords.
(Sometimes) determine expected answer type (location, person, color, number, etc.).
Candidate document selection---doing a search on your knowledge base using an information retrieval system.
Candidate document analysis.
Answer extraction---select some part of the document (sentence(s), paragraph(s)).
Response generation.
Scores and ranks each answer.
Displays the most confident answer(s).
Research
If you're really want to dig deeply into this area, I'd suggest using Google Scholar and search for some of the terms I've mentioned, which will give you some research papers that go into detail about many of these topics. Some papers to get you started:
Natural language question answering: The view from here
Answering complex, list and context questions with LCC's Question-Answering Server
The structure and performance of an open-domain question answering system
Learning surface text patterns for a question answering system
Learning question classifiers
What is not in the Bag of Words for Why-QA?
Shameless plug
I've recently taken a course on natural language processing, and developed a rudimentary QA system that uses Wikipedia as a knowledge base. (Actually, I used the Simple English Wikipedia because it was much easier to work with; though the system does work with the full version just much more slowly.)
If you are interested in looking at some Python code as a reference, you may do so on the project's GitHub page: bwbaugh/causeofwhy. In addition, there is some more detailed documentation on what goes on in each step of the system components.
There is also a very basic working demo of the QA system in action that is (currently) available, however bear in mind the system is a proof-of-concept and can take upwards of 30 seconds to respond to a question (depending on the question).

What are the mature CMSs and Blogs built on web2py?

In search of technologies for developing web applications and portals, I recently dabbled into Ruby and Python (from a non-sysadmin point of view .. ie, towards web application development) and immediately fell in love with python. I have since wanted to only spend time on python based technology for everything (LOL). I have an immediate need to build a weblog that is also able to function as a corporate website, so I started seeking possible python solutions.
I have researched all the major frameworks and like zope/plone on the enterprise, so I will eventually do a lot in plone. However, I also need an 'instant' sort of framework that I can roll out very rapidly and use to test out some concepts in a weblog.
Given the amount of excellent python projects out there, that ought to be easy right? .. Well, WRONG. That has been real tough, and in the end I could never figure out whether to go with Django or web2py. Each had excellent advantages. In the end, I have decided to spare myself the agony and play with both of them initially .. with a hope to quickly discovering the strengths of each of them that are better suited to different projects.
I am going with:
Django --> Django-cms, other plugins
web2py --> InstantPress ?, KPax?, other
plugins?
My my main beef is that there seems to be very little extra information about the web2py based Kpax and Instant press, beyond being listed under 'free appliances' in web2py website. I have also not seen a well-established alternative to kpax and instantpress.
Question (to those familiar and experienced with web2py): where can I read more about instantpress or kpax beyond watching 3-year old movies of them? Or is the idea that I should just get on with it by installing and playing with them?
Thanks in advance for all suggestions and info ..

Make sure you've got the current version, Instant Press 2.0. Here's a recent video. Unfortunately, I don't think there's much documentation, though I believe Martin (the creator) is working on that. Note, IP 2.0 is based on Powerpack 2.0 (see video).
More basic options are web2py-cms and VCMS. You might also consider making use of plugin_wiki.
Also, a few text editing plugins that may be useful: web2py_ckeditor, elRTE WYSIWYG Widget, and plugin_managed_html
KPAX is fairly old -- probably not the best option at this point.
Also, there was a recent discussion among some folks who are interested in joining forces to build a full-featured CMS, so hopefully this will receive some attention right after the upcoming release of web2py 2.0 (very soon).
UPDATE: There are also a few new efforts under development:
web2cms
Movuca (a social CMS)
nanahoshi-cms

I am developing a Social-CMS, By now I don't have too much working besides the core compoments. But the plan is to release the 'alpha' in one month. (my deadline with a client)
https://github.com/rochacbruno/Movuca
The project is inspired in vikuit.com (but will work outside GAE)
While mine social-CMS is not ready, you can go with InstantPress or PowerPack (best options for web2py by now)
Here is a link to the demo. Movuca Demo Link

I was in a similar dilemma, but at the end decided to use Django & Django-CMS or FeinCMS.
Although web2py community is nice, there is simply not a single actively developed CMS with some significant number of developers, community etc.
Here is my post to web2py mailing list about it:
It might be that it's not difficult to write decent CMS and/or strong blog engine using web2py framework, but I consider there are more important things to do (or write) than writing Yet Another CMS/blog. ;)

what next after 'dive into python'

I've been meaning to learn another language than java. So I started to poke around with python. I've gone over 'dive into python' so I have a decent knowledge about python now.
where do you suggest I go from here? I dont want to go through another advanced book again and would like to use the python knowledge towards building 'something'.
I've heard that python is good for web crawling, however, I did not see that in dive into python. Can the community suggest how to use my pythong knowledge towards web crawlers or spiders?

That really kind of depends on what you enjoy, or would like to build. Since you haven't said, I'll recommend something I enjoyed instead. Programming Collective Intelligence by Toby Segaran is a fun book, and the examples are all in Python. It might be more interesting to you -- if nothing else, it would give your web crawler something to do with the pages it gathers.
Edit: Fusspawn's suggestion of PyGame is very good, if don't want any more books and just want to "dive in" to something.

You can try my Building Skills in OO Design.
http://homepage.mac.com/s_lott/books/oodesign.html

If you like math try learning Python by solving Project Euler problems using python. Each problem is not too much code and it helped me increase my python skills.

I always find making a small game is a nice way to learn a language
PyGame makes it simple and could help learn more about python. I suggest giving it ago if your that way inclined.

To get started with web crawling, consider the Scrapy framework.
http://scrapy.org/
"Scrapy is a high level scraping and web crawling framework for writing spiders to crawl and parse web pages for all kinds of purposes, from information retrieval to monitoring or testing web sites."
It's still edging towards a first release, but is usable and has decent documentation.
For very basic web scraping, check out Mechanize (for basic web "browsing") and BeautifulSoup (for parsing "html soup"):
http://wwwsearch.sourceforge.net/mechanize/
http://www.crummy.com/software/BeautifulSoup/
One fun thing to do would be to combine these interests with some natural language processing projects. The NLTK book recently published by O'Reilly is available online as well:
http://www.nltk.org/book
Lots of fun to be had combining these interests. :-)

If you want to expand beyond web crawling and don't want to start a your own project (or don't know what to do), check out The Python Challenge. It's a game where you have to solve puzzles with a bit of python code. I really enjoyed it.

Is web crawling something you want to do or just something you think you can accomplish? Python is a good tool for web crawling(see here and here), but if you really just want ANY project to work on to get more familiar to the language/APIs I'd suggest you pick a project that you have a general interest in regardless. That way it'll be easier to stick with to fruition as you already have an interest in the project in addition to an interest in the language.

Find an interesting open source project to participate in. You could start looking on pythonsource or sourceforge.

The Tools/webchecker/ directory, which should be in your Python distribution (otherwise you can get it via the link I gave), is a start -- with lots of limitations (no threading except in wsgui.py, no async operation, ...), but removing some of them would be a great learning experience!
A vastly superior spidering system could be built on top of Twisted, e.g. starting with the snippet at the bottom of this mail (which only gets one page, but in the proper asynchronous way!) and adding the other functionality you see exemplified in webchecker (parse and respect robots.txt, get links from pages, etc, etc).

If you wanna "advanced book", I recommend Alex's Python in a Nutshell, Second Edition, learn quite a lot from the book, and Tarek's Expert Python Programming,we all know it's a advanced book for it's title:) .
For read some open source project, recommend SQLAlchemy and Django.
Maybe try to start you own project is the best way.

Others have said it but I'll repeat: work on something you are interested in or it won't be fun.
If you do decide that a crawler would be fun, take a look at google-kongulo, web spider plugin for Google desktop search. The code is quite short and well-written, so this might make a good base for when you decide what you want to crawl.

If you're specifically interested in crawling the Web, check out the three-part talk called "Scrape the Web" given at PyCon 2009. It's part of this RSS feed.

Read Dive Into Python again, it discusses HTML processing and HTTP web services in chapters 8 and 11.

Web programming tutorial [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I'm a programmer with some experience working on various languages and platforms, but I lack any web development experience (apart of some very basic HTML produced by PHP).
I'm looking for a tutorial about the basics of web programming under Linux. I am less interested with apache configuration and server maintenance which I know quite well, but with the actual building of a website using modern techniques. I am familiar with python, but I'll handle any scripting language quite well.
Do you have any recommendations? Can you tell anything about the W3Schhools tutorials?
Bunch of thanks,
Udi

This is a fairly broad question you are asking. You have to be aware that there are a lot of potential answers, the ones already given here being decent ones. And you have to be aware that it is very much a platform decision that you make, whatever tutorial you choose. And that's because web (application) development is a complex thing that can be addressed on various levels (particularly outside the MS world).
I have no close knowledge about the W3Schools you mention, but on first glance it looks they will be teaching you a lot of basic frontend technology: HTML, XHTML, Javascript, CSS and the like. This is not bad and will give you a solid foundation in these things. But web development is usually not done on this level, as it is too tedious and inflexible for larger applications. And you would be missing out on backend/database technology altogether.
Then there are platforms (and I would guess this is the majority) which have a templating approach. You implement page and business logic in a mix of HTML and programming code in some language (Python, Perl, PHP, ...) within an HTML file that is then processed by an engine to generate the final HTML for the user interface and transaction code for the database. Django and TurboGears are the prominent Python representatives of this, Ruby on Rails probably the biggest name currently. But there are a lot others (how about Scala/Lift?), so it's worth taking the time to see which one you like best. They usually do a good job for the database handling. On the UI side you still have page changes.
In that vein there are platforms that try to move away from HTML with embedded code to a pure programmatical approach. You just write code and use specific APIs of the given platform. "Compiling" your project in one way or the other will then generate all the necessary stuff which you have to deploy in a runtime environment. I think Google's GWT and Eclipse RAP are such approaches, and if you think, dream and breath in Java, this is probably for you.
Yet another approach is interesting when page changes in the browser (the most disruptive part of the web experience) is not good enough anymore, when you want desktop-like user interfaces. The way to attack this is to create "fat web clients", with lots of interaction logic built in, usually in Javascript, and have them interact with a server backend only for essential data transfer using Ajax, REST or RPC protocols. Candidates for the client technology are e.g. qooxdoo or Dojo. On the server side you can still use whatever technology you are comfortable with (from RoR to Servlets and beyond). If I had my pick, I would choose qooxdoo for the frontend, and Erlang/CouchDb on the backend.
You have specifically asked about tutorials, and I haven't mentioned a lot. The point I was trying to make was whatever you choose, it is most likely that you will invest quite a bit of time and effort in that technology since they are all quite deep, and will stick with it for some time. During your evaluation you will also check the instructional material for the given platform (don't forget online videos - they're big these days), but this will inevitably be specific. AFAICS, there is no such thing as a "general introduction" to web programming.

With your Python knowledge, you'll might find tutorials like Django useful. It is modern enough to be used in Google App Engine.
Also try the TurboGears tutorial, another Python web framework. This will give you a different angle on (modern) web programming.
Find an introduction and many pointers to other frameworks on Wikipedia.

Ruby on Rails is really interesting for rapid development. It's clean, it's neat, and it lets you focus on the important things like your database and front end.
There are a plethora of RoR tutorials. There are almost two hundred Railscast tutorial videos on loads of subjects. They get pretty in depth too.
There are also plenty of places to look for help on your current app. APIDock is pretty good for finding method uses and how different parts of Rails work. You might also consider going on freenode IRC and getting in the Ruby chatroom: #ruby.
Hope that's helpful!

ok ... the most important thing is to completely abstract your output mechanism (this may even seem trivial to you, but the truth is, too many people disobey that rule and too few tutorials emphasize this point), so that behind a concise API you have some rendering engine (bet it for HTML, XML, JSON or what so ever), most probably using templates ... this is one of the fundamental aspects of request based web applications (this is the actual difference to desktop apps to me) and covered by any better framework ... using MVC architectures is the next step ... there are tons of MVC frameworks for nearly any server language that do A LOT of work for you ... and MVC is perfect for request based apps ... the seperation between business logic and output generation works just about PERFECT ... the key point to a scalable web application is the implementation of your business logic, which in general always involves databases ... this is also a thing you'll have to work with a lot ... creating good HTML templates is a hell of a work, but i'd claim it is relatively easy once you get the hang of it ... no need to come up with super creative solutions and new approaches here ... plus, to me, styling and skinning is replacable ... it is far more difficult to design a good UI that exposes your functionality in the most efficient way, than to implement it, or even make it fancy ...
in your place, i wouldn't delve too much into CSS unless you really want to DESIGN pages (find someone else to do it. maybe even the HTML templates. seriously, you will learn to hate that VERY quickly, especially if you try to get it work in IE7 or lower). rather try to produce rocksolid semantically well structured HTML (good for SEO and accessibility (look at progressive enhancement for that matter)) and learn JavaScript. look at some good frameworks ... jQuery, Ext ... whatever ... don't reinvent the wheel here ...
apart from that, haxe might be of interest for you ... many helpful libraries on
haxelib ...
well, hope that helps ... ;)
greetz
back2dos

There are a few sources to learn HTML, javascript and CSS which is what you were asking for. w3schools is a little company from Norway with not always very good article, but can be used as a quick reference.
I would recommend the following two
WASP InterAct Curriculum
Web Standards Curriculum
There is also HTTP which most people do not really grok. A good way of understanding HTTP is going through REST as an architecture style. Joe Gregorio has created a wonderful series of articles to implement a Web service step by step.
The RESTful Web
Hope it helps.

If you think testing is important, you might be interested in following a TDD (test-driven-development) approach - so, learning how to test Python web apps is as important as learning how to code Python web apps...
I've written a tutorial that starts from zero, aiming to teach Python web development and TDD at the same time. It covers browser-based testing with Selenium as well as unit testing.
http://www.tdd-django-tutorial.com/
comments and suggestions welcomed!

Where is Python used? I read about it a lot on Reddit

I have downloaded the Pyscripter and learning Python. But I have no Idea if it has any job value , especially in India. I am learning Python as a Hobby. But it would be comforting to know if Python programmers are in demand in India.

Everywhere. It's used extensively by google for one.
See list of python software for more info, and also who uses python on the web?

In many large companies it is a primary scripting language.
Google is using it along with Java and C++ and almost nothing else.
Also many web pages are built on top of python and Django.
Another place is game development. Many games have their engines written in C++ but all the logic in Python.
In other words it is one of the most valuable tools.
This might be of interest for you as well:
Is Python good for big software projects (not web based)?
Are there any good reasons why I should not use Python?
What did you use to teach yourself python?

It definitely has job value. For instance Google requires it. Have a look at Google openings in India:
Excellent programming skills in at
least one of the following languages:
C, C++, Java or Python (C++/Python
preferred)

Not sure about India, but you can get a decent overview of available Python jobs on the python.org jobs page here.

Try looking at Mark Pilgrim's excellent book "Dive Into Python" which is available for download under GNU Free Documentation License.
HTH
cheers,
Rob

In 10 years of web development I've had 1 client have me write an email parsing app with it. Not that it doesn't get used, but I've seen Ruby/php/.net way more often in the wild.
Edit:
From the other posts if you plan on working at Google, it sounds like the language to learn - LOL!

It's juste one example but I know it is widely used in large scientific institutions with high tech machinery where non-programmers (typically physicists) need quick prototypes or tools to cover their data collection/processing needs. The easy-to access scripting language aspect clearly plays its role here. So I don't know about building a career out of that only but I'd definitely say that knowing Python is a very valuable asset on your resume, it'll strengthen your "smell of usefulness".

The google app engine lets you use python (or Java). I HIGHLY recommend that you check it out. If you want to have a FREE website with a database (actually a datastore but it works much like a database) using python, THIS IS IT. It scales up too. If you start to get enough traffic you would have to start paying for the usage it requires.
http://code.google.com/appengine/docs/python/overview.html
You could make your own python based site and run some ads. Voila, make some money. Also, I'm sure google could be impressed by some good python because I hear they use it for much of their own sites.