My goal is to write some sort of chatterbot that speaks in spanish. I've seen examples for simple python chatterbots, but their answers are mostly just random. I want the bot to partially understand the question and formulate a response to it, like http://cleverbot.com does. I'm guessing I would want to use python for this since I want it to be a web app. What's involved in making something like this? Are there any libraries that do this already?
Thanks in advance for any help.
For a starting place, I'd take a look at the Natural Language Toolkit: http://www.nltk.org/.
Spanish chatterbots are listed on:
http://www.chatbots.org/language/spanish/
There is a forum for beginners 'New to chatbot programming'
http://www.chatbots.org/ai_zone/viewforum/35/
And a document with the very first guidelines:
/ai_zone/viewthread/492/
And a support forum for a recent launched tool to build a chatbot your self, called ChatScript:
/ai_zone/viewforum/44/
As I new member, I'm only allowed to leave 2 hyperlinks, which I understand, but you probably know how to extend to the full URL :-)
Related
Firstly, apologies for the very basic question. I have looked into other answers but they haven't quite answered what I'm after. I'm confident designing a site in HTML/CSS and have very very basic knowledge of Python.
I want to run a very basic Python script on my website. It analyses tweets about a specific topic, and then posts a sentiment analysis score. I want it to run this sentiment analysis every hour and cache the score.
I have a working Python script which does this in Jupyter Notebook. Could you give me an overview of how I would make this script function online and cache the results? I've read into using Python web frameworks, but from my limited understanding, they seem like overkill?
Thank you for your help!
Could you give me an overview of how I would make this script function online
The key thing would be to uncouple the two parts of your system:
Producing the data
Showing it in a website.
So the first thing to do is have your sentiment-analysis script push its value to a database. The database could be something as simple as a csv file, or it could be a key/value store, or something like MySQL or CouchDB (or hundreds of other choices).
Over on the website you have to make a decision between:
Server-side
Client-side
If the former, you could program in Python if that is what you are most familiar with. Whatever language/framework combination you go for, there will an example tutorial of how to read a value from a database and display it: it is just about the most fundamental thing.
If client-side you will usually be programming in JavaScript. Again you need to choose a framework, but again you should easily be able to find a tutorial to follow.
(Unless you have a good reason to prefer server-side, such as familiarity with an existing framework, or security issues with accessing your database, I'd go with a client-side approach.)
I've read into using Python web frameworks... overkill?
Yes and no. You are going to need some kind of database, and some kind of framework. It would be good to understand the basics of web security, too. If the sentiment analysis is your major goal, all that is going to be a distraction, and it might be better to find a friend who already knows web programming to work with. Or just find a tutorial that is very close to what you want to do, and adapt that.
(P.S. I was going to flag your question as "too broad", but you did ask for an overview, so I hope this helps.)
Hey guys, I have an idea that I'd like to start implementing that at the crux of it, will basically be a chat website, and will need to support multiple rooms. Quite frankly, I'm not too sure where to begin with regards to setting up a very sturdy/scalable chat system in python (or another language if you guys believe it to be a better alternative), so any suggestions that can get me pointed in the right direction will be greatly appreciated.
Look into XMPP. Here's the list of Python libraries.
Google AppEngine supports python provides scalable framework. Probably you'll save lots of time if you use it.
For my GAE app I need to do some natural language processing to extract the subject and object from an input sentence.
Apparently NLTK can't be installed (easily) on GAE so I am looking for another solution.
I noticed GAE comes with Antlr3 but from browsing their documentation it solves a different kind of grammar problem.
Any ideas?
You can easily build and NTLK RPC server on some machine and access it.
Another option is to find another web based service that already does that (such as opencalais).
With regards to the NLTK problem specifically, my solution would probably be to fix the weird imports that NLTK is doing, and use that as originally planned. When you're done, submit a patch of course.
That said, if this ultimately involves touching the data store, the answer is that it probably can't be done in a performant way, unless your data set is small or for some reason your NLP stuff doesn't need to hit some kind of full-text index. The GAE guys are working on it, but they have indicated that no one should be expecting a quick resolution to this particular issue.
I've been meaning to learn another language than java. So I started to poke around with python. I've gone over 'dive into python' so I have a decent knowledge about python now.
where do you suggest I go from here? I dont want to go through another advanced book again and would like to use the python knowledge towards building 'something'.
I've heard that python is good for web crawling, however, I did not see that in dive into python. Can the community suggest how to use my pythong knowledge towards web crawlers or spiders?
That really kind of depends on what you enjoy, or would like to build. Since you haven't said, I'll recommend something I enjoyed instead. Programming Collective Intelligence by Toby Segaran is a fun book, and the examples are all in Python. It might be more interesting to you -- if nothing else, it would give your web crawler something to do with the pages it gathers.
Edit: Fusspawn's suggestion of PyGame is very good, if don't want any more books and just want to "dive in" to something.
You can try my Building Skills in OO Design.
http://homepage.mac.com/s_lott/books/oodesign.html
If you like math try learning Python by solving Project Euler problems using python. Each problem is not too much code and it helped me increase my python skills.
I always find making a small game is a nice way to learn a language
PyGame makes it simple and could help learn more about python. I suggest giving it ago if your that way inclined.
To get started with web crawling, consider the Scrapy framework.
http://scrapy.org/
"Scrapy is a high level scraping and web crawling framework for writing spiders to crawl and parse web pages for all kinds of purposes, from information retrieval to monitoring or testing web sites."
It's still edging towards a first release, but is usable and has decent documentation.
For very basic web scraping, check out Mechanize (for basic web "browsing") and BeautifulSoup (for parsing "html soup"):
http://wwwsearch.sourceforge.net/mechanize/
http://www.crummy.com/software/BeautifulSoup/
One fun thing to do would be to combine these interests with some natural language processing projects. The NLTK book recently published by O'Reilly is available online as well:
http://www.nltk.org/book
Lots of fun to be had combining these interests. :-)
If you want to expand beyond web crawling and don't want to start a your own project (or don't know what to do), check out The Python Challenge. It's a game where you have to solve puzzles with a bit of python code. I really enjoyed it.
Is web crawling something you want to do or just something you think you can accomplish? Python is a good tool for web crawling(see here and here), but if you really just want ANY project to work on to get more familiar to the language/APIs I'd suggest you pick a project that you have a general interest in regardless. That way it'll be easier to stick with to fruition as you already have an interest in the project in addition to an interest in the language.
Find an interesting open source project to participate in. You could start looking on pythonsource or sourceforge.
The Tools/webchecker/ directory, which should be in your Python distribution (otherwise you can get it via the link I gave), is a start -- with lots of limitations (no threading except in wsgui.py, no async operation, ...), but removing some of them would be a great learning experience!
A vastly superior spidering system could be built on top of Twisted, e.g. starting with the snippet at the bottom of this mail (which only gets one page, but in the proper asynchronous way!) and adding the other functionality you see exemplified in webchecker (parse and respect robots.txt, get links from pages, etc, etc).
If you wanna "advanced book", I recommend Alex's Python in a Nutshell, Second Edition, learn quite a lot from the book, and Tarek's Expert Python Programming,we all know it's a advanced book for it's title:) .
For read some open source project, recommend SQLAlchemy and Django.
Maybe try to start you own project is the best way.
Others have said it but I'll repeat: work on something you are interested in or it won't be fun.
If you do decide that a crawler would be fun, take a look at google-kongulo, web spider plugin for Google desktop search. The code is quite short and well-written, so this might make a good base for when you decide what you want to crawl.
If you're specifically interested in crawling the Web, check out the three-part talk called "Scrape the Web" given at PyCon 2009. It's part of this RSS feed.
Read Dive Into Python again, it discusses HTML processing and HTTP web services in chapters 8 and 11.
I have downloaded the Pyscripter and learning Python. But I have no Idea if it has any job value , especially in India. I am learning Python as a Hobby. But it would be comforting to know if Python programmers are in demand in India.
Everywhere. It's used extensively by google for one.
See list of python software for more info, and also who uses python on the web?
In many large companies it is a primary scripting language.
Google is using it along with Java and C++ and almost nothing else.
Also many web pages are built on top of python and Django.
Another place is game development. Many games have their engines written in C++ but all the logic in Python.
In other words it is one of the most valuable tools.
This might be of interest for you as well:
Is Python good for big software projects (not web based)?
Are there any good reasons why I should not use Python?
What did you use to teach yourself python?
It definitely has job value. For instance Google requires it. Have a look at Google openings in India:
Excellent programming skills in at
least one of the following languages:
C, C++, Java or Python (C++/Python
preferred)
Not sure about India, but you can get a decent overview of available Python jobs on the python.org jobs page here.
Try looking at Mark Pilgrim's excellent book "Dive Into Python" which is available for download under GNU Free Documentation License.
HTH
cheers,
Rob
In 10 years of web development I've had 1 client have me write an email parsing app with it. Not that it doesn't get used, but I've seen Ruby/php/.net way more often in the wild.
Edit:
From the other posts if you plan on working at Google, it sounds like the language to learn - LOL!
It's juste one example but I know it is widely used in large scientific institutions with high tech machinery where non-programmers (typically physicists) need quick prototypes or tools to cover their data collection/processing needs. The easy-to access scripting language aspect clearly plays its role here. So I don't know about building a career out of that only but I'd definitely say that knowing Python is a very valuable asset on your resume, it'll strengthen your "smell of usefulness".
The google app engine lets you use python (or Java). I HIGHLY recommend that you check it out. If you want to have a FREE website with a database (actually a datastore but it works much like a database) using python, THIS IS IT. It scales up too. If you start to get enough traffic you would have to start paying for the usage it requires.
http://code.google.com/appengine/docs/python/overview.html
You could make your own python based site and run some ads. Voila, make some money. Also, I'm sure google could be impressed by some good python because I hear they use it for much of their own sites.