As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I am a machine learning beginner. I'd like to learn the basics by teaching computers to play checkers. Actually, the games I want to learn are Domineering and Hex. My language of choice is Python
These games are pretty easy to store and the rules are much simpler than chess, but there aren't too many people who play. If I can get this idea off the ground it would be great for experimenting Combinatorial Game Theory to see if a computer and find the optimal move.
I found this old paper on checkers from the 1960's by a guy at IBM. Originally I had asked about neural networks, but they are saying it's the wrong tool.
EDIT: It could be that machine learning is not the right strategy. In that case, what goes wrong? and what is a better way?
You might want to take a look at the following: Chinook, Upper Confidence Trees, Reinforcement Learning, and Alpha-Beta pruning. I personally like to combine Alpha-Beta Pruning and Upper Confidence Trees (UCT) for perfect information games where each player has less than 10 reasonable moves. You can use Temporal Difference Learning to create a position evaluation function. Game AI is probably the most fun way to learn machine learning.
For links to all of these topics, click on
http://artent.net/blog/2012/09/26/checkers-and-machine-learning/
(I was not able to include more links because the stack overflow software considers me a newbie!)
Get the book called "Machine learning" by McGraw Hill and read the first chapter. It's extremely well written and the first chapter will teach you enough to make a program that plays checkers. Personally I made a program that plays 5 in a row on miniclip.com, also in python.
http://www.amazon.com/Learning-McGraw-Hill-International-Editions-Computer/dp/0071154671
When playing checkers, you seek to gain an advantage over your opponent by taking his or her pieces and crowning your own. Losing your pieces and allowing your opponent to crown his or her pieces is not desirable, so you avoid doing it.
Board game engines usually revolve around a position evaluation function. For checkers, my first guess would be something like this:
score = number of allies - number of opponents
+ 3 * number of crowned allies - 3 * number of crowned opponents
Given a board, this function will return the score of the board. The higher the score, the better your position. The lower the score, the worse your position.
To make a naive checkers "engine", all you need to do is find the best move given a board position, which is just searching through all immediate legal moves and finding the one that maximizes your score.
Your engine won't think ahead more than one move, but it will be able to play against you somewhat.
The next step would to give your engine the ability to plan ahead, which essentially is predicting your opponent's responses. To do that, just find your opponent's best move (here comes recursion) and subtract it from your score.
Related
I want to develop a Math quiz program using reinforcement learning.
Assume that we have 1000 questions in hand and 25 questions to be asked in each quiz. Instead of asking questions at random, program has to learn from the way user answer and ask the next question.
Quiz programme should be a reinforcement learning agent. How to design the solution and which are the reinforcement learning techniques to be used?
Example :
BoT: what is 5+ 1:
User: 3 (Wrong Answer)
Bot: Asked easy question or if correct answer asked a difficult question.
PPO is a very common technique for these type of RL applications in the edTech space. You can take a lot of inspiration from this article. They use the RLgraph package and the PPO algorithm.
You would first have to define your goal/reward function. In your case, I would define the reward function to have something to do with the percentage of previous questions that was answered correctly. If this percentage is 0% or 100%, the reward is low (it's too difficult/easy). If it's close to 50% you might choose the reward to be high.
That way, the algo will move towards questions that get 50% correctness (are medium difficulty). You can play with the range (last 2 q's or last 10 q's).
As state-space, you can also include questions answered correctly, maybe characteristics like age etc. to help initiate the algo when the user hasn't used it too much yet.
As action-space, you can have all the questions. You could also cluster(e.g. difficult/easy, or geometry/algebra) questions based on your intuition and make the clusters actions, to decrease the action space.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I'm going to develop a 3D game, that a player walks in a maze with a 3D first-person perspective, collects things and escapes a monster. The game itself is very simple, but as it is not for entertainment, but for a biological experiment, so it has some specific features:
We will project the graphics to a spherical screen with 3 projectors, so the graphics should be in a fisheye trasformation, and be easily further transformable (to deal with the merging between projectors).
There should be a functionality to record data, like the path of the player, and the time points when the monster appears etc. All the events should be recordable.
The game program could interact with external devices via USB. For example, whenever the player press a certain key, the program will tell an Arduino board to do something.
As my invesigation, I found three candidates of tool chain to develop such a game:
Develop a MOD on Quake3 engine + Fisheye Quake. The problem I think would be that the Quake3 runs with a virtual machine, so that is it possible to implement the feature 2 and 3 above?
Panda3D + FisheyeLens API
PyOpenGL. This is the most flexible way, but with the greatest workload I think.
I'm quite familiar with C/C++/Python, but this is my first time to develop a 3D game. My question is which tool chain is fittest for this project (or any other good options) ? What problem would I encounter?
As there's no answer yet, let me give my own thoughs here. I have no experience on 3D development and don't know if these technology would work. For some side reasons, I prefer Panda3D.
Please note that I'm still open to accept other answers.
Why Panda3D?
It is well-documented. This is always an import factor to choose technology.
It's in Python, which is a plus for me.
It's general-purposed (than Quake3). Panda3D can also be used on robotics simulator etc. which would be useful for me.
Why not Quake3?
It runs on virtual machine (QVM). I'm afraid that it's hard to record data or access external device.
At first I thought of Quake3 because it's at least a very classical code to read, but currently I realized that Quake3 can not do anything else except FPS-like game. If it worth reading, just read it, but not have to work with it.
It's not that flexible to transform the rendering of graphics. It's a game engine for purpose, not a general graphics processing library.
Why not OpenGL?
At this moment I think the Panda3D is bottom enough. If it's proved to be not as flexible as I need, I would turn to OpenGL then.
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
So I've been thinking of creating a program similar to this in Python. The problem is that I know next to nothing about AI. Right now I'm reading up on genetic algorithms and neural networks and such, but there's a lot of things to absorb.
My largest problem now is I don't know how the entire thing is put together. So if anyone can tell me the general framework for a program like this, I'll be really grateful. For example, how the creatures can "see" things in their surroundings, the kind of "chromosomes" they have and how the environment is created.
Also, can anyone suggest suitable libraries for AIs, any engines, etc? I've been thinking of using neurolab, will that be a good idea?
Thanks!
Interestingly, I've created what you're describing, in Python.
Here's a pic:
The green dots are food, and the red ones are organisms that need to evolve and learn how to eat the food blocks.
The best source of inspiration and guidance for something like this is definitely Polyworld: youtube.com/watch?v=_m97_kL4ox0.
Another great example which helped me learn about backpropagation, which I suggest you should learn too, is here: arctrix.com/nas/python/bpnn.py.
Although this question is simply too vague to answer correctly, the general tip is that you first get a solid ground on how ANN's function, and then construct a program with this general algorithm:
Check for user input (should the camera move? should the program quit?)
For each organism:
Check if it is alive, etc.
Take input from the world
Evaluate its neural network
Apply the results to the physical body
Apply positive or negative feedback (as you'll see, it all comes down to this)
Update the world (you can use a physics engine)
Optionally, draw the world
For example, your input neurons could correspond with physical or visual stimulation. Is there something in front of me? Then set neuron X to 1, otherwise to 0.
And your output neurons could map into forces. Is neuron Y active? If so, apply an impulse to this organism's in this world cycle.
I also recommend that you don't use other people's libraries for your neural network computation, because this is a great way to actually learn how to implement them yourself. You could you Pygame for rendering, and PyBox2D for physics, though.
Now, about "seeing" other organisms... These problems are best solved when isolated. You need a good software design, that will enable you to divide the whole problem into many subproblems, which are easier to solve.
In general, seeing can be done by raycasting. Create a directional, normalized vector (where the length is 1 unit, that is, sqrt(x*x + y*y) == 1), and then scale it (x *= factor; y *= factor;). That way, you get a line that's stretching outwards. Before you scale it, you can then loop through all of the organisms in the world (you'll have to optimize this), and check if the vector is inside of them. If it is, calculate its length, and your organism has some information about its surroundings.
There are many simpler ways. For example, divide space around each organism into four quadrants, which four represent eyes (X - up, right, down, left of the organism). Then calculate the length from the organism to every other organism (optimize!). Find the closest organism. Find out in which quadrant it is. Voila, an active eye!
I hope I was helpful, but you'll have to be more precise and come back when you have an actual problem. Read the FAQ as well.
http://scikit-learn.org/ is a great resource (and framework) for you to learn about machine learning in python.
If you want to build a neural network, http://pybrain.org/ is a good framework.
I am a neural-network beginner. I'd like to learn the basics of neural networks by teaching computers to play checkers. Actually, the games I want to learn are Domineering and Hex.
These games are pretty easy to store and the rules are much simpler than chess, but there aren't too many people who play. If I can get this idea off the ground it would be great for experimenting Combinatorial Game Theory.
PyBrain seems to be the clear winner for Python neural networks, but who can walk me through how to set up a neural net for my game-playing task? A google search turned up Blondie24 in 2001 but it uses some genetic algorithms - I don't want to complicate things.
Once you replace "neural networks" by machine learning (or even artificial intelligence, rather, imho) as the comments rightly suggest, I think you're better off starting with Alpha-beta pruning, the Minimax algorithm, and Branch and bound ideas.
Basically :
At each step, you build the tree of all possible futures, and evaluate leaf positions with an evaluation function (e.g. board domination, connectivity, material, etc.)
Propagate the results up in the tree, choosing the best play you can make, and the worse your opponent can (the best for him), until you know what move to play in the position you're at.
Rinse, repeat. Branch and bound saves you a lot of computation if you have a few good heuristics, and the level of your programm will basically be how deep it'll be able to search the game tree.
This will most probably be the basic framework in which anyone would introduce new ideas, so if you're not familiar with it, go for it :-)
As long as I've been a programmer I still have a very elementary level education in algorithms (because I'm self-taught). Perhaps there is a good beginner book on them that you could suggest in your answer.
As a general note, Introduction to Algorithms. That book will get you through pretty much everything you need to know about general algorithms.
Edit:
As AndrewF mentioned, it doesn't actually contain minimax specifically, but it's still a very good resource for learning to understand and implement algorithms.
Look at the wikipedia article on Negamax: http://en.wikipedia.org/wiki/Negamax. It's a slight simplification of minimax that's easier to implement. There's pseudocode on that page.
There is an implementation of minimax as part of an othello game here (and for browsers here).
Stepping through this with a debugger and/or through use of logging statements
may supplement theoretical descriptions of the algorithm.
This visualization applet may also help.
At each stage the player will choose the move which is best for himself. What's best for one player will be worst for the other player. So at one stage the game state with the minimum score will be chosen, and at the next stage the game state available with the maximum score will be chosen, etc.