Progress Report – October 2009

In this month, I continued my studies in Relational Reinforcement Learning by reviewing the article Towards Informed Reinforcement Learning from the proceedings of the 2004 Machine Learning workshop of Relational Reinforcement Learning.   Basically the articles summarizes that an agent with limited information can find an optimal policy and can achieve a goal or goal states with limited information about its environment.  The experiments reported seems to suggest this type of exploration is possible.  According to Google Scholar search, there are 11 subsequent articles that reference this one.  In the RRL arena, my goal is to repeat the block’s world experiment as reported in Relational Reinforcement Learning article by Dzeroski, De Raedt, and Blockeel.

Progress Report – June 2008

In this report, I focused on my presentation for the upcoming ICAI ‘09 conference in Las Vegas.  It has currently 18 slides.

I am also viewing the video lectures from the Summer Schools in Logic and Learning from Video Lectures dot Net.  Thus, I have completed the Introduction to Logic video lectures.

I was also reading Michalski’s article on A Theory and Methodology of Induction Learning. Basically it was an article describing the induction process and some the issues associated with induction such as descriptive language, background information, examples, generalization, and hypothesis space.  All this sounded familar when I was reading the two books on ILP.  Induction is a common theme with the two books.  Various familar algorithms are discussed amongst this material.  It appears there are no easy answers in the induction of rules from examples.

Progress Report – May 2009

This month I submitted my paper A Brute Force Approach to Solving the Knight’s Tour using Prolog to the ICAI ‘09 conference, located at the Monte Carlo in Las Vegas, Nevada.  According to the conference chair, the conference acceptance rate was 27 percent.  This conference is in conflict the IJCAI conference in Pasadena, California.

Nevertheless, besides my research paper, much of my activity this month has been around logic programming and inductive logic programming (ILP).  I ordered from Amazon several books recently around this area, in particular The Art of Prolog and Inductive Logic Programming: From Machine Learning to Software Engineering.  The Art of Prolog has a gentle introduction to logic programming – it is very clear and instructive.  From this book I was able to understand early articles in Logic Programming.

Wumpus World Revisited

I have noticed there has been many searches for the wumpus world problem.  The LISP version is located at the AIMA website.  The Prolog version by Professor Larry Holder is located at this URL, which is written for Quintus Prolog.  Both versions are based on the analysis by Stuart and Norvig’s book.

Wumpus World

I have been getting many searches for the Wumpus World code.  The LISP version is located at the AIMA website.

For the java version, goto the AIMA Java Code web page.

There are prolog versions of the Wumpus World as well.

Progress Report – December 2008

The month began by reviewing my college Probability and Statistics book (Devore 1991).  I needed to review standard probability concepts mentioned in articles and the Stanford machine learning class.  For example, the article Learning with a Mixture of Trees (Meila and Jordan 2000) the authors explain the new algorithm by using Joint Probabilities and probabilistic mathematics to describe the algorithm – a similar used by Dr. Ng in his machine learning lectures.

WISE Compiled

After fixing the return of thread status call from a boolean to the thread.state type, inspite of the other 60 coding warnings, I successfully started the WISE software, and exited.

I will explore the User’s manual further on setting up a trial run.

WISE

This week I downloaded the WISE software from University of Texas, Arlington (UTA). According to the build instructions it was developed using Java 1.3.0; however, that is not available and the latest is Java v1.6 u11. After installing the JDK in my Linux environment, I started to compile the program ran into 73 warnings and 1 one error. Two changes are to the dialog box show() and hide() functions; they were replaced by setVisible(true) and setVisible(false), respectively. There are other errors, but I am still reviewing them.

AIMA Book

I was reading Chapters 6 and 7 of the AIMA book (Russell, Norvig 1995). They describe propositional calculus, which from their perspective is a weak form logic. The authors start with the basics and build on the fundamentals by demonstrating the language using Backus Naur Form (BNF).  They used the Wumpus World as an example and demostrated that it will take at 6400 propositional logic statements to adequately describe the 4×4 Wumpus World.

In the next Chapter, the authors discuss predicate calculus and show its BNF.  The main focus is on first order predicate calculus and then applying it to the Wumpus World example.  They discuss the frame problem and situation calculus.  They used predicate calculus to demonstrate the differences and static components between two situations, which includes the frame axioms.

The discussion was neatly tied together between a problem and predicate calculus unlike the Luger book that I read earlier.  In the Luger the frame problem was used to illustrate robotic planning in the block’s world.

My goals are to run the AIMA wumpus world lisp code and complete the predicate calculus of the Wumpus world problem.