Progress Report – April 2010

My main focus this month has been on researching in topics in reinforcement learning, non-deterministic programming, agent implementations, and finally Prolog and LISP programming techniques.  First my article in Q-learning shares my recent insights into Q-learning.  My purpose to gain a better understanding of Q-learning as to implement it in my block’s world environment. Read my blog entry on Q-learning.

Next, I have been studying Prolog techniques.  For example, in the Sterling and Shapiro book, I was reading about non-determinism in Prolog as a programming technique.  The generate and test approach to logic programming allows the generation of solution X and it gets tested.  Read more my blog entry in non-deterministic programming.

Progress Report – January 2010

In this month I wrote blog entries for Agent Design and EASSS09.  The agent design entry questions on the appropriate programming language to develop an agent and its environment.  I concluded that Java is an appropriate programming language.  Java has become a popular programming language for teaching computer science in academia, and industry has accepted Java as the appropriate programming language for developing and implementing web based applications as well as batch processing.

The EASSS09 entry refers to the European Agent Systems Summer School for learning agents.  The material that I have reviewed thus far has been excellent due to the broaden view of agent and multi-agent design.  The instructors provide excellent materials and references for the courses.

EASSS09

In this past summer in Turino, Italy, a summer school for Agent Systems was offered.  I reached this site when I was searching for other Google AI groups to join and found the news group for the European Agent System Laboratory.  The group had a list of recommended books for students to get in the study of Agents and Multi-Agent Systems.  Also, the site made a reference to the European Agent Systems Summer School.  I downloaded the course notes from the site with some interesting material.  The material was very excellent, and it is a worthwhile material to review to learn more about agent and multi-agent systems.

Agent Design

As I was surfing the web for articles with Google Scholar, I encounter a course syllabus regarding Designing Intelligent Agents using Java.  My first question was Java the appropriate choice for creating software agents.  Does Java offer certain advantages over the traditional programming languages such as Prolog and LISP?  Based on my experience with creating an agent using Prolog, the main disadvantage with Prolog is the lack of object oriented abstraction.  I had to take a structure and using the univ operator convert the structure into a list, which in my opinion is major performance hit.  Although LISP with the CLOS has the object oriented features, it lacks a multi-threading option for each software agents to run in its own thread.  Of course my other options are procedural languages of C++ and Java.  The C++ language requires the use of thread library that is not platform independent.  Hence, the natural choice is Java.  The down side of these procedural languages is lack of the power offered  by LISP and Prolog.  At this point I will rewrite my blocks world planner agent in Java.

Progress Report – November 2009

The month of November was very productive.  I began the month reviewing the RRL paper.  From there I reviewed from [Luger and Stubblefield 1993] the blocks world and its Prolog version of the blocks world planner.  Next step was to take the algorithm in Chapter 2, Figure 2.14 from [Russell and Norvig 1995], and to create an agent frame work in Prolog.  Upon review of [Covington, Nute, and Vellino 1997], I was able to apply advanced prolog tips to the agent frame work.  My initial objective is to have a functional blocks world planner in an agent frame work.  The basic planner agent has been created and returns an action.  However, the planner agent needs to generate a plan and return each action from the plan.  This needs further development; in other words, a work in progress.  The final blocks world planner agent will be posted upon completion.

Agent Framework

As part of my ongoing agent training, I re-read the agent environment algorithm in Chapter 2 of AIMA (Russell and Norvig, 1995). The key challenge is how to convert an algorithm with repeat and for loops into viable Prolog code. At first inspection, the implementation is best suited for a language with control loops built into the programming language (e.g., LISP). My primary Prolog books [Bratko 1990] and [Sterling and Shapiro 1994] did not provide the required help. However, upon rereading Prolog Programming In Depth (Covington, Nute, and Vellino 1997), I was able to figure out how to write loops.

Progress Report – July 2009

In this progress report, I attended the ICAI 09 conference in Las Vegas (the IJCAI 2009 is located in Pasadena).  I presented my work on A Brute Force Approach to Solving the Knights Tour Using Prolog.  One thing that I learned is the participants are proud to share their work with others.  Another lesson is to make your presentation within the time allotted.  A few presenters did not finish their presentations in the 20 minutes time slot.  I rehearsed my presentation a few times to have the timing within 20 minutes.  I also learned that I need to get more training with MS Power Point.

I am also continue to view the video lectures from the Summer Schools in Logic and Learning from Video Lectures.  Thus, I have completed the Intelligent Agents video lectures by John Lloyd. 

AIMA Part VI

Part VI of the AIMA book covers learning and various techniques on agents can learn.  The material covers Learning from Observations, Neural Networks, Reinforcement Learning, and Knowledge in Learning.  In the second edition, the chapter in Neural Networks is replaced with Statistical Learning Methods.

I read Chapter 18 of the AIMA book called Learning from Observations.  The Chapter focuses on decision trees and decision lists as some computational learning theory.  The main classification algorithm is the ID3 algorithm for classifying examples and generating decision trees.  The ID3 algorithm is based on Hunt’s Concept Learning System (CLS).  The ID3 algorithm uses information theory for obtaining the decision tree covering most the examples.  I have studied the ID3 algorithm quite extensively since the original application was to classify the winning position in the king and rook versus the king and knight endgame (Quinlan 1983).

Progress Report – May 2009

This month I submitted my paper A Brute Force Approach to Solving the Knight’s Tour using Prolog to the ICAI ’09 conference, located at the Monte Carlo in Las Vegas, Nevada.  According to the conference chair, the conference acceptance rate was 27 percent.  This conference is in conflict the IJCAI conference in Pasadena, California.

Nevertheless, besides my research paper, much of my activity this month has been around logic programming and inductive logic programming (ILP).  I ordered from Amazon several books recently around this area, in particular The Art of Prolog and Inductive Logic Programming: From Machine Learning to Software Engineering.  The Art of Prolog has a gentle introduction to logic programming – it is very clear and instructive.  From this book I was able to understand early articles in Logic Programming.

Progress Report – December 2008

The month began by reviewing my college Probability and Statistics book (Devore 1991).  I needed to review standard probability concepts mentioned in articles and the Stanford machine learning class.  For example, the article Learning with a Mixture of Trees (Meila and Jordan 2000) the authors explain the new algorithm by using Joint Probabilities and probabilistic mathematics to describe the algorithm – a similar used by Dr. Ng in his machine learning lectures.