Progress Report – February 2010

For this month, I continued to complete books that I started.  As a result, I continued to complete the book called An Introduction to Bayesian Inference and Decision.  I finished the Chapter 2 exercises and continued into Chapter 3, completing 18 of 60 exercises.  The material from Chapter 3 starts information about discrete random variables, expectation, variance, probability mass function, continuous distribution function, joint probability distributions, conditional probabilities and the Laws of Expectation.  The next section introduces terms such as prior probabilities, likelihood, and posterior probabilities as an introduction to bayesian inference.  Basically Bayes Theorem is mechanism to update probabilities when new information is available.  As additional information or samples are collected, then prior probability distributions are updated to generate revised probability distributions.

Progress Report – January 2010

In this month I wrote blog entries for Agent Design and EASSS09.  The agent design entry questions on the appropriate programming language to develop an agent and its environment.  I concluded that Java is an appropriate programming language.  Java has become a popular programming language for teaching computer science in academia, and industry has accepted Java as the appropriate programming language for developing and implementing web based applications as well as batch processing.

The EASSS09 entry refers to the European Agent Systems Summer School for learning agents.  The material that I have reviewed thus far has been excellent due to the broaden view of agent and multi-agent design.  The instructors provide excellent materials and references for the courses.

Progress Report – October 2009

In this month, I continued my studies in Relational Reinforcement Learning by reviewing the article Towards Informed Reinforcement Learning from the proceedings of the 2004 Machine Learning workshop of Relational Reinforcement Learning.   Basically the articles summarizes that an agent with limited information can find an optimal policy and can achieve a goal or goal states with limited information about its environment.  The experiments reported seems to suggest this type of exploration is possible.  According to Google Scholar search, there are 11 subsequent articles that reference this one.  In the RRL arena, my goal is to repeat the block’s world experiment as reported in Relational Reinforcement Learning article by Dzeroski, De Raedt, and Blockeel.

Progress Report – September 2009

This month marks my first year in utilizing wordpress to host my blog.  It has been a great journey so far and hopefully will be better this coming year.

The month of September has been busy.  Unfortunately I did not write any blog entries due to my busy schedule.  However, from statistical point of view, my blog had the second highest total of number of views (362 in total).  Also, I covered various topics this month with Probability and Statistics, Bayesian Inference, Reinforcement Learning, and finally a LISP review.

I started to review the Reinforcement Learning book (Sutton and Barto 1997) only reading Part 1 and part of Part 2.  I did download the lisp code associated with the book.  I ran the tic-tac-toe program in my test environment.