MML Decision Trees and Decision Graphs

Below is a list of publications pertaining to Minimum Message Length (MML) decision trees and decision graphs. Among other things, these papers show how to cost a decision tree and/or a decision graph using MML.

[See also Ray Solomonoff (1926-2009) 85th memorial conference (Wedn 30 Nov - Fri 2 Dec 2011), 1st Call for Papers.]

Publications:
[The papers below are requestable in printed hard copy from either [more reliable] writing a letter to my ``snail mail'' postal address {see my home page, http://www.csse.monash.edu.au/~dld} or [perhaps less reliable] e-mail to enquiries At cs.monash.edu.au (if you send your name and postal address and make it clear what you want).]

Wallace, C.S. and J.D. Patrick (1993). Coding Decision Trees, Machine Learning 11, 7-22, 1993.

J J Oliver and C. S. Wallace (1991). Inferring decision graphs, International Joint Conference on Artificial Intelligence (IJCAI) '91 Workshop 8, Sydney, 1991.

J J Oliver, D L Dowe and C. S. Wallace (1992). Inferring decision graphs using the minimum message length principle, Proc. 5th Australian joint conf. on Artificial Intelligence, Hobart, pp361-367, November 1992. [p361, p362, p363, p364, p365, p366, p367] (50% acceptance rate)

J J Oliver (1993). Decision graphs - an extension of decision trees, Proc. 4th Int. Conf. on Artificial Intelligence and Statistics, Fort Lauderdale, Florida, USA, 343-50, 1993.

D L Dowe and C S Wallace (1998). Kolmogorov complexity, minimum message length and inverse learning, abstract, page 144, 14th Australian Statistical Conference (ASC-14), Gold Coast, Qld, 6 - 10 July 1998.

Wallace, C.S. and D L Dowe (1999a). Minimum Message Length and Kolmogorov Complexity, Computer Journal (special issue on Kolmogorov complexity), Vol. 42, No. 4, pp270-283 - http://comjnl.oxfordjournals.org/cgi/reprint/42/4 /270 and abstract.
[** This article is the Computer Journal's most downloaded ``full text as .pdf''. See, e.g., the Computer Journal, Editorial, vol. 48, no. 4 (2005), p381 (http://comjnl.oxfordjournals.org/cgi/reprint/48/4/381). **]

Needham, S.L. and D.L. Dowe (2001). Message Length as an Effective Ockham's Razor in Decision Tree Induction. Proc. 8th International Workshop on Artificial Intelligence and Statistics (AI+STATS 2001), pp253-260, Key West, Florida, U.S.A., Jan. 2001. www.csse.monash.edu.au/~dld/Publications/2001/Needham+Dowe2001_Ockham.ps (was also once at http://www.ai.mit.edu/conferences/aistats2001/files/needham122.ps) http://www.csse.monash.edu.au/~dld/Publications/2001/Needham+Dowe2001.ref

Tan, P.J. and D.L. Dowe (2002). MML Inference of Decision Graphs with Multi-Way Joins, Proc. 15th Australian Joint Conference on Artificial Intelligence, Canberra, Australia, 2-6 Dec. 2002, Published in Lecture Notes in Artificial Intelligence (LNAI) 2557, Springer-Verlag, pp131-142. (This paper is requestable from ptan at csse.monash dot edu.au or downloadable from http://www.csse.monash.edu.au/~dld/Publications/2002/Tan+Dowe2002_MMLDecisionGraphs.pdf www.csse.monash.edu.au/~dld/Publications/2002/Tan+Dowe2002.ref .)

Comley, Joshua W. and D L Dowe (2003). General Bayesian Networks and Asymmetric Languages, Proc. 2nd Hawaii International Conference on Statistics and Related Fields, 5-8 June, 2003. % {This concerns all of Generalised Bayesian nets, MML Bayesian nets, and
% MML Bayesian networks (or Generalised Bayes nets, MML Bayes nets, and
% MML Bayes networks (or Minimum Message Length Bayes nets and
% Minimum Message Length Bayes networks) or even mixed Bayes nets or
% mixed Bayesian nets or mixed Bayesian nets or mixed Bayesian networks)
% (or Generalised graphical models or MML graphical models, or
% Generalised directed graphical models or MML directed graphical models,
% or even mixed graphical models or MML mixed graphical models, or
% mixed directed graphical models or MML mixed directed graphical models,
% or MML Bayesian belief nets, or MML Bayesian belief networks);
% *and* deals with a mix of both continuous and discrete variables
% (or a mix of both numeric and symbolic variables)
% (or a mix of both numerical and categorical variables).
% There are decision trees (possibly with their own internal nodes) in the nodes of these Bayesian nets.}
http://www.csse.monash.edu.au/~dld/Publications/2003/Comley+Dowe03_HICS2003.ref http://www.csse.monash.edu.au/~dld/Publications/2003/Comley+Dowe03_HICS2003_GeneralBayesianNetworksAsymmetricLanguages.pdf http://www.hicstatistics.org/2003StatsProceedings/Joshua Comley.pdf

P. J. Tan and D. L. Dowe (2003). MML Inference of Decision Graphs with Multi-Way Joins and Dynamic Attributes, Proc. 16th Australian Joint Conference on Artificial Intelligence (AI'03), Perth, Australia, 3-5 Dec. 2003, Published in Lecture Notes in Artificial Intelligence (LNAI) 2903, Springer-Verlag, pp269-281. http://www.csse.monash.edu.au/~dld/Publications/2003/Tan+Dowe2003_MMLDecisionGraphs.pdf http://www.csse.monash.edu.au/~dld/Publications/2003/Tan+Dowe2003.ref

P. J. Tan and D. L. Dowe (2004). MML Inference of Oblique Decision Trees, Proc. 17th Australian Joint Conference on Artificial Intelligence (AI'04), Cairns, Qld., Australia, Dec. 2004, Lecture Notes in Artificial Intelligence (LNAI) 3339, Springer-Verlag, pp1082-1088. http://www.csse.monash.edu.au/~dld/Publications/2004/Tan+Dowe2004.ref www.csse.monash.edu.au/~dld/Publications/2004/Tan+DoweAI2004.ps www.csse.monash.edu.au/~dld/Publications/2004/Tan+DoweAI2004.pdf
% Among other things, this gives a reasonable initial model of MML SVMs (MML SVM)
% (Minimum Message Length Support Vector Machines, Minimum Message Length SVM, MML Support Vector Machines).
% It puts the probabilistic support vector machines in the leaf nodes of the decision trees.

Comley, Joshua W. and D L Dowe (2005). Minimum Message Length and Generalized Bayesian Nets with Asymmetric Languages, Chapter 11 (pp265-294) in P. Gru:nwald, I. J. Myung and M. A. Pitt (eds.), Advances in Minimum Description Length: Theory and Applications, M.I.T. Press (MIT Press), April 2005, ISBN 0-262-07262-9. [Final camera ready copy was submitted in October 2003.] pp265-284 and pp285-294; p265, p266, p267, p268, p269, p270, p271, p272, p273, p274, p275, p276, p277, p278, p279, p280, p281, p282, p283, p284, p285, p286, p287, p288, p289, p290, p291, p292, p293, p294
[The paper was originally submitted with the title: ``Minimum Message Length, MDL and Generalised Bayesian Networks with Asymmetric Languages'', but this was (unfortunately?) changed to ``Minimum Message Length and Generalized Bayesian Nets with Asymmetric Languages'']
http://mitpress.mit.edu/catalog/item/default.asp?sid=4C100C6F-2255-40FF-A2ED-02FC49FEBE7C&ttype=2&tid=10478 Table of contents is at www.mitpress.mit.edu/catalog/item/default.asp?sid=C89E957F-2B61-42E0-9438-842E83E534BF&ttype=2&tid=10478&mode=toc

% {This concerns all of Generalised Bayesian nets, MML Bayesian nets, and
% MML Bayesian networks (or Generalised Bayes nets, MML Bayes nets, and
% MML Bayes net, MML Bayes networks (or Minimum Message Length Bayes nets
% and Minimum Message Length Bayes networks) or even mixed Bayes nets or
% mixed Bayesian nets or mixed Bayesian nets or mixed Bayesian networks)
% (or Generalised graphical models or MML graphical models, or
% Generalised directed graphical models or MML directed graphical models,
% or even mixed graphical models or MML mixed graphical models, or
% mixed directed graphical models or MML mixed directed graphical models,
% or MML Bayesian belief nets, or MML Bayesian belief networks);
% *and* deals with a mix of both continuous and discrete variables
% (or a mix of both numeric and symbolic variables)
% (or a mix of both numerical and categorical variables).
% There are decision trees in the internal nodes of these Bayesian net s.}

Book: Wallace, C.S. (2005) [posthumous], Statistical and Inductive Inference by Minimum Message Length, Springer (Series: Information Science and Statistics), 2005, XVI, 432 pp., 22 illus., Hardcover, ISBN: 0-387-23795-X. (Link to table of contents, chapter headings and more.)

Tan, P.J. and D. L. Dowe (2006). "Decision Forests with Oblique Decision Trees" (and .pdf here), [Proc. 5th Mexican International Conference on Artificial Intelligence (MICAI 2006),] Lecture Notes in Artificial Intelligence (LNAI) 4293, pp593-603, Springer. [Best MICAI 2006 student paper, equal best paper overall.]

Tan, P.J., D. L. Dowe and T. I. Dix (2007). "Building classification models from microarray data with tree-based classification algorithms", in press, Proc. 20th Australian Joint Conference on Artificial Intelligence, Gold Coast, Queensland, December 2007.

Links to (links to) some of David Dowe's publications from some of the following years: 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007.

Other links - links to pages on:

Minimum message length (MML),

Chris Wallace (1933-2004) (developer of MML in 1968),

Bayesian Net s using Minimum message length (MML) [with optional decision trees in internal nodes],

clustering and mixture modelling,

comparisons between MML and the subsequent Minimum Description Length (MDL) principle,

data links,

Occam's razor (Ockham's razor),

Snob (program for MML clustering and mixture modelling),

(econometric) time series using MML,

medical research,

a probabilistic sports prediction competition (and further reading on probabilistic scoring),

chess and game theory research,

do-goody stuff and saving the planet.

This page, http://www.csse.monash.edu.au/~dld/MMLDecisionTree.html, was last updated no earlier than 2005.

Copyright David L. Dowe, Monash University, Australia, 2005, etc.
Copying is not permitted without expressed permission from David L. Dowe. This WWW page is http://www.cs.monash.edu.au/~dld/MMLDecisionTree.html , and was last updated no earlier than Wed 2nd Nov 2005. E-mail David Dowe, d l d at cs.monash dot edu.au, for more information. WWW (URL): http://www.cs.monash.edu.au/~dld/ . Copyright Dr David L. Dowe, School of Computer Science and Softw. Engineering, Monash University, Clayton, Vic. 3168, Australia; 2 Nov 2005, etc.