Machine Learning with ID3 Decision Trees in Ruby
Introduction to ID3 algorithm

AI4R implements the ID3 algorithm (Quinlan) as one of its automatic classifiers. Given a set of preclassified examples, it builds a top-down induction of decision tree, biased by the information gain and entropy measure.

The good thing about this automatic learning method is that humans learns as well. Unlike other AI techniques like neural networks, classifiers can generate ruby code with if / else sentences. You can use this to evaluate parameters on realtime, copy paste them in a code, or just read them to learn about your problem domain.

Marketing target strategy example using ID3 Decision Trees in Ruby

Let's suppose that you are writting an application that must identify people as relevant marketing targets or not. The only information that you have is a collection of examples, provided by a marketing survey:

80', 'F', 'Y'] ] ]]>

You can create an ID3 Decision tree to do the dirty job for you:

The Decision tree will automatically create the "rules" to parse new data, and identify new posible marketing targets:

if age_range=='<30' then marketing_target='Y' elsif age_range=='[30-50)' and city=='Chicago' then marketing_target='Y' elsif age_range=='[30-50)' and city=='New York' then marketing_target='N' elsif age_range=='[50-80]' then marketing_target='N' elsif age_range=='>80' then marketing_target='Y' else raise 'There was not enough information during training to do a proper induction for this data element' end id3.eval(['New York', '<30', 'M']) # => 'Y' ]]>
Better data loading

In real life you will use many more data training examples, with more attributes. Consider moving your data to an external CSV (comma separate values) file.

A good tip for data evaluation

The ID3 class provides a method to evaluate new data.

'Y' ]]>

But instead of going through the tree every time, you can take advantage of the fact that the method "get_rules" generates proper ruby code!

'Y' ]]>
More about ID3 and decision trees

Wikipedia article on Decision trees Wikipedia article on ID3 Algorithm