cvrDocuClassifiers.h

/*
 * Copyright (C) 1998-2006
 * Lehrstuhl fuer Technische Informatik, RWTH-Aachen, Germany
 *
 *
 * This file is part of the Computer Vision and Robotics Library (CVR-Lib)
 *
 * The CVR-Lib is free software; you can redistribute it and/or
 * modify it under the terms of the BSD License.
 *
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are met:
 *
 * 1. Redistributions of source code must retain the above copyright notice,
 *    this list of conditions and the following disclaimer.
 *
 * 2. Redistributions in binary form must reproduce the above copyright notice,
 *    this list of conditions and the following disclaimer in the documentation
 *    and/or other materials provided with the distribution.
 *
 * 3. Neither the name of the authors nor the names of its contributors may be
 *    used to endorse or promote products derived from this software without
 *    specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
 * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
 * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
 * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
 * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
 * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
 * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
 * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 * POSSIBILITY OF SUCH DAMAGE.
 */



/*----------------------------------------------------------------
 * project ....: LTI Digital Image/Signal Processing Library
 * file .......: cvrDocuClassifiers.h
 * authors ....: Peter Doerfler
 * organization: LTI, RWTH Aachen
 * creation ...: 08.11.2002
 * revisions ..: $Id: cvrDocuClassifiers.h,v 1.3 2006/01/03 19:56:00 alvarado Exp $
 */

#ifndef _CVR_DOCUCLASSIFIERS_H_
#define _CVR_DOCUCLASSIFIERS_H_

/**
\page docuClassifiers How to use the Classifiers.

- \ref types
- \ref internalclasses
- \ref related
  - \ref viz
  - \ref classStat
  - \ref sammon
- \ref unsupervised
- \ref instance
- \ref sequence

\section types Types of Classifiers

All classifiers follow the same principle. First, they are trained
using some kind of training data. The trained classifier can then be
used to classify new data, i.e. to assign the same label to the new
data as to the most similar training data.

Three different categories of classifiers exist in the LTI-Lib:
- unsupervised classifiers
- supervised sequence classifiers
- supervised instance classifiers

The first category includes all algorithms that are trained without
knowing the proper labels or results for each training
pattern. Clustering algorithms are included here, although most are
not designed for later classifying new data. Supervised classifiers
are trained knowing the expected result for each training
pattern. They are divided in two groups: sequence classifiers work on
time series whereas instance classifiers work on n-dimensional
vectors.

All classifiers have four essential properties:

- The \b parameters are static properties of the classifier. All
  classifiers have default settings for the parameters
  object. However, many classifiers will not work properly without
  adjustment of some of the parameters.
- The \b status contains error messages. If any of the methods,
  particularly train and classify, return false. Information of the
  nature of the failure can be obtained by calling getStatusString().
- The \b progress of the training can be monitored with a
  progressObject. See \ref progress.
- The \b outputTemplate contains information on how internal results
  are mapped to an ouputVector which is the result of a
  classification. This member should only be changed by experienced
  users. Usually, the outputTemplate is set by the classifier itself
  during the training.

The results of the classification are returned in an ouputVector. This
object contains labels and corresponding recognition values. These can
often be interpreted as probabilities. Section \ref outvec gives
details about this data structure.

\section internalclasses Helper Classes

\subsection progress Progress Objects

The progressInfo object gives information on how many steps an
algorithm will take until finished and how many have already been
accomplished. Also the name of the classifier is usually given. The
following progress infos exist:

- cvr::streamProgressInfo writes ASCII into a stream. The default
  stream is the console output.

\subsection params Parameters

The parameters of the classifier class define an enumeration
eDistanceMeasure which is used to specify which distance is used in
the classifier. Options are the L1 and L2 distances.

See also cvr::classifier::parameters.

\subsection outvec Output Vector

The output vector is the result of a classification, i.e. calling the
classify method of a classifier. It assigns values to the labels. In
the case of supervised classifiers these labels where supplied by the
user during training. In case of unsupervised classification the
classifier usually assigns labels from 0 to C-1 with C the number of
classes found.

Output vectors can be the final result of a process. In this case they
are usually displayed by an application or used for statistical
analysis of the classification process. For the later the
classificationStatistics %functor can be used. It is also possible to
combine the results of several classifiers using the combination
%functor.

For further reading see the documentation of cvr::classifier::outputVector.

\section related Related Topics


\subsection viz Visualization of Classifiers and Data

\subsubsection vizData Visualizing Data

cvr::draw
cvr::epsDraw
cvr::draw3D
cvr::draw2DDistribution

\subsubsection vizClass Visualizing Classification Results

cvr::classifier2DVisualizer

\subsection classStat Classification Statistics

cvr::classificationStatistics

\subsection sammon Sammon's Mapping

Sammon's Mapping transforms points in n-dimensional space to points in
m-dimensional space while trying to preserve all distances between the
points. Usually, m will be 2 or 3 so that the points can be displayed
using one of the cvr::draw classes and an appropriate
cvr::viewer. Sammon's mapping can be very useful to get an idea of the
distribution of higher dimensional data without losing as much
information as when using e.g. cvr::principalComponents to reduce the
dimensionality.

However, the mapping is a very difficult task and might easily fail to
converge at a minimum. Check the error to get an idea about the performance. In case it is bad there are several options:
- try to reduce the amount of data (point). This can be done e.g. by
  using a cvr::clustering algorithm with the number of clusters a
  fraction of the number of data points, say a third. Then use the
  cluster centers as data points. This is an easy quantization method.
- reduce the number of dimensions a little e.g. to 10 with
  cvr::principalComponents or a feature selection method. Use the
  lower dimensional points for Sammon's mapping. Thus, at least you'll
  get the information from 10 dimensions.

For more information see cvr::sammonsMapping.

*/

/**
\page unsupervised Unsupervised Classifiers

Unsupervised classifiers are methods that try to find 'natural' or
'sensible' structure in data. To this end each data point is usually
member of a newly found group. A distinct label is assigned to each
group. Some algorithms allow membership of each point to more then one
group. One example is the \ref fcm.

The most popular application of unsupervised classifiers is data
analysis and possibly visualization. However, some methods are also
used for data compression or quantization.

This page first deals with clustering techniques. These are usually
statistical methods for finding groups (clusters). The second section
deals with an artificial neural network, the Self-Organizing Feature
Maps, which perform unsupervised classification.

- \ref cluster
  - \ref kMeans
  - \ref fcm
  - \ref adapt
- \ref som
  - \ref som2
  - \ref somViz

Related classes:
  - cvr::unsupervisedClassifier

\section cluster Clustering

- \ref kMeans
- \ref fcm
- \ref adapt

\subsection kMeans k-Means Clustering

cvr::kMeansClustering

\subsection fcm Fuzzy-C-Means Clustering

cvr::fuzzyCMeans

\subsection adapt Adaptive k-Means Clustering

cvr::adaptiveKMeans

\section som Self Organizing Feature Maps

cvr::SOFM

- \ref som2
- \ref somViz

\subsection som2 2D SOMs

cvr::SOFM2D

\subsection somViz Visualization of SOMs

Still to be done.

*/

/**
\page instance Supervised Instance Classifiers

- \ref stat
  - \ref svm
  - \ref shClass
  - \ref trees
- \ref ann
  - \ref mlp
  - \ref lvq
  - \ref rbf

\section stat Statistical Classifiers

- \ref svm
- \ref shClass
- \ref trees

\subsection svm Support Vector Machine

cvr::svm

\subsection shClass Sparse Histogram-based Classifier

cvr::shClassifier

\subsection trees Decision Trees

cvr::decisionTree

\section ann Artificial Neural Networks

- \ref mlp
- \ref lvq
- \ref rbf

\subsection mlp Multi Layer Perceptron

cvr::MLP

\subsection lvq Learning Vector Quantization

cvr::lvq

\subsection rbf Radial Basis Function Network

cvr::rbf

*/

/**
\page sequence Supervised Sequence Classifiers

cvr::hmmClassifier
*/


*/

#endif