Support Vector-based Predictive Model

A support vector-based predictive model is a predictive model (w/out probability estimates) that is based on support vectors.

• AKA: Fitted SVM Model.
• Context:
  • It can range from being:
    • a Support Vector Machine Classifier (e.g. a Support Vector Machine Binary Classifier.
    • a Support Vector Machine Ranker.
    • a Support Vector Machine Estimator.
  • It can be produced by a Support Vector Machine System (that implements an SVM algorithm).
• Example(s):
  • a Linear SVM Model.
  • ...
• Counter-Example(s):
  • a Decision Tree-based Predictive Model.
• See: Kernel Function, Relevance Vector Machine.

References

2004

• (Hastie et al., 2004) ⇒ Trevor Hastie, Saharon Rosset, Robert Tibshirani, and Ji Zhu. (2004). "The Entire Regularization Path for the Support Vector Machine." In: The Journal of Machine Learning Research, 5.
  • The support vector machine (SVM) is a widely used tool for classification. Many efficient implementations exist for fitting a two-class SVM model. ... We then derive an algorithm that can fit the entire path of SVM solutions for every value of the cost parameter, with essentially the same computational cost as fitting one SVM model.

• (Bishop, 2004) ⇒ Christopher M. Bishop. (2004). "Recent Advances in Bayesian Inference Techniques." Keynote Presentation at SIAM Conference on Data Mining.
  • Limitations of the SVM:
    • two classes
    • large number of kernels (in spite of sparsity)
    • kernels must satisfy Mercer criterion.
    • cross-validation to set parameters C (and ε)
    • decisions at outputs instead of probabilities

• Concept