another QP form of SVM with valuable geometric messages and almost no dependence on the dimension of transformation

kernel as a shortcut to (transform + inner product): allowing a spectrum of models ranging from simple linear ones to infinite dimensional ones with margin control

a new primal formulation that allows some penalized margin violations, which is equivalent to a dual formulation with upper-bounded variables

soft-classification by an SVM-like sparse model using two-level learning, or by a "kernelized" logistic regression model using representer theorem

kernel ridge regression via ridge regression + representer theorem, or support vector regression via regularized tube error + Lagrange dual

blending known diverse hypotheses uniformly, linearly, or even non-linearly; obtaining diverse hypotheses from bootstrapped data

"optimal" re-weighting for diverse hypotheses and adaptive linear aggregation to boost weak algorithms

recursive branching (purification) for conditional aggregation of simple hypotheses

bootstrap aggregation of randomized decision trees with automatic validation

aggregating trees from functional + steepest gradient descent subject to any error measure

automatic feature extraction from layers of neurons with the back-propagation technique for stochastic gradient descent

an early and simple deep learning model that pre-trains with denoising autoencoder and fine-tunes with back-propagation

linear aggregation of distance-based similarities to prototypes found by clustering

linear models of items on extracted user features (or vice versa) jointly optimized with stochastic gradient descent for recommender systems

summary from the angles of feature exploitation, error optimization, and overfitting elimination towards practical use cases of machine learning