﻿ 基于特权信息的SVM模型研究及应用 Research and Application of SVM Model Based on Privileged Information

Advances in Applied Mathematics
Vol.06 No.09(2017), Article ID:23258,7 pages
10.12677/AAM.2017.69150

Research and Application of SVM Model Based on Privileged Information

Ying Cao, Fenyan Wang, Changjing Lu

School of Mathematics and Physics, China University of Geosciences, Wuhan Hubei

Received: Dec. 5th, 2017; accepted: Dec. 22nd, 2017; published: Dec. 29th, 2017

ABSTRACT

When using Support Vector Machine (SVM) to training classification model, we may encounter additional information in training samples. Because it will seriously affect classification accuracy of test samples, it really can’t be ignored. In this paper, a SVM model based on privilege information is proposed, which contains many cases of privilege information and can solve many kinds of distribution problems effectively. This paper first described the basic principles of SVM classification, and then it introduced the concept of privilege information. Next this paper proposed a SVM model based on privilege information and gave its special cases. Then, the application of SVM model based on privilege information was introduced. Finally, the existing problems and the development direction of this research were summarized.

Keywords:Privilege Information, Support Vector Machine (SVM), Application

Copyright © 2017 by authors and Hans Publishers Inc.

1. 引言

2. 支持向量机(SVM)分类原理

SVM对数据的学习是把分类问题转化为一个有约束的二次规划问题进行求解，得到最优解以构造最优决策分类超平面来实现分类 [6] 。以二分类样本集的学习问题为例，已知样本集为 $\left\{\left({x}_{i},{y}_{i}\right),i=1,2,\cdots ,l\right\}$ ，其中 ${x}_{i}\in {R}^{l}$ 表示输入变量， ${y}_{i}\in \left\{-1,1\right\}$ 表示输出变量， $l$ 为样本个数。则得到SVM模型为

$\begin{array}{l}\underset{\omega ,b,\xi }{\mathrm{min}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\frac{1}{2}{‖w‖}^{2}+C\underset{i=1}{\overset{l}{\sum }}{\xi }_{i}\\ \text{ }\text{s}\text{.t}\text{.}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left(\left(w\cdot \varphi \left({x}_{i}\right)\right)+b\right)\ge 1-{\xi }_{i}\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{\xi }_{i}\ge 0,i=1,\cdots ,l\end{array}$

$\begin{array}{l}\underset{\alpha }{\mathrm{max}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\underset{i=1}{\overset{l}{\sum }}{\alpha }_{i}-\frac{1}{2}\underset{i=1}{\overset{l}{\sum }}\underset{j=1}{\overset{l}{\sum }}{\alpha }_{i}{\alpha }_{j}{y}_{i}{y}_{j}K\left({x}_{i},{x}_{j}\right)\\ \text{\hspace{0.17em}}\text{s}\text{.t}\text{.}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\underset{i=1}{\overset{l}{\sum }}{y}_{i}{\alpha }_{i}=0\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }0\le {\alpha }_{i}\le C,i=1,\cdots ,l\end{array}$

${b}^{\ast }={y}_{j}-\underset{i=1}{\overset{l}{\sum }}{y}_{i}{\alpha }_{i}^{\ast }K\left({x}_{i},{x}_{j}\right),\forall j\in \left\{j|{\alpha }_{j}^{\ast }>0\right\}$

$f\left(x\right)=\mathrm{sgn}\left\{\underset{i=1}{\overset{l}{\sum }}{\alpha }_{i}^{\ast }{y}_{i}K\left({x}_{i},x\right)+{b}^{\ast }\right\}$

3. 具有特权信息的SVM模型

3.1. 基于特权信息的监督学习

3.2. 具有特权信息的SVM模型及其特例

3.2.1. 具有特权信息的SVM模型

(1) 其中 $n\left(n\in {N}^{+}\right)$ 个样本无特权信息，其余 $\left(l-n\right)$ 个样本具有特权信息；

(2) 具有特权信息的 $\left(l-n\right)$ 个样本来自于 $t\left(t\in {N}^{+}\right)$ 个不同的特权空间。

$\begin{array}{l}\underset{w,{w}^{1},\cdots ,{w}^{t},b,{b}^{1},\cdots ,{b}^{t},\xi ,{\xi }^{1},\cdots ,{\xi }^{t}}{\mathrm{min}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\frac{1}{2}\left[\left(w\cdot w\right)+\gamma \left(\left({w}^{1}\cdot {w}^{1}\right)+\cdot \cdot \cdot +\left({w}^{j}\cdot {w}^{j}\right)+\cdot \cdot \cdot +\left({w}^{t}\cdot {w}^{t}\right)\right)\right]+C\underset{i=1}{\overset{n}{\sum }}{\xi }_{i}\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }+{C}_{1}\underset{n+1}{\overset{m}{\sum }}\left[\left({w}^{1}\cdot {z}_{i}^{1}\right)+{b}^{1}\right]+{\theta }_{1}{C}_{1}\underset{i=n+1}{\overset{m}{\sum }}{\xi }_{i}^{1}+\cdot \cdot \cdot +{C}_{j}\underset{A}{\overset{B}{\sum }}\left[\left({w}^{j}\cdot {z}_{i}^{j}\right)+{b}^{j}\right]\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }+{\theta }_{j}{C}_{j}\underset{i=A}{\overset{B}{\sum }}{\xi }_{i}^{j}+\cdot \cdot \cdot +{C}_{k}\underset{d}{\overset{l}{\sum }}\left[\left({w}^{t}\cdot {z}_{i}^{t}\right)+{b}^{t}\right]+{\theta }_{k}{C}_{k}\underset{i=d}{\overset{l}{\sum }}{\xi }_{i}^{t}\end{array}$

$\begin{array}{l}\text{ }\text{ }\text{s}\text{.t}\text{.}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left[\left(w\cdot {z}_{i}\right)+b\right]\ge 1-{\xi }_{i},i=1,\cdot \cdot \cdot ,n\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left[\left(w\cdot {z}_{i}\right)+b\right]\ge 1-\left[\left({w}^{j}\cdot {z}_{i}^{j}\right)+{b}^{j}+{\xi }_{i}^{j}\right],j=1,2,\cdot \cdot \cdot ,t\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }{\xi }_{i}\ge 0,i=1,\cdot \cdot \cdot ,n\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }{\xi }_{i}^{1},{\xi }_{i}^{2},\cdot \cdot \cdot ,{\xi }_{i}^{t}\ge 0\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\left({w}^{j}\cdot {z}_{i}^{j}\right)+{b}^{j}\ge 0,j=1,2,\cdot \cdot \cdot ,t\end{array}$

$\begin{array}{l}\underset{\alpha ,\beta }{\mathrm{max}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\underset{i=1}{\overset{l}{\sum }}{\alpha }_{i}-\frac{1}{2}\underset{i,k=1}{\overset{l}{\sum }}{\alpha }_{i}{\alpha }_{k}{y}_{i}{y}_{k}K\left({x}_{i},{x}_{k}\right)\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }-\frac{1}{2\gamma }\underset{i,k=n+1}{\overset{m}{\sum }}\left({\alpha }_{i}+{\beta }_{i}-{C}_{1}\right)\left({\alpha }_{k}+{\beta }_{k}-{C}_{1}\right){K}^{1}\left({x}_{i}^{1},{x}_{k}^{1}\right)\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }-\cdot \cdot \cdot -\frac{1}{2\gamma }\underset{i,k=A}{\overset{B}{\sum }}\left({\alpha }_{i}+{\beta }_{i}-{C}_{j}\right)\left({\alpha }_{k}+{\beta }_{k}-{C}_{j}\right){K}^{j}\left({x}_{i}^{j},{x}_{k}^{j}\right)\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\cdot -\cdot \cdot -\frac{1}{2\gamma }\underset{i,k=d}{\overset{l}{\sum }}\left({\alpha }_{i}+{\beta }_{i}-{C}_{t}\right)\left({\alpha }_{k}+{\beta }_{k}-{C}_{t}\right){K}^{t}\left({x}_{k}^{t},{x}_{k}^{t}\right)\end{array}$

$\begin{array}{l}\text{s}\text{.t}\text{.}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\underset{i=1}{\overset{l}{\sum }}{\alpha }_{i}{y}_{i}=0\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\underset{i=1}{\overset{n}{\sum }}\left({\alpha }_{i}+{\beta }_{i}-C\right)=0\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\underset{i=n+1}{\overset{l}{\sum }}\left({\alpha }_{j}+{\beta }_{j}-{C}_{j}\right)=0,j=1,\cdot \cdot \cdot ,t;\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }0\le {\alpha }_{i}\le {\theta }_{j}{C}_{j},i=1,\cdot \cdot \cdot ,l;j=1,\cdot \cdot \cdot ,t\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{\alpha }_{i}\ge 0,{\beta }_{i}\ge 0\end{array}$

$f\left(x\right)=\left(w\cdot z\right)+b=\underset{i=1}{\overset{l}{\sum }}{\alpha }_{i}{y}_{i}K\left({x}_{i},x\right)$

3.2.2. 具有特权信息的SVM模型特例

1) 全部样本存在特权信息的SVM模型

$\begin{array}{l}\underset{\omega ,{\omega }^{\ast },b,{b}^{\ast }}{\mathrm{min}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\frac{1}{2}\left[\left(w\cdot w\right)+\gamma \left({w}^{\ast }\cdot {w}^{\ast }\right)\right]+C\underset{i=1}{\overset{l}{\sum }}\left[\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\right]\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{s}\text{.t}.\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left[\left(w\cdot {z}_{i}\right)+b\right]\ge 1-\left[\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\right],i=1,\cdots ,l\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\ge 0,i=1,\cdots ,l\end{array}$

2) 全部样本存在特权信息且松弛变量改动的SVM模型

${\xi }_{i}=\left[\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\right]+{\xi }_{i}^{\ast },i=1,\cdots ,l$

$\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\ge 0,{\xi }_{i}^{\ast }\ge 0,i=1,\cdots ,l$

$\begin{array}{l}\underset{\omega ,{\omega }^{\ast },b,{b}^{\ast },{\xi }^{\ast }}{\mathrm{min}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\frac{1}{2}\left[\left(w\cdot w\right)+\gamma \left({w}^{\ast }\cdot {w}^{\ast }\right)\right]+C\underset{i=1}{\overset{l}{\sum }}\left[\left({w}^{\ast }\cdot {z}^{\ast }\right)+{b}^{\ast }\right]+\theta C\underset{i=1}{\overset{l}{\sum }}{\xi }_{i}^{\ast }\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{s}\text{.t}\text{.}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left[\left(w\cdot {z}_{i}\right)+b\right]\ge 1-\left[\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\right]-{\xi }_{i}^{\ast }\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\ge 0\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{\xi }_{i}^{\ast }\ge 0\end{array}$

3) 只有部分训练样本存在特权信息的SVM模型

$\begin{array}{l}\underset{\omega ,{\omega }^{\ast },b,{b}^{\ast }}{\mathrm{min}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\frac{1}{2}\left[\left(w\cdot w\right)+\gamma \left({w}^{\ast }\cdot {w}^{\ast }\right)\right]+C\underset{i=1}{\overset{n}{\sum }}{\xi }_{i}+{C}^{\ast }\underset{i=n+1}{\overset{l}{\sum }}\left[\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\right]\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{s}\text{.t}\text{.}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left[\left(w\cdot {z}_{i}\right)+b\right]\ge 1-{\xi }_{i},i=1,\cdots ,n\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left[\left(w\cdot {z}_{i}\right)+b\right]\ge 1-\left[\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\right],i=n+1,\cdots ,l\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{\xi }_{i}\ge 0,i=1,\cdots ,n\text{ }\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\ge 0,i=n+1,\cdots ,l\text{ }\text{ }\text{ }\end{array}$

4) 来自多空间特权信息的SVM模型

$\begin{array}{l}\underset{\omega ,{\omega }^{\ast },{\omega }^{\ast \ast }，b,{b}^{\ast },{b}^{\ast \ast }}{\mathrm{min}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\frac{1}{2}\left[\left(w\cdot w\right)+\gamma \left(\left({w}^{\ast }\cdot {w}^{\ast }\right)+\left({w}^{\ast \ast }\cdot {w}^{\ast \ast }\right)\right)\right]\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }+C\underset{i=1}{\overset{n}{\sum }}\left[\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\right]+{C}^{\ast }\underset{i=n+1}{\overset{l}{\sum }}\left[\left({w}^{\ast \ast }\cdot {z}_{i}^{\ast \ast }\right)+{b}^{\ast \ast }\right]\end{array}$

$\begin{array}{l}\text{s}\text{.t}\text{.}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left[\left(w\cdot {z}_{i}\right)+b\right]\ge 1-\left[\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\right],i=1,\cdots ,n\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left[\left(w\cdot {z}_{i}\right)+b\right]\ge 1-\left[\left({w}^{\ast \ast }\cdot {z}_{i}^{\ast \ast }\right)+{b}^{\ast \ast }\right],i=n+1,\cdots ,l\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\ge 0,i=1,\cdots ,n\text{ }\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\left({w}^{\ast \ast }\cdot {z}_{i}^{\ast \ast }\right)+{b}^{\ast \ast }\ge 0,i=n+1,\cdots ,l\end{array}$

4. 具有特权信息的SVM模型应用

1) 具有特权信息的SVM模型在恶意软件检测中的应用

Burnaev提供了改进分类问题的方法即允许合并特权信息。从实验结果可以看出，在某些情况下，特权信息可以显著提高异常检测的准确性。在特权信息对于相关问题的结构没有用的情况下，特权信息不会对分类功能产生重大影响 [10] 。

2) 基于特权信息的SVM模型在高级学习范式中的应用

3) 基于特权信息的SVM模型与经验风险最小化算法的综合应用

5. 总结与讨论

(1) SVM算法的核心是核函数及其参数，它们的正确选取对SVM的预测及泛化性能影响很大 [16] 。对于具体问题，基于特权信息的SVM模型究竟选择哪种核函数并找到最优的参数对求解问题至关重要。因此，如何快速准确地选择核函数及对应的参数是亟待解决的问题。

(2) 在大规模及实时性要求较高的系统中，基于特权信息的SVM算法受制于求解问题的收敛速度和系统规模的复杂程度。尤其要处理大规模数据时，基于特权信息的SVM算法需要解决样本规模和速度间的矛盾，提高训练的效率和精度。

(3) 如何有效地将二分类有效地扩展到多分类问题上，基于特权信息的多分类SVM模型的优化设计也是今后研究的内容。

(4) 针对特定问题如何实现基于特权信息的SVM模型与其他算法的融合，从而顺利地解决问题也是今后需要研究的方向。

Research and Application of SVM Model Based on Privileged Information[J]. 应用数学进展, 2017, 06(09): 1248-1254. http://dx.doi.org/10.12677/AAM.2017.69150

1. 1. Vapnik, V.N. (1998) Statistical Learning Theory. John Wiley & Sons, Hoboken.

2. 2. 熊平. 数据挖掘算法与Clementine实践[M]. 北京: 清华大学出版社, 2011.

3. 3. Bishop, C.M. (2006) Pattern Recognition and Machine Learning. Springer, Berlin.

4. 4. 朱明. 数据挖掘[M]. 合肥: 中国科学技术大学出版社, 2002.

5. 5. Burnaev, E. and Smolyakov, D. (2016) One-Class SVM with Privileged Information and Its Application to Malware Detection. ArXiv, Los Alamos.

6. 6. Gine, E. and Koltchinskii, V. (2006) Concentration Inequalities and Asymptotic Results for Ratio Type Empirical Processes. Annals of Probability, 34, 1143-1216.
https://doi.org/10.1214/009117906000000070

7. 7. 邓乃扬, 田英杰. 数据挖掘的新方法—支持向量机[M]. 北京: 科学出版社, 2004.

8. 8. Manevitz, L.M. and Yousef, M. (2001) One-Class SVMs for Document Classification. Journal of Machine Learning Research, 2, 139-154.

9. 9. Vapnik, V. and Vashist, A. (2009) A New Learning Paradigm: Learning Using Privileged Information. Neural Networks, 22, 544-557.
https://doi.org/10.1016/j.neunet.2009.06.042

10. 10. Boser, G. (1992) A Training Algorithm for Optimal Margin Classifiers. Proceedings of the Fifth Annual Workshop on Computational Learning Theory, Pittsburgh, 27-29 July 1992, 144-152.
https://doi.org/10.1145/130385.130401

11. 11. Cristianini, N., Shawe-Taylor, J. 支持向量机导论[M]. 李国正, 王猛, 曾华军, 译. 北京: 电子工业出版社, 2004.

12. 12. Pechyony, D. and Vapnik, V. On the Theory of Learning with Privileged Information. John Wiley & Sons, Hoboken.

13. 13. Boucheron, S., Bousquet, O. and Lugosi, G. (2005) Theory of Classification: A Survey of Some Recent Advances. ESAIM: Probability and Statistics, 9, 329-375.
https://doi.org/10.1051/ps:2005018

14. 14. Cortes, C. and Vapnik, V. (1995) Support-Vector Networks. Machine Learning, 20, 273-297.
https://doi.org/10.1007/BF00994018

15. 15. Schölkopf, B., Smola, A.J., Williamson, R.C. and Bartlett, P.L. (2000) New Support Vector Algorithms. Neural Computation, 12, 1207-1245.
https://doi.org/10.1162/089976600300015565

16. 16. Micchelli, C. and Pontil, M. (2005) Learning the Kernel Function via Regularization. Journal of Machine Learning Research, 6, 1099-1125.