K-近邻手写体识别

魔法师LQ

构建K近邻手写体识别分类器。

流程:

  1. 收集数据:文本,下载
  2. 准备数据:编写函数img2vector,将图像转换成向量格式
  3. 分析数据:检查并确保数据符合要求
  4. 训练算法:K近邻算法不适用此步
  5. 测试算法:将数据集中一部分作为测试数据,来测试样本。测试数据和非测试数据的区别在于,测试数据是已经完成分类的数据。
  6. 使用算法:从图像中提取数字并完成数字识别

准备数据:将图像转换为测试向量

1
2
3
4
5
6
7
8
9
import numpy as np
# 将文件中的32*32的图像转换成为1*1024的向量
def img2vector(filename):
returnVec = []
with open(filename, 'r') as f:
for i in range(32):
line = list(f.readline().strip())
returnVec += line
return np.array(returnVec, dtype=np.int32).reshape(1, 1024)
1
testVect = img2vector('digits/trainingDigits/0_0.txt')
1
testVect[0, 0:31]
array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0,
       0, 0, 0, 0, 0, 0, 0, 0, 0])
1
2
3
# 查看一下数据
for line in testVect.reshape(32, 32):
print(line)
[0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0]
[0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0]
[0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0]
[0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0]
[0 0 0 0 0 0 1 1 1 1 0 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0]
[0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]

测试算法:使用K-近邻算法识别手写数字

首先需要确保from os import listdir,列出制定目录的文件名

1
from os import listdir
1
from KNN import classify0 # 见上一篇文章
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
def handwritingClassTest(k=3):
# 从文件中解析手写字母的类型
trainingFileList = listdir('digits/trainingDigits')
hwLabels = []
trainingMat = np.zeros((len(trainingFileList), 1024))
for i, filename in enumerate(trainingFileList):
hwLabels += filename.split('.')[0].split('_')[0]
trainingMat[i, :] = img2vector('digits/trainingDigits/%s'%filename)
# trainingMat = np.array(trainingMat).reshape(len(trainingFileList), 1024)
# 测试k近邻算法
testFileList = listdir('digits/testDigits')
errorCount = 0.0
m = len(testFileList)
for i in range(m):
fileNameStr = testFileList[i]
classNumStr = fileNameStr.split('.')[0].split('_')[0]
vectorUnderTest = img2vector('digits/testDigits/%s'%fileNameStr)
classifierResult = classify0(vectorUnderTest, trainingMat, hwLabels, k)
print('Classify result: ', classifierResult, ' Real class: ', classNumStr)
if (classifierResult!=classNumStr):
errorCount+=1.0
print('\nThe total number of errors is: %d'%errorCount)
print('\nTotal error rate is: %f (%d/%d)'%(errorCount/float(m), errorCount, m))
1
help(classify0)
Help on function classify0 in module KNN:

classify0(inX, dataSet, labels, k)
    使用K近邻算法,对输入特征进行分类。
    输入参数:
    - inX: 输入特征向量
    - dataSet: 数据集,计算输入向量和数据集中每个点的距离。

1
handwritingClassTest()
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  0  Real class:  0
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  7  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  1  Real class:  1
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  2  Real class:  2
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  9  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  3  Real class:  3
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  4  Real class:  4
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  3  Real class:  5
Classify result:  6  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  5  Real class:  5
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  6  Real class:  6
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  7  Real class:  7
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  6  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  3  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  1  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  1  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  8  Real class:  8
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  1  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  7  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9
Classify result:  9  Real class:  9

The total number of errors is: 10

Total error rate is: 0.010571 (10/946)

小节

K近邻算法是简单有效的分类算法,但算法的执行效率并不高。对于每个测试点,都需要计算其和所有其他点的距离,每个距离计算包含了特征向量这么多维度的浮点运算,总共执行测试点这么多的次。当数据集很大时需要消耗很大的存储空间。
此外,K近邻算法的另一个缺陷就是其无法给出任何数据的基础结构信息,所以我们无法知晓平均实力样本和典型样例有什么特征
K决策树是K近邻算法的优化版本,可以减少算法存储空间和计算时间开销。

本例中,使用KNN构建手写体识别系统任务,剩余的工作例如从图像中提取数字,数据的检查和规格化等没有完全完成,因此侧重于算法理解和应用。