# 一. 背景

## 1. 意义

ConceptLearning作为一个比较早期出现的机器学习方法, 已经"显得"没有特别大的实用价值. 但是, Concept Learning却有比较大的理论价值, 在迈入神经网络等复杂方法之前, 它能够帮助初学者很好地去理解机器学习的一些基本思想和理念(e.g.样本空间, 假设空间).

## 2. 要点

Learning from example
General to specific ordering over hypo(归纳学习)
Version spaces and candidate elimination algo!! (核心)
picking new examples
The need for inductive bias
note: assume no noise, illustrate key concepts

## 1. Problem

Automatically inferring the general definition of some concept, given examples labeled as members or nonmembers of the concept.

## 2. Concept Learning

Inferring a boolean–valued function from training examples of its input and output.

sky temp humid wind water forecst enjoysport(Y)
sunny warm normal strong warm same Y
rainny cold high strong warm change N

Training Example: EnjoySport

# 三. Representing hypo 形式化表示

## 2. Notation(very important)

### 2)c: target concept

c一般就是一个函数, 输入X集合中的某个记录x[i], 就能返回一个y[i]值 --> `y[i] = c(x[i])`

### 3)traning example

an instance x from X, along with its target concept value c(x).

### 6)h: 假设的布尔函数/模型

each hypothesis h in H represents a boolean-valued function defined over X; that is, h: X {0, 1}.

### 1) instance X

We have multiple records, each representing a day. All these make a set of instances that called X.
attributes: Sky, Air, ...

### 2) target concept c

we try to get an concept, an model of EnjoySport based on our data.
target concept c: EnjoySport: X {0, 1}

### 4) D

D = {<attrs, c>, <attrs, c>, ...}

### 5) H, h

Hypo H : Conjunction of literals 取值的集合

Note: 在机器学习中hypo h基本上是一个具体模型/函数公式的同义词

# 四. Concept learning as search

Idea: any hypothesis found to approximate the target function well over a sufficiently large set of training examples will also approximate the target function well over other unobserved examples.

## 1. Instance space X

Sky: 3 possible values
AirTemp: 2 possible values
Humid: 2
Wind: 2
Water: 2
Forecast: 2

The instance space X contains 96 distinct instance: `3*2*2*2*2*2=96`

## 3. Generality 泛化能力/适用面大小

h1=< Sunny, ?, ?, Strong, ?, ? >
h2=< Sunny, ?, ?, ?, ?, ? >
h2 is more general than h1.

# 五. 基础: 只看positive example的Find-S算法

``````1.   Initialize h to the most specific hypothesis in H
2.   For each positive training instance x:
For each attribute constraint a[i] in h:
If the constraint  a[i] in h is satisfied by x:
then do nothing
Else:
replace ai in h by the next more general constraint that is satised by x
3.  Out put hypothesis h
``````

Find-S作为基础算法, 实际上存在很多问题.

# 六. Version Spaces和改进的Candidate-Elimination algo

## 1. Consistent & VersionSpace

Consistent(全体正确性): a hypo h is consistent only if h(x) = c(x) for each training example <x, c(x)> in D

Version Space: VS<H, D> = {for each h∈ H | Consistent(h, D)}, 也就是说H中所有的hypo h都能满足consistent(正确性).

## 2. the List-Then-Eliminate Algo

In a nutshell: generate a list, then try and eliminate list item, finally get a leftover list.

``````1.  Version Space  a list containing every  hypothesis in H
2.  For each training example <x, c(x)>:
remove from Version Space any hypothesis h for  which h(x) != c(x)
3.  Output the list of hypotheses in Version Space
``````

# 七. Compact-Elimination algo

## 1. G and S boundary

G: the general bound G, 是最general的member set
S: the specific bound S, 是最specific的member set

## 2. algo

``````define: G  maximally general hypotheses in H
define: S  maximally specific hypotheses in  H

For each training example d:
If d is a positive example:
Remove from G any hypothesis inconsistent with d
For each hypothesis s in S that is not consistent with d:
Remove s from S //此处其实没有错误, 这个写法相当于先删掉, 再按照all ds的共性添加新的s. 另外一个更有效率的方法, 应该是修改与新引入d矛盾的字段, 将其放松.
Add to S all minimal generalizations h of s  such that  //保持S的最具体性
1.  h is consistent with d and previous ds, and
2.  some member of G is more general than h
Remove from S any hypothesis that is more  general than another hypothesis in S  //去除同质性, 如果s2包含了s1, 那么应该去掉s2.
If d is a negative example:
Remove from S any hypothesis inconsistent with d
For each hypothesis g in G that is not consistent with d  //保持G的最泛化性
Remove g from G
Add to G all minimal specializations h of g  such that
1.  h  is consistent with d, and
2.  some member of S is more specific than h //保证G和S能一致
Remove from G any hypothesis that is less  general than another hypothesis in G  //去除同质性, 如果g2包含了g1, 那么应该去掉g1

``````

## 3. process illustration

### 推荐阅读更多精彩内容

• 原创首发 【龙七七6.7】能不能好好玩耍？ ——坚持第143天 今天是高考的日子，这对很多孩子很多家长来说都是一个...
龙七七阅读 45评论 2 1
• 宿舍的窗打开着，透进光，一片敞亮，对面屋顶青色的瓦，背后藏着白色天蓝。蝉叫得不卖力，云飘了一刻钟。 屋里有点乱，书...
用心玩儿的小超阅读 51评论 1 1
• 心中始终盘桓的强烈挫败感，久久萦绕，片刻不离。很多次想逃离这一切，奈何生存成了最大的绊脚石。于是，就在纠结与痛苦中...
蛾子时述之阅读 5评论 0 0
• 这两天我一直在跟罗振宇和罗永浩的“双罗长谈”，对谈非常精彩而且深刻，有几句话听后就有心灵一击的震撼。 对谈的最后，...
蔷薇石Elsa阅读 35评论 0 1