机器学习

# 计算自举样本
samplePoints = np.random.randint(0,nPoints,(nPoints,nSamples)) # 从0到nPoints-1之间随机生成整数，形状为(nPoints, nSamples)
classifiers = [] # 存储训练好的分类器
for i in range(nSamples): # 遍历每个自举样本
    sample = [] # 当前自举样本的特征数据
    sampleTarget = [] # 当前自举样本的目标标签
    for j in range(nPoints): # 为当前自举样本填充数据
        # 假设这里的 'data' 和 'targets' 分别是您的原始特征和标签数据
        # sample.append(data[samplePoints[j,i]]) # 实际代码中可能需要从原始数据中提取特征
        sampleTarget.append(targets[samplePoints[j,i]]) # 从原始标签中提取标签

    # 训练分类器 (假设 self.tree 是一个可以训练决策树的实例)
    classifiers.append(self.tree.make_tree(sample,sampleTarget,features))

Bootstrap aggregating, often abbreviated as bagging, involves having each model in the
ensemble vote with equal weight. In order to promote model variance, bagging trains each model in the
ensemble using a randomly drawn subset of the training set. As an example, the random
forest algorithm combines random decision trees with bagging to achieve very high classification
accuracy.
The simplest method of combining classifiers is known as bagging, which stands for bootstrap
aggregating, the statistical description of the method. This is fine if you know what a bootstrap is, but
fairly useless if you don’t. A bootstrap sample is a sample taken from the original dataset with
replacement, so that we may get some data several times and others not at all. The bootstrap sample is
the same size as the original, and lots and lots of these samples are taken: B of them, where B is at least
50, and could even be in the thousands. The name bootstrap is more popular in computer science than
anywhere else, since there is also a bootstrap loader, which is the first program to run when a computer
is turned on. It comes from the nonsensical idea of ‘picking yourself up by your bootstraps,’ which
means lifting yourself up by your shoelaces, and is meant to imply starting from nothing.
Bootstrap sampling seems like a very strange thing to do. We’ve taken a perfectly good dataset,
mucked it up by sampling from it, which might be good if we had made a smaller dataset (since it would
be faster), but we still ended up with a dataset the same size. Worse, we’ve done it lots of times. Surely
this is just a way to burn up computer time without gaining anything. The benefit of it is that we will get
lots of learners that perform slightly differently, which is exactly what we want for an ensemble
method. Another benefit is that estimates of the accuracy of the classification function can be made
without complicated analytic work, by throwing computer resources at the problem (technically,
bagging is a variance reducing algorithm; the meaning of this will become clearer when we talk about
bias and variance). Having taken a set of bootstrap samples, the bagging method simply requires that
we fit a model to each dataset, and then combine them by taking the output to be the majority vote of
all the classifiers. A NumPy implementation is shown next, and then we will look at a simple example.
# Compute bootstrap samples
samplePoints = np.random.randint(0,nPoints,(nPoints,nSamples))
classifiers = []
for i in range(nSamples):
sample = []
sampleTarget = []
for j in range(nPoints):
sampleTarget.append(targets[samplePoints[j,i]])
# Train classifiers
classifiers.append(self.tree.make_tree(sample,sampleTarget,features))
The example consists of taking the party data that was used to demonstrate the decision tree, and
restricting the trees to stumps, so that they can make a classification based on just one variable
When we want to construct the decision tree to decide what to do in the evening, we start by listing
everything that we’ve done for the past few days to get a suitable dataset (here, the last ten days):
The output of a decision tree that uses the whole dataset for this is not surprising: it takes the two
largest classes, and separates them. However, using just stumps of trees and 20 samples, bagging can
separate the data perfectly, as this output shows: