机器学习
“人工神经网络”一词源自生物神经网络,后者构成了人脑的结构。与相互连接的神经元组成人脑类似,人工神经网络也拥有在网络各层中相互连接的神经元。这些神经元被称为节点。
下图描绘了典型的生物神经网络图。
典型的人工神经网络看起来如下图所示。
生物神经网络中的树突在人工神经网络中代表输入,细胞核代表节点,突触代表权重,而轴突代表输出。
生物神经网络与人工神经网络的关系:
生物神经网络 | 人工神经网络 |
树突 | 输入 |
细胞核 | 节点 |
突触 | 权重 |
轴突 | 输出 |
人工神经网络是人工智能领域的一个分支,它试图模仿构成人脑的神经元网络,以便计算机能够像人类一样理解事物并做出决策。人工神经网络通过编程使计算机像相互连接的脑细胞一样简单地运行来设计。
人脑大约有 1000 亿个神经元。每个神经元都有约 1,000 到 100,000 个关联点。在人脑中,数据以分布式的方式存储,我们可以在必要时并行地从记忆中提取不止一条数据。可以说,人脑是由令人难以置信的并行处理器组成的。
我们可以用一个例子来理解人工神经网络,考虑一个接收输入并给出输出的数字逻辑门。“或”门,它接收两个输入。如果其中一个或两个输入都是“开”,那么输出就是“开”。如果两个输入都是“关”,那么输出就是“关”。这里的输出取决于输入。我们的大脑不执行相同的任务。由于我们大脑中的神经元在“学习”,输入与输出的关系不断变化。
人工神经网络的架构:
- 输入层(Input Layer): 顾名思义,它以程序员提供的多种不同格式接受输入。
- 隐藏层(Hidden Layer): 隐藏层位于输入层和输出层之间。它执行所有计算以查找隐藏的特征和模式。
- 输出层(Output Layer): 输入通过隐藏层进行一系列转换,最终产生输出,并通过此层进行传递。
人工神经网络接收输入并计算输入的加权和,并包含一个偏置项(bias)。这种计算以传递函数的形式表示。
它确定加权总和作为输入传递给激活函数以产生输出。激活函数选择一个节点是否应该“激活”(fire)。只有那些被激活的节点才能到达输出层。有不同的激活函数可用,可以根据我们正在执行的任务类型进行应用。
人工神经网络(ANN)的优点
- 并行处理能力: 人工神经网络具有数值能力,可以同时执行多项任务。
- 将数据存储在整个网络上: 传统编程中使用的数据存储在整个网络上,而不是存储在数据库中。某个地方的几条数据丢失不会妨碍网络工作。
- 处理不完整知识的能力: 在 ANN 训练后,即使数据不完整,信息也可能产生输出。这里的性能损失取决于缺失数据的重要性。
- 具有记忆分布: 对于 ANN 来说,为了能够适应,确定示例并根据期望的输出来鼓励网络,通过向网络演示这些示例是很重要的。网络的成功与所选实例成正比,如果事件无法以其所有方面出现在网络中,它可能会产生错误的输出。
- 具有容错性: 破坏 ANN 的一个或多个单元不会妨碍它生成输出,此功能使网络具有容错性。
人工神经网络的缺点:
- 确保适当的网络结构: 没有特定的指导原则来确定人工神经网络的结构。适当的网络结构是通过经验、反复试验来实现的。
- 网络行为无法识别: 这是 ANN 最重要的问题。当 ANN 产生测试解决方案时,它不提供关于为什么以及如何的见解。这降低了对网络的信任。
- 硬件依赖: 人工神经网络根据其结构需要具有并行处理能力的处理器。因此,设备的实现是依赖于硬件的。
- 向网络展示问题的难度: ANN 可以处理数值数据。在引入 ANN 之前,问题必须转换为数值。这里要解决的呈现机制将直接影响网络的性能。这取决于用户的能力。
- 网络持续时间未知: 网络被缩减到特定的误差值,而这个值并没有给我们带来最佳结果。
“20世纪中叶进入世界的人工神经网络正在呈指数级发展。在当前,我们已经研究了人工神经网络的优点以及在利用过程中遇到的问题。不应忽视的是,作为一门蓬勃发展的科学分支,ANN 网络的缺点正在被逐一消除,而它们的优点正在日益增加。这意味着人工神经网络将逐步成为我们生活中不可替代的一部分,变得越来越重要。”
人工神经网络如何工作?
人工神经网络可以最好地表示为加权有向图,其中人工神经元形成节点。神经元输出和神经元输入之间的关联可以看作是带有权重的有向边。人工神经网络以模式和图像的矢量形式接收来自外部源的输入信号。然后,这些输入通过符号 x(n) 为每个 n 个输入进行数学赋值。
随后,每个输入都乘以其相应的权重(这些权重是人工神经网络用于解决特定问题的细节)。一般来说,这些权重通常表示人工神经网络内部神经元之间互连的强度。所有加权输入都在计算单元内进行汇总。
如果加权和等于零,则添加偏置项以使输出非零或通过其他方式扩展系统响应。偏置项具有相同的输入,权重等于 1。这里加权输入的总和可以在 0 到正无穷的范围内。为了将响应保持在期望值的限制内,这里设定了一个特定的最大值作为基准,并将加权输入的总和通过激活函数。
激活函数是指用于实现所需输出的传递函数集。有不同类型的激活函数,但主要分为线性和非线性函数集。一些常用的激活函数集是二元、线性以及 tanh Sigmoidal 激活函数。让我们详细了解它们中的每一个:
- 二元: 在二元激活函数中,输出要么是 1,要么是 0。这里,为了实现这一点,设置了一个阈值。如果神经元的净加权输入大于 1,则激活函数的最终输出返回为 1,否则输出返回为 0。
- Sigmoidal Hyperbolic(S 形双曲正切): S 形双曲正切函数通常被看作是一个“S”形曲线。这里使用 tanh 函数来近似从实际净输入中得到的输出。该函数定义为: β 被认为是陡度参数。 其中
人工神经网络的类型:
根据人脑神经元和网络功能,人工神经网络也执行类似的任务。大多数人工神经网络将与更复杂的生物对应物有一些相似之处,并且在预期任务中非常有效。例如,分割或分类。
- 反馈 ANN(Feedback ANN): 在这种类型的 ANN 中,输出返回到网络内部,以获得最佳演化结果。根据马萨诸塞州洛厄尔大学大气研究中心的说法,反馈网络将信息反馈到自身,非常适合解决优化问题。内部系统错误校正利用反馈 ANN。
- 前馈 ANN(Feed-Forward ANN): 前馈网络是一个基本的神经网络,由一个输入层、一个输出层和至少一个神经元层组成。通过审查其输入来评估其输出,网络的强度可以根据关联神经元的群体行为来观察,并决定输出。这种网络的主要优点是它能够评估和识别输入模式。
先决条件
在开始本教程之前,不需要任何特定的专业知识。
受众
我们的人工神经网络教程是为初学者和专业人士开发的,旨在帮助他们理解 ANN 的基本概念。
The term "Artificial Neural Network" is derived from Biological neural networks that develop the structure
of a human brain. Similar to the human brain that has neurons interconnected to one another, artificial neural
networks also have neurons that are interconnected to one another in various layers of the networks. These neurons
are known as nodes.
The given figure illustrates the typical diagram of Biological Neural Network.
The typical Artificial Neural Network looks something like the given figure.
Dendrites from Biological Neural Network represent inputs in Artificial Neural Networks, cell nucleus represents
Nodes, synapse represents Weights, and Axon represents Output.
Relationship between Biological neural network and artificial neural network:
Biological Neural Network
Artificial Neural Network
Dendrites
Inputs
Cell nucleus
Nodes
Synapse
Weights
Axon
Output
An Artificial Neural Network in the field of Artificial intelligence where it attempts to mimic the network of
neurons makes up a human brain so that computers will have an option to understand things and make decisions in
a human-like manner. The artificial neural network is designed by programming computers to behave simply like
interconnected brain cells.
There are around 1000 billion neurons in the human brain. Each neuron has an association point somewhere in the
range of 1,000 and 100,000. In the human brain, data is stored in such a manner as to be distributed, and we can
41
extract more than one piece of this data when necessary from our memory parallelly. We can say that the human
brain is made up of incredibly amazing parallel processors.
We can understand the artificial neural network with an example, consider an example of a digital logic gate that
takes an input and gives an output. "OR" gate, which takes two inputs. If one or both the inputs are "On," then we
get "On" in output. If both the inputs are "Off," then we get "Off" in output. Here the output depends upon input.
Our brain does not perform the same task. The outputs to inputs relationship keep changing because of the neurons
in our brain, which are "learning."
The architecture of an artificial neural network:
Input Layer:
As the name suggests, it accepts inputs in several different formats provided by the programmer.
Hidden Layer:
The hidden layer presents in-between input and output layers. It performs all the calculations to find
hidden features and patterns.
Output Layer:
The input goes through a series of transformations using the hidden layer, which finally results in output
that is conveyed using this layer.
The artificial neural network takes input and computes the weighted sum of the inputs and includes a bias. This
computation is represented in the form of a transfer function.
It determines weighted total is passed as an input to an activation function to produce the output. Activation
functions choose whether a node should fire or not. Only those who are fired make it to the output layer. There are
distinctive activation functions available that can be applied upon the sort of task we are performing.
Advantages of Artificial Neural Network (ANN)
Parallel processing capability:
Artificial neural networks have a numerical value that can perform more than one task simultaneously.
Storing data on the entire network:
Data that is used in traditional programming is stored on the whole network, not on a database. The
disappearance of a couple of pieces of data in one place doesn't prevent the network from working.
Capability to work with incomplete knowledge:
After ANN training, the information may produce output even with inadequate data. The loss of
42
performance here relies upon the significance of missing data.
Having a memory distribution:
For ANN is to be able to adapt, it is important to determine the examples and to encourage the network
according to the desired output by demonstrating these examples to the network. The succession of the network is
directly proportional to the chosen instances, and if the event can't appear to the network in all its aspects, it can
produce false output.
Having fault tolerance:
Extortion of one or more cells of ANN does not prohibit it from generating output, and this feature makes
the network fault-tolerance.
Disadvantages of Artificial Neural Network:
Assurance of proper network structure:
There is no particular guideline for determining the structure of artificial neural networks. The appropriate
network structure is accomplished through experience, trial, and error.
Unrecognized behavior of the network:
It is the most significant issue of ANN. When ANN produces a testing solution, it does not provide insight
concerning why and how. It decreases trust in the network.
Hardware dependence:
Artificial neural networks need processors with parallel processing power, as per their structure. Therefore,
the realization of the equipment is dependent.
Difficulty of showing the issue to the network:
ANNs can work with numerical data. Problems must be converted into numerical values before being
introduced to ANN. The presentation mechanism to be resolved here will directly impact the performance of the
network. It relies on the user's abilities.
The duration of the network is unknown:
The network is reduced to a specific value of the error, and this value does not give us optimum results.
“Science artificial neural networks that have steeped into the world in the mid-20th century are exponentially
developing. In the present time, we have investigated the pros of artificial neural networks and the issues
encountered in the course of their utilization. It should not be overlooked that the cons of ANN networks, which are
a flourishing science branch, are eliminated individually, and their pros are increasing day by day. It means that
artificial neural networks will turn into an irreplaceable part of our lives progressively important.”
How do artificial neural networks work?
Artificial Neural Network can be best represented as a weighted directed graph, where the artificial
neurons form the nodes. The association between the neurons outputs and neuron inputs can be viewed as the
directed edges with weights. The Artificial Neural Network receives the input signal from the external source in the
form of a pattern and image in the form of a vector. These inputs are then mathematically assigned by the notations
x for every n number of inputs.
43
Afterward, each of the input is multiplied by its corresponding weights ( these weights are the details
utilized by the artificial neural networks to solve a specific problem ). In general terms, these weights normally
represent the strength of the interconnection between neurons inside the artificial neural network. All the weighted
inputs are summarized inside the computing unit.
If the weighted sum is equal to zero, then bias is added to make the output non-zero or something else to
scale up to the system's response. Bias has the same input, and weight equals to 1. Here the total of weighted inputs
can be in the range of 0 to positive infinity. Here, to keep the response in the limits of the desired value, a certain
maximum value is benchmarked, and the total of weighted inputs is passed through the activation function.
The activation function refers to the set of transfer functions used to achieve the desired output. There is a
different kind of the activation function, but primarily either linear or non-linear sets of functions. Some of the
commonly used sets of activation functions are the Binary, linear, and Tan hyperbolic sigmoidal activation
functions. Let us take a look at each of them in details:
Binary:
In binary activation function, the output is either a one or a 0. Here, to accomplish this, there is a threshold
value set up. If the net weighted input of neurons is more than 1, then the final output of the activation function is
returned as one or else the output is returned as 0.
Sigmoidal Hyperbolic:
The Sigmoidal Hyperbola function is generally seen as an "S" shaped curve. Here the tan hyperbolic
function is used to approximate output from the actual net input. The function is defined as:
F(x) = (1/1 + exp(-????x))
Where ???? is considered the Steepness parameter.
Types of Artificial Neural Network:
There are various types of Artificial Neural Networks (ANN) depending upon the human brain neuron and
network functions, an artificial neural network similarly performs tasks. The majority of the artificial neural
networks will have some similarities with a more complex biological partner and are very effective at their expected
tasks. For example, segmentation or classification.
Feedback ANN:
In this type of ANN, the output returns into the network to accomplish the best-evolved results internally.
As per the University of Massachusetts, Lowell Centre for Atmospheric Research. The feedback networks feed
information back into itself and are well suited to solve optimization issues. The Internal system error corrections
utilize feedback ANNs.
Feed-Forward ANN:
A feed-forward network is a basic neural network comprising of an input layer, an output layer, and at least
one layer of a neuron. Through assessment of its output by reviewing its input, the intensity of the network can be
noticed based on group behavior of the associated neurons, and the output is decided. The primary advantage of this
network is that it figures out how to evaluate and recognize input patterns.
44
Prerequisite
No specific expertise is needed as a prerequisite before starting this tutorial.
Audience
Our Artificial Neural Network Tutorial is developed for beginners as well as professionals, to help them
understand the basic concept of ANNs.