| 查看: 1264 | 回复: 3 | ||
| 当前只显示满足指定条件的回帖，点击这里查看本话题的所有回帖 | ||
xuh1991木虫 (正式写手)
|
[求助]
BP神经网络和梯度下降法的疑问？ 已有2人参与
|
|
| 在BP神经网络中，默认的训练函数就是梯度下降法，即按照梯度下降法来更新权值，那么这样的BP神经网络和梯度下降法的区别在哪里呢？ |
回复此楼» 猜你喜欢
复试调剂
已经有23人回复
-
求调剂学校
已经有15人回复
-
085801电气专硕272求调剂
已经有19人回复
-
材料工程281还有调剂机会吗
已经有44人回复
-
一志愿华中农业071010，320求调剂
已经有11人回复
-
289 分105500药学专硕求调剂(找B区学校)
已经有3人回复
-
山东省基金2026
已经有12人回复
-
药学求调剂
已经有12人回复
-
一志愿沪9，生物学326求调剂
已经有3人回复
-
307中医考研调剂
已经有3人回复
picklas
木虫 (著名写手)
- 应助: 4 (幼儿园)
- 金币: 6161.9
- 散金: 200
- 红花: 4
- 帖子: 2140
- 在线: 84.9小时
- 虫号: 4104480
- 注册: 2015-09-26
- 性别: GG
- 专业: 凝聚态物性I:结构、力学和
4楼2016-09-04 04:57:27
syddesk
木虫 (正式写手)
- 应助: 125 (高中生)
- 金币: 2181.4
- 散金: 302
- 红花: 17
- 帖子: 474
- 在线: 130小时
- 虫号: 3188437
- 注册: 2014-05-07
- 性别: GG
- 专业: 控制理论与方法
2楼2016-09-02 17:56:02
FMStation
至尊木虫 (知名作家)
- 应助: 591 (博士)
- 贵宾: 0.03
- 金币: 18669.2
- 红花: 97
- 帖子: 8891
- 在线: 1491.4小时
- 虫号: 2400059
- 注册: 2013-04-04
- 专业: 计算机应用技术
【答案】应助回帖
感谢参与，应助指数 +1
|
An analogy for understanding gradient descent The basic intuition behind gradient descent can be illustrated by a hypothetical scenario. A person is stuck in the mountains and is trying to get down (i.e. trying to find the minima). There is heavy fog such that visibility is extremely low. Therefore, the path down the mountain is not visible, so he must use local information to find the minima. He can use the method of gradient descent, which involves looking at the steepness of the hill at his current position, then proceeding in the direction with the steepest descent (i.e. downhill). If he was trying to find the top of the mountain (i.e. the maxima), then he would proceed in the direction steepest ascent (i.e. uphill). Using this method, he would eventually find his way down the mountain. However, assume also that the steepness of the hill is not immediately obvious with simple observation, but rather it requires a sophisticated instrument to measure, which the person happens to have at the moment. It takes quite some time to measure the steepness of the hill with the instrument, thus he should minimize his use of the instrument if he wanted to get down the mountain before sunset. The difficulty then is choosing the frequency at which he should measure the steepness of the hill so not to go off track. https://en.wikipedia.org/wiki/Fi ... o_input_weights.png In this analogy, the person represents the backpropagation algorithm, and the path taken down the mountain represents the sequence of parameter settings that the algorithm will explore. The steepness of the hill represents the slope of the error surface at that point. The instrument used to measure steepness is differentiation (the slope of the error surface can be calculated by taking the derivative of the squared error function at that point). The direction he chooses to travel in aligns with the gradient of the error surface at that point. The amount of time he travels before taking another measurement is the learning rate of the algorithm. https://en.wikipedia.org/wiki/Ba ... ng_gradient_descent |
3楼2016-09-03 19:51:07
