Wondershare EdrawMind
EdrawMindNeural Networks and Deep Learning Recurrent Neural Networks
MindMap Gallery Neural Networks and Deep Learning Recurrent Neural Networks
- 19
Neural Networks and Deep Learning Recurrent Neural Networks
Summarized the basic contents of the recurrent neural network RNN, such as the basic structural principles of RNN, the BPTT algorithm for RNN calculation training, the long short-term memory model LSTM and the basic principles of the gated recurrent unit GRU, etc.
Edited at 2023-02-27 15:20:16- One Hundred Years of Solitude Character Relationship Chart
One Hundred Years of Solitude is the masterpiece of Gabriel Garcia Marquez. Reading this book begins with making sense of the characters' relationships, which are centered on the Buendía family and tells the story of the family's prosperity and decline, internal relationships and political struggles, self-mixing and rebirth over the course of a hundred years.
- One Hundred Years of Solitude Character Relationship Chart
One Hundred Years of Solitude is the masterpiece of Gabriel Garcia Marquez. Reading this book begins with making sense of the characters' relationships, which are centered on the Buendía family and tells the story of the family's prosperity and decline, internal relationships and political struggles, self-mixing and rebirth over the course of a hundred years.
- Project Management Process Template
Project management is the process of applying specialized knowledge, skills, tools, and methods to project activities so that the project can achieve or exceed the set needs and expectations within the constraints of limited resources. This diagram provides a comprehensive overview of the 8 components of the project management process and can be used as a generic template for direct application.
Neural Networks and Deep Learning Recurrent Neural Networks
- One Hundred Years of Solitude Character Relationship Chart
One Hundred Years of Solitude is the masterpiece of Gabriel Garcia Marquez. Reading this book begins with making sense of the characters' relationships, which are centered on the Buendía family and tells the story of the family's prosperity and decline, internal relationships and political struggles, self-mixing and rebirth over the course of a hundred years.
- One Hundred Years of Solitude Character Relationship Chart
One Hundred Years of Solitude is the masterpiece of Gabriel Garcia Marquez. Reading this book begins with making sense of the characters' relationships, which are centered on the Buendía family and tells the story of the family's prosperity and decline, internal relationships and political struggles, self-mixing and rebirth over the course of a hundred years.
- Project Management Process Template
Project management is the process of applying specialized knowledge, skills, tools, and methods to project activities so that the project can achieve or exceed the set needs and expectations within the constraints of limited resources. This diagram provides a comprehensive overview of the 8 components of the project management process and can be used as a generic template for direct application.
- Recommended to you
- Outline
Neural Networks and Deep Learning recurrent neural network
Basic RNN
sequentially
Data arranged in order, adjacent members of the sequence are often related
Convolutional neural network CNN can process this kind of sequence data but is not flexible enough. Recurrent neural network RNN with feedback loop is more suitable for processing this kind of sequence data.
Schematic diagram of RNN principle
Equation of state
State variables
Indicates system changes and memory capabilities
The current state is related to the state of the last moment and the current input
The state variable remembers the contribution of the input sequence from the starting time
linear dynamic system
nonlinear dynamic system
RNN calculation and training
unit calculation
Expand calculation graph
BPTT algorithm
Deep RNN network
long term dependence
reason
Backward error propagation term
After sequence time step N
W eigenvalue is greater than 1
gradient explosion
W eigenvalue is less than 1
gradient disappears
Solution
Gradient truncation method (threshold limiting gradient)
Gated RNN method
Long short-term memory model LSTM
Gated Recirculating Unit GRU
Long short-term memory model LSTM
LSTM calculation expansion diagram
Method to realize
Ideas
Divide memory vectors into "long-term memory" and "short-term memory"
long term memory
memory cell vector
Controlling inner loop memory and forgetting
short term memory
hidden vector
Input to gating network after loop delay (outer loop)
gated memory process
oblivion door
control memory level
memory
forget
partial memory
input gate
Candidate memory cells
Effects of input on memory cells
Current composition of new memory cells
Memory cells retain components at the previous moment. The impact of current input on memory cells.
output gate
Gated Recirculating Unit GRU
GRU calculation expansion diagram
gated memory process
reset gate
Affects new memory generation
update door
Controlling memory and forgetting of historical and new input
candidate hidden state (candidate memory)
Control new memorized elements
status output
The hidden state retention component of the previous moment and the influence of the current input
LSTM and GRU comparison
GRU can be regarded as a simplified version of LSTM, which is equivalent to LSTM in performance. It uses update gate to replace forget gate and input gate.