<>1. RNN Realize stock forecast

* data source ： use tushare Module downloaded SH600519 Sun in Maotai, Guizhou k Line data , Using only this data C Column data , Using continuous 60 Day opening price forecast No 61 Day's opening price .
* Source code ：p37_tushare.py import tushare as ts import matplotlib.pyplot as plt df1 =
ts.get_k_data('600519', ktype='D', start='2010-04-26', end='2020-04-26')
datapath1= "./SH600519.csv" df1.to_csv(datapath1)

<>2. LSTM

tradition RNN Short term memory can be realized through memory , Forecast continuous data . But when the sequence of consecutive data becomes longer , It will become longer with the deployment time step , When backpropagation updates parameters , Gradients are multiplied continuously in time steps , Yes, the gradient disappears . In response to this problem ,Hochreitere Waiting for you 1997 It was proposed in LSTM：

Hochreiter S , Schmidhuber J . Long Short-Term Memory[J]. Neural computation,
1997, 9(8):1735-1780.

<>1） Three thresholds

The three thresholds are the input characteristics of the current time x t x_t xt​ And the short-term memory of the last moment h t − 1 h_{t-1} ht−1​ Function of . In these three formulas W i , W
f and W o W_i, W_f and W_oWi​,Wf​ and Wo​ Is the training parameter matrix , b i , b f and b o b_i, b_f and b_o bi​,bf​ and bo
​ It's a bias to be trained . They all passed by sigmoid Activation function , Make the threshold range within 0-1 between .

* Input gate （ threshold ）： i t = σ ( W t ⋅ [ h t − 1 , x t ] + b i ) i_t=\sigma(W_t·[h_{t-1},
x_t]+b_i)it​=σ(Wt​⋅[ht−1​,xt​]+bi​)
* Forgetting gate （ threshold ）： f t = σ ( W f ⋅ [ h t − 1 , x t ] + b f ) f_t=\sigma(W_f·[h_{t-1},
x_t]+b_f)ft​=σ(Wf​⋅[ht−1​,xt​]+bf​)
* Output gate （ threshold ）： o t = σ ( W o ⋅ [ h t − 1 , x t ] + b o ) o_t=\sigma(W_o·[h_{t-1},
x_t]+b_o)ot​=σ(Wo​⋅[ht−1​,xt​]+bo​)
<>2） Cellular state （ long-term memory ）

* What you remember in your mind is today ppt Page 1 to page 45 Long term memory of pages C t C_t Ct​
, It consists of two parts. The first part is ppt Page 1 to page 44 Contents of the page , That's the long-term memory of the last moment C t − 1 C_{t-1} Ct−1​
, But you can't remember everything word for word , Will not consciously forget some , So the long-term memory of the last moment C t − 1 C_{t-1} Ct−1​
Multiply by the forgetting gate , The product term represents the memory of the past that remains in your mind .
* What I'm talking about now is new knowledge , It's the memory of the present that will be stored in your mind . Memory now consists of two parts , Part of it is what I'm talking about 45 page ppt, Is the input of the current moment x t x_t xt​
, There is also a part of it 44 page ppt Short term memory retention , The last moment of memory is short-term h t − 1 h_{t-1} ht−1​. Your brain inputs the current moment x t x_t xt​
And the short-term memory of the last moment h t − 1 h_{t-1} ht−1​ Induction forms the present memory that will be stored in your mind C t C_t Ct​ Wave sign .
* Memory now C t C_t Ct​ The wave is multiplied by the input gate and stored together with past memory as long-term memory .
* Cellular state （ long-term memory ）： C t = f f + C t − 1 + i t ∗ C ~ t C_t=f_f+C_{t-1}+i_t*\widetilde
C_tCt​=ff​+Ct−1​+it​∗C t​
* Memory （ Short term memory ）： h t = o t ∗ t a n h ( C t ) h_t=o_t*tanh(C_t) ht​=ot​∗tanh(Ct​)
* Candidate states （ New knowledge induced ）： C ~ t = t a n h ( W c ⋅ [ h t − 1 , x t ] + b c ) \widetilde
C_t=tanh(W_c·[h_{t-1}, x_t]+b_c)C t​=tanh(Wc​⋅[ht−1​,xt​]+bc​)

<>3） Memory （ Short term memory ）

* When you retell the story to your friend , You can't tell it all the time , You're talking about long-term memories that exist in your brain , Output the filtered content , This is the output of memory h t h_t ht​.
* When there are multilayer cyclic networks , Input of layer 2 cyclic network x t x_t xt​ It's the output of the first layer of cyclic network h t h_t ht​
. Input the second layer network is the essence of the first level network extraction .
*
for instance ： I'm playing the first layer of circular network , Every page ppt I am exporting the essence from a single article to you . The data you receive as the second layer of the loop network is what I remember for a long time tanh Activation function , And then multiply the short-term memory extracted by the output gate
h t h_tht​.
* Memory （ Short term memory ）： h t = o t ∗ t a n h ( C t ) h_t=o_t*tanh(C_t) ht​=ot​∗tanh(Ct​)
<>4）tensorflow describe LSTM layer
tf.keras.layers.LSTM( Number of memory , return_sequences= Return output ) return_sequences = True #
Output of each time step ht return_sequences = False # Output in last time step only ht（ default ） model = tf.keras.Sequential(
[ LSTM(80, return_sequences=True), Dropout(0.2), LSTM(100), Dropout(0.2), Dense(
1) ])
<>3. GRU

* stay 2014 year cho It is simplified by et al LSTM structure ,GRU Make memory h t h_t ht​ It combines long-term memory with short-term memory .
Cho K , Van Merrienboer B , Gulcehre C , et al. Learning Phrase
Representations using RNN Encoder-Decoder for Statistical Machine
Translation[J]. Computer Science, 2014.

h t h_t ht​ Contains information about the past h t − 1 h_{t-1} ht−1​ And now information h t h_t ht​ Wave sign . Now information is past information h t −
1 h_{t-1}ht−1​ The over reset gate is determined jointly with the current input , The value range of the two thresholds is also 0 reach 1 between . This memory update formula is used directly for forward propagation , You can calculate the value of each moment h t
h_tht​ It's worth it .

* tensorflow describe GRU layer tf.keras.layers.GRU( Number of memory , return_sequences= Return output )
return_sequences= True # Output of each time step ht return_sequences = False # Output in last time step only ht（ default ）
model= tf.keras.Sequential([ GRU(80, return_sequences=True), Dropout(0.2), GRU(
100), Dropout(0.2), Dense(1) ])

Technology