| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
LSTM Op.
This introduces a connection between forward and backward cells across subsequent layers when stacking bidirectional LSTM Ops on top of each other.
In more detail:
Previously, the Op had only one input that was fed into the layer in the
following way:
INPUT (INPUT_REVERSED)
| |
-----------------------
| FW_LSTM BW_LSTM | <----- bidi-LSTM cell (with one input / two outputs)
-----------------------
| |
FW_OUT BW_OUT
Now, the Op can have an (optional) auxiliary input in the following way:
AUX_INPUT (AUX_INPUT_REVERSED)
| |
INPUT | (INPUT_R'D.)|
| | | |
-------------------------
| \ / \ / |
| FW_LSTM BW_LSTM | <----- bidi-LSMT cell (with 2 inputs / 2 outputs)
-------------------------
| |
FW_OUT BW_OUT
When stacking these Ops, previously, only the following flow was allowed:
Input
/ \
FW_LSTM1 BW_LSTM1
| |
| |
FW_LSTM2 BW_LSTM2
| |
| |
FW_LSTM3 BW_LSTM3
\ /
Output
With the introduction of an auxiliary input to the bidi-LSTM layer, the forward
(FW_LSTMi) output of the ith layer is fed into as the input to the next layer
(hence, inputs to both FW_LSTM{i+1} and BW_LSTM{i+1}) and the backward output is
fed as the auxiliary inputs to both FW_LSTM{i+1} and BW_LSTM{i+1}). This way, the
stacking can be changed to allow for the "cross-linking" between subsequent
layer in the following way:
Input
/ \
FW_LSTM1 BW_LSTM1
| \ / |
| / \ |
FW_LSTM2 BW_LSTM2
| \ / |
| / \ |
FW_LSTM3 BW_LSTM3
\ /
Output
PiperOrigin-RevId: 211659472
|
A. Unique TensorFlower