| Commit message (Collapse) | Author | Age |
|
|
|
| |
PiperOrigin-RevId: 215870962
|
|
|
|
|
|
|
|
| |
output tensor.
This is useful if the output of both directions will be passed to the next layer as a single output, as it avoids adding a concatenation op, which can be expensive on mobile devices where memory movement is relatively expensive.
PiperOrigin-RevId: 215616140
|
|
|
|
| |
PiperOrigin-RevId: 214618170
|
|
|
|
| |
PiperOrigin-RevId: 212064501
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
LSTM Op.
This introduces a connection between forward and backward cells across subsequent layers when stacking bidirectional LSTM Ops on top of each other.
In more detail:
Previously, the Op had only one input that was fed into the layer in the
following way:
INPUT (INPUT_REVERSED)
| |
-----------------------
| FW_LSTM BW_LSTM | <----- bidi-LSTM cell (with one input / two outputs)
-----------------------
| |
FW_OUT BW_OUT
Now, the Op can have an (optional) auxiliary input in the following way:
AUX_INPUT (AUX_INPUT_REVERSED)
| |
INPUT | (INPUT_R'D.)|
| | | |
-------------------------
| \ / \ / |
| FW_LSTM BW_LSTM | <----- bidi-LSMT cell (with 2 inputs / 2 outputs)
-------------------------
| |
FW_OUT BW_OUT
When stacking these Ops, previously, only the following flow was allowed:
Input
/ \
FW_LSTM1 BW_LSTM1
| |
| |
FW_LSTM2 BW_LSTM2
| |
| |
FW_LSTM3 BW_LSTM3
\ /
Output
With the introduction of an auxiliary input to the bidi-LSTM layer, the forward
(FW_LSTMi) output of the ith layer is fed into as the input to the next layer
(hence, inputs to both FW_LSTM{i+1} and BW_LSTM{i+1}) and the backward output is
fed as the auxiliary inputs to both FW_LSTM{i+1} and BW_LSTM{i+1}). This way, the
stacking can be changed to allow for the "cross-linking" between subsequent
layer in the following way:
Input
/ \
FW_LSTM1 BW_LSTM1
| \ / |
| / \ |
FW_LSTM2 BW_LSTM2
| \ / |
| / \ |
FW_LSTM3 BW_LSTM3
\ /
Output
PiperOrigin-RevId: 211659472
|
|
|
|
| |
PiperOrigin-RevId: 211552101
|
|
|
|
| |
PiperOrigin-RevId: 211378028
|
|
|
|
| |
PiperOrigin-RevId: 205744600
|
|
|
|
| |
PiperOrigin-RevId: 204341675
|
|
|
|
| |
PiperOrigin-RevId: 196932028
|
|
|
|
| |
PiperOrigin-RevId: 196340200
|
|
|
|
| |
PiperOrigin-RevId: 195374319
|
|
|
|
| |
PiperOrigin-RevId: 194877173
|
|
|
|
| |
PiperOrigin-RevId: 187370622
|
|
PiperOrigin-RevId: 186497571
|