1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
|
/* Copyright 2016 Google Inc. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/core/framework/op.h"
namespace tensorflow {
// --------------------------------------------------------------------------
REGISTER_OP("SdcaSolver")
.Attr("loss_type: {'logistic_loss', 'squared_loss', 'hinge_loss'}")
.Attr("num_sparse_features: int >= 0")
.Attr("num_dense_features: int >= 0")
.Attr("l1: float >= 0")
.Attr("l2: float >= 1")
.Attr("num_inner_iterations: int >= 2")
.Attr("container: string")
.Attr("solver_uuid: string")
.Input("sparse_features_indices: num_sparse_features * int64")
.Input("sparse_features_values: num_sparse_features * float")
.Input("dense_features: num_dense_features * float")
.Input("example_weights: float")
.Input("example_labels: float")
.Input("example_ids: string")
.Input("sparse_weights: Ref(num_sparse_features * float)")
.Input("dense_weights: Ref(num_dense_features * float)")
.Doc(R"doc(
Stochastic Dual Coordinate Ascent (SDCA) optimizer for linear models with
L1 + L2 regularization. As global optimization objective is strongly-convex, the
optimizer optimizes the dual objective at each step. The optimizer applies each
update one example at a time. Examples are sampled uniformly, and the optimizer
is learning rate free and enjoys linear convergence rate.
Proximal Stochastic Dual Coordinate Ascent, Shalev-Shwartz, Shai; Zhang, Tong.
2012arXiv1211.2717S: http://arxiv.org/pdf/1211.2717v1.pdf
Loss objective = \sum f_{i}(wx_{i}) + l2 * |w|^2 + l1 * |w|
loss_type: Type of the primal loss. Currently SdcaSolver supports logistic,
squared and hinge losses.
num_sparse_features: Number of sparse feature groups to train on.
num_dense_features: Number of dense feature groups to train on.
l1: Symmetric l1 regularization strength.
l2: Symmetric l2 regularization strength.
num_inner_iterations: Number of iterations per mini-batch.
container: Name of the Container that stores data across invocations of this
Kernel. Together with SolverUUID form an isolation unit for this solver.
solver_uuid: Universally Unique Identifier for this solver.
sparse_features_indices: a list of matrices with two columns that contain
example_indices, and feature_indices.
sparse_features_values: a list of vectors which contains feature value
associated with each feature group.
dense_features: a list of vectors which contains the dense feature values.
example_weights: a vector which contains the weight associated with each
example.
example_labels: a vector which contains the label/target associated with each
example.
example_ids: a vector which contains the unique identifier associated with each
example.
sparse_weights: a list of vectors where each value is the weight associated with
a feature index.
dense_weights: a list of vectors where the value is the weight associated with
a dense feature group.
)doc");
REGISTER_OP("SdcaShrinkL1")
.Attr("num_sparse_features: int >= 0")
.Attr("num_dense_features: int >= 0")
.Attr("l1: float >= 0")
.Attr("l2: float >= 1")
.Input("sparse_weights: Ref(num_sparse_features * float)")
.Input("dense_weights: Ref(num_dense_features * float)")
.Doc(R"doc(
Applies L1 regularization shrink step on the parameters.
num_sparse_features: Number of sparse feature groups to train on.
num_dense_features: Number of dense feature groups to train on.
l1: Symmetric l1 regularization strength.
l2: Symmetric l2 regularization strength.
sparse_weights: a list of vectors where each value is the weight associated with
a feature index.
dense_weights: a list of vectors where the value is the weight associated with
a dense feature group.
)doc");
// TODO(katsiapis): We should expand this scope of this op to compute other
// statistics about the data.
REGISTER_OP("ComputeDualityGap")
.Attr("num_sparse_features: int >= 0")
.Attr("num_dense_features: int >= 0")
.Attr("l1: float >= 0")
.Attr("l2: float >= 1")
.Attr("container: string")
.Attr("solver_uuid: string")
.Input("sparse_weights: Ref(num_sparse_features * float)")
.Input("dense_weights: Ref(num_dense_features * float)")
.Output("duality_gap: float")
.Doc(R"doc(
Computes duality gap over all examples seen by the optimizer.
num_sparse_features: Number of sparse feature groups to train on.
num_dense_features: Number of dense feature groups to train on.
l1: Symmetric l1 regularization strength.
l2: Symmetric l2 regularization strength.
container: Name of the Container that stores data across invocations of this
Kernel. Together with SolverUUID form an isolation unit for this solver.
solver_uuid: Universally Unique Identifier for this solver.
sparse_weights: a list of vectors where each value is the weight associated with
a feature index.
dense_weights: a list of vectors where the value is the weight associated with
a dense feature group.
duality_gap: duality gap over all examples seen by the optimizer.
)doc");
} // namespace tensorflow
|
