| Commit message (Collapse) | Author | Age |
|
|
|
| |
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
|
|
|
|
|
|
| |
This fix updates librdkafka to 0.11.5.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
|
|\
| |
| |
| | |
PiperOrigin-RevId: 208070785
|
|/ |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Update kafka to v0.11.4
This fix updates kafka from v0.11.1 to v0.11.4
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add additional source files in kafka
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix build issue with KafkaDataset
This fix tries to address the issue raised in 17210 where
error of `NotFoundError: Op type not registered 'KafkaDataset' in binary.`
returned from kafka ops. The issue was that the inclusion of kafka ops
was removed due to the conflict merge from the other PR. This fix
fixes the issue.
This fix fixes 17210.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Change `import readers.Dataset` to `import dataset_ops.Dataset`,
due to the changes in some other places.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Fix library dependency issues in bazel
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add dependency to bazel rules
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add license to lib and pip package
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Remove unneeded changes in bazel
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Address review feedback
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Fix sanity check
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add zlib dependency and include path
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add copts in bazel to address the discrepancy in clang and gcc
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
|
|
* Add KafkaReader for processing streaming data with Apache Kafka
Apache Kafka is a widely used distributed streaming platform in
open source community. The goal of this fix is to create a contrib
Reader ops (inherits ReaderBase and is similiar to
TextLineReader/TFRecordReader) so that it is possible to reader
Kafka streaming data from TensorFlow in a similiar fashion.
This fix uses a C/C++ Apache Kafka client library librdkafka which
is released under the 2-clause BSD license, and is widely used in
a number of Kafka bindings such as Go, Python, C#/.Net, etc.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add KafkaReader Python wrapper.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add BUILD file and op registration for KafkaReader.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add C++ Kernel for KafkaReader
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add librdkafka to third_party packages in Bazel
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add contrib/kafka to part of the contrib bazel file.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Update workspace.bzl
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Comment out clean_deps of `tensorflow/core:framework` and `tensorflow/core:lib`
so that it is possible to build with ReaderBase.
See 1419 for details.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add group id flag.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Sync offset
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add test cases and scipt to start and stop Kafka server (with docker)
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Convert to KafkaConsumer from the legacy Consumer with librdkafka
so that thread join does not hang.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Only output offset as the key.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add timeout attr so that Kafka Consumer could use
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Build Kafka kernels by default, so that to get around the linkage issue.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Convert KafkaReader to KafkaDataset.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Fix workspace.bzl for kafka with tf_http_archive
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add public visibility
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Address review feedbacks
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Optionally select Kafka support through ./configure
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
|