aboutsummaryrefslogtreecommitdiffhomepage
path: root/third_party/kafka
Commit message (Collapse)AuthorAge
* Fix missing inclue file in kafka/BUILDGravatar Yong Tang2018-08-10
| | | | Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Update Kafka to 0.11.5Gravatar Yong Tang2018-08-10
| | | | | | This fix updates librdkafka to 0.11.5. Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Merge pull request #20541 from rongjiecomputer:kafkaGravatar TensorFlower Gardener2018-08-09
|\ | | | | | | PiperOrigin-RevId: 208070785
| * [Bazel/MSVC] Make Kafka builds on WindowsGravatar Loo Rong Jie2018-07-20
|/
* Update kafka to v0.11.4 (#20417)Gravatar Yong Tang2018-06-29
| | | | | | | | | | | | * Update kafka to v0.11.4 This fix updates kafka from v0.11.1 to v0.11.4 Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Add additional source files in kafka Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Fix build issue with KafkaDataset (#17418)Gravatar Yong Tang2018-03-07
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Fix build issue with KafkaDataset This fix tries to address the issue raised in 17210 where error of `NotFoundError: Op type not registered 'KafkaDataset' in binary.` returned from kafka ops. The issue was that the inclusion of kafka ops was removed due to the conflict merge from the other PR. This fix fixes the issue. This fix fixes 17210. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Change `import readers.Dataset` to `import dataset_ops.Dataset`, due to the changes in some other places. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Fix library dependency issues in bazel Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Add dependency to bazel rules Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Add license to lib and pip package Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Remove unneeded changes in bazel Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Address review feedback Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Fix sanity check Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Add zlib dependency and include path Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Add copts in bazel to address the discrepancy in clang and gcc Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add KafkaReader for processing streaming data with Apache Kafka (#14098)Gravatar Yong Tang2018-01-26
* Add KafkaReader for processing streaming data with Apache Kafka Apache Kafka is a widely used distributed streaming platform in open source community. The goal of this fix is to create a contrib Reader ops (inherits ReaderBase and is similiar to TextLineReader/TFRecordReader) so that it is possible to reader Kafka streaming data from TensorFlow in a similiar fashion. This fix uses a C/C++ Apache Kafka client library librdkafka which is released under the 2-clause BSD license, and is widely used in a number of Kafka bindings such as Go, Python, C#/.Net, etc. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Add KafkaReader Python wrapper. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Add BUILD file and op registration for KafkaReader. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Add C++ Kernel for KafkaReader Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Add librdkafka to third_party packages in Bazel Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Add contrib/kafka to part of the contrib bazel file. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update workspace.bzl Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Comment out clean_deps of `tensorflow/core:framework` and `tensorflow/core:lib` so that it is possible to build with ReaderBase. See 1419 for details. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Add group id flag. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Sync offset Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Add test cases and scipt to start and stop Kafka server (with docker) Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Convert to KafkaConsumer from the legacy Consumer with librdkafka so that thread join does not hang. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Only output offset as the key. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Add timeout attr so that Kafka Consumer could use Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Build Kafka kernels by default, so that to get around the linkage issue. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Convert KafkaReader to KafkaDataset. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Fix workspace.bzl for kafka with tf_http_archive Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Add public visibility Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Address review feedbacks Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Optionally select Kafka support through ./configure Signed-off-by: Yong Tang <yong.tang.github@outlook.com>