aboutsummaryrefslogtreecommitdiffhomepage
path: root/tensorflow/core/kernels/batching_util
Commit message (Collapse)AuthorAge
* Automated rollback of commit 6a787235b95dd3040fc5ff7fb7104585e746c66aGravatar Christopher Olston2018-10-01
| | | | PiperOrigin-RevId: 215248737
* Public no-op.Gravatar Christopher Olston2018-09-26
| | | | PiperOrigin-RevId: 214648140
* Public no-op.Gravatar Christopher Olston2018-09-26
| | | | PiperOrigin-RevId: 214643933
* Adds a mechanism for adaptive shared batch scheduler that the earliest ↵Gravatar A. Unique TensorFlower2018-07-25
| | | | | | | | closed batches are scheduled immediately if batch_thread_pool_ has an idle thead. This helps to prevent overloads when traffic suddenly increase from very low to high. PiperOrigin-RevId: 206077249
* Fix flaky serial_device_batch_scheduler_test.Gravatar A. Unique TensorFlower2018-06-22
| | | | PiperOrigin-RevId: 201721119
* Disable flaky serial_device_batch_scheduler_testGravatar A. Unique TensorFlower2018-06-20
| | | | PiperOrigin-RevId: 201429850
* Add SerialDeviceBatchScheduler which offers similar performance as the ↵Gravatar A. Unique TensorFlower2018-05-30
| | | | | | | | | | | | AdaptiveSharedBatchScheduler, but increased reliablility and stability. ASBS assumes request latency can be minimized at a specific number of batch processing threads. Under reasonable load, this is true and ASBS performs well, but under low load latency is basically unaffected by the number of threads, and ASBS can learn a wide variety of 'optimal' values. If load resumes suddenly, these values can give very poor latencies. In most cases, ASBS will recover, eventually rediscovering the correct value, but we have observed other cases where the latency is so large and noisy that ASBS can't get a good signal to guide its learning and the number of threads remains stuck at the bad value. In addition, the incremental learning nature of this algorithm means that ASBS is always exploring to some extent, which can give rise to periods of non-optimal latency. This is most significant at high utilization where the wrong number of threads can potentially overload the system. ASBS uses latency as a proxy for keeping the tensorflow processing pipeline optimally loaded. SDBS, on the other hand, uses a direct measurement of the pipeline fullness, and adjusts its number of batch processing threads accordingly. This solves the exploration problem. SDBS solves the low load problem by not adjusting its thread count when the threads pass some idleness threshold. PiperOrigin-RevId: 198638918
* Merge changes from github.Gravatar Yifei Feng2018-05-24
| | | | | | | Revert #18413. Too many internal test failures due to the name scope change caused by this change. Revert #18192. Cannot use re2::StringPiece internally. Need alternative for set call. Will pull and clean this up in a separate change. PiperOrigin-RevId: 197991247
* Add a parameter to the adaptive shared batcher which allows the user to set ↵Gravatar A. Unique TensorFlower2018-05-16
| | | | | | | | a lower bound for in_flight_batches_limit. This can help prevent overloads which may occur during large traffic shifts - a small value learned during a period of low load can be unsuitable at high load. PiperOrigin-RevId: 196893320
* Merge changes from github.Gravatar Yifei Feng2018-04-23
| | | | PiperOrigin-RevId: 194031845
* Internal ChangeGravatar A. Unique TensorFlower2018-04-23
| | | | PiperOrigin-RevId: 193929733
* Automated g4 rollback of changelist 192768744Gravatar A. Unique TensorFlower2018-04-13
| | | | PiperOrigin-RevId: 192770717
* Split byte_order.h off cpu_info.hGravatar A. Unique TensorFlower2018-04-13
| | | | PiperOrigin-RevId: 192768744
* Remove all_opensource_files. It's not needed any more.Gravatar Martin Wicke2018-03-28
| | | | PiperOrigin-RevId: 190878279
* Add a configurable preference for scheduling fuller batches sooner to the ↵Gravatar A. Unique TensorFlower2018-03-02
| | | | | | | | adaptive shared batcher. A full batch will now be scheduled before an older, nearly empty batch as long as the age gap is less than full_batch_scheduling_boost_micros. This parameter improves latency under heavy load, but too large a value will harm tail latency. PiperOrigin-RevId: 187644796
* Remove old implementation of the adaptive shared batcher, the in flight ↵Gravatar A. Unique TensorFlower2018-02-26
| | | | | | batches implemntation delivers similar performance but is simpler and requires less tuning. PiperOrigin-RevId: 187111685
* Cleanup: Ran clang-format on files in tensorflow/core/.../*.{cc,h}.Gravatar A. Unique TensorFlower2018-01-30
| | | | PiperOrigin-RevId: 183848459
* Remove THIRD_PARTY_ from #include guardsGravatar Sanjoy Das2018-01-24
| | | | | | They don't make sense in the open source repository. PiperOrigin-RevId: 183140889
* Reorganize the shared batching scheduler headers, leaving forwarding shims.Gravatar Alexandre Passos2018-01-12
PiperOrigin-RevId: 181795909
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-26 03:50:31 +0000