Commit message (Collapse) | Author | Age | |
---|---|---|---|
* | STYLE: Remove CMake-language block-end command arguments | Hans Johnson | 2019-10-31 |
| | | | | | | Ancient versions of CMake required else(), endif(), and similar block termination commands to have arguments matching the command starting the block. This is no longer the preferred style. | ||
* | 1. Fix a bug in psqrt and make it return 0 for +inf arguments. | Rasmus Munk Larsen | 2019-11-15 |
| | | | | | | | | | | | | | | | | 2. Simplify handling of special cases by taking advantage of the fact that the builtin vrsqrt approximation handles negative, zero and +inf arguments correctly. This speeds up the SSE and AVX implementations by ~20%. 3. Make the Newton-Raphson formula used for rsqrt more numerically robust: Before: y = y * (1.5 - x/2 * y^2) After: y = y * (1.5 - y * (x/2) * y) Forming y^2 can overflow for very large or very small (denormalized) values of x, while x*y ~= 1. For AVX512, this makes it possible to compute accurate results for denormal inputs down to ~1e-42 in single precision. 4. Add a faster double precision implementation for Knights Landing using the vrsqrt28 instruction and a single Newton-Raphson iteration. Benchmark results: https://bitbucket.org/snippets/rmlarsen/5LBq9o | ||
* | bug #1774: fix VectorwiseOp::begin()/end() return types regarding constness. | Gael Guennebaud | 2019-11-14 |
| | |||
* | Enable CompleteOrthogonalDecomposition::pseudoInverse with non-square ↵ | Gael Guennebaud | 2019-11-13 |
| | | | | fixed-size matrices. | ||
* | bug #1752: make is_convertible equivalent to the std c++11 equivalent and ↵ | Gael Guennebaud | 2019-10-10 |
| | | | | fallback to std::is_convertible when c++11 is enabled. | ||
* | bug #1752: disable some is_convertible tests for recent compilers. | Gael Guennebaud | 2019-10-10 |
| | |||
* | Disable an expected warning in unit test | Gael Guennebaud | 2019-10-08 |
| | |||
* | PR 719: fix real/imag namespace conflict | Gael Guennebaud | 2019-10-08 |
| | |||
* | Merged in rmlarsen/eigen (pull request PR-704) | Rasmus Larsen | 2019-09-24 |
|\ | | | | | | | Add generic PacketMath implementation of the Error Function (erf). | ||
* | | bug #1746: Removed implementation of standard copy-constructor and standard ↵ | Christoph Hertzberg | 2019-09-24 |
| | | | | | | | | copy-assign-operator from PermutationMatrix and Transpositions to allow malloc-less std::move. Added unit-test to rvalue_types | ||
| * | Add generic PacketMath implementation of the Error Function (erf). | Rasmus Munk Larsen | 2019-09-19 |
|/ | |||
* | Merging eigen/eigen. | Srinivas Vasudevan | 2019-09-16 |
|\ | |||
* | | Add Bessel functions to SpecialFunctions. | Srinivas Vasudevan | 2019-09-14 |
|/ | | | | | | | | | - Split SpecialFunctions files in to a separate BesselFunctions file. In particular add: - Modified bessel functions of the second kind k0, k1, k0e, k1e - Bessel functions of the first kind j0, j1 - Bessel functions of the second kind y0, y1 | ||
* | Add packetized versions of i0e and i1e special functions. | Srinivas Vasudevan | 2019-09-11 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - In particular refactor the i0e and i1e code so scalar and vectorized path share code. - Move chebevl to GenericPacketMathFunctions. A brief benchmark with building Eigen with FMA, AVX and AVX2 flags Before: CPU: Intel Haswell with HyperThreading (6 cores) Benchmark Time(ns) CPU(ns) Iterations ----------------------------------------------------------------- BM_eigen_i0e_double/1 57.3 57.3 10000000 BM_eigen_i0e_double/8 398 398 1748554 BM_eigen_i0e_double/64 3184 3184 218961 BM_eigen_i0e_double/512 25579 25579 27330 BM_eigen_i0e_double/4k 205043 205042 3418 BM_eigen_i0e_double/32k 1646038 1646176 422 BM_eigen_i0e_double/256k 13180959 13182613 53 BM_eigen_i0e_double/1M 52684617 52706132 10 BM_eigen_i0e_float/1 28.4 28.4 24636711 BM_eigen_i0e_float/8 75.7 75.7 9207634 BM_eigen_i0e_float/64 512 512 1000000 BM_eigen_i0e_float/512 4194 4194 166359 BM_eigen_i0e_float/4k 32756 32761 21373 BM_eigen_i0e_float/32k 261133 261153 2678 BM_eigen_i0e_float/256k 2087938 2088231 333 BM_eigen_i0e_float/1M 8380409 8381234 84 BM_eigen_i1e_double/1 56.3 56.3 10000000 BM_eigen_i1e_double/8 397 397 1772376 BM_eigen_i1e_double/64 3114 3115 223881 BM_eigen_i1e_double/512 25358 25361 27761 BM_eigen_i1e_double/4k 203543 203593 3462 BM_eigen_i1e_double/32k 1613649 1613803 428 BM_eigen_i1e_double/256k 12910625 12910374 54 BM_eigen_i1e_double/1M 51723824 51723991 10 BM_eigen_i1e_float/1 28.3 28.3 24683049 BM_eigen_i1e_float/8 74.8 74.9 9366216 BM_eigen_i1e_float/64 505 505 1000000 BM_eigen_i1e_float/512 4068 4068 171690 BM_eigen_i1e_float/4k 31803 31806 21948 BM_eigen_i1e_float/32k 253637 253692 2763 BM_eigen_i1e_float/256k 2019711 2019918 346 BM_eigen_i1e_float/1M 8238681 8238713 86 After: CPU: Intel Haswell with HyperThreading (6 cores) Benchmark Time(ns) CPU(ns) Iterations ----------------------------------------------------------------- BM_eigen_i0e_double/1 15.8 15.8 44097476 BM_eigen_i0e_double/8 99.3 99.3 7014884 BM_eigen_i0e_double/64 777 777 886612 BM_eigen_i0e_double/512 6180 6181 100000 BM_eigen_i0e_double/4k 48136 48140 14678 BM_eigen_i0e_double/32k 385936 385943 1801 BM_eigen_i0e_double/256k 3293324 3293551 228 BM_eigen_i0e_double/1M 12423600 12424458 57 BM_eigen_i0e_float/1 16.3 16.3 43038042 BM_eigen_i0e_float/8 30.1 30.1 23456931 BM_eigen_i0e_float/64 169 169 4132875 BM_eigen_i0e_float/512 1338 1339 516860 BM_eigen_i0e_float/4k 10191 10191 68513 BM_eigen_i0e_float/32k 81338 81337 8531 BM_eigen_i0e_float/256k 651807 651984 1000 BM_eigen_i0e_float/1M 2633821 2634187 268 BM_eigen_i1e_double/1 16.2 16.2 42352499 BM_eigen_i1e_double/8 110 110 6316524 BM_eigen_i1e_double/64 822 822 851065 BM_eigen_i1e_double/512 6480 6481 100000 BM_eigen_i1e_double/4k 51843 51843 10000 BM_eigen_i1e_double/32k 414854 414852 1680 BM_eigen_i1e_double/256k 3320001 3320568 212 BM_eigen_i1e_double/1M 13442795 13442391 53 BM_eigen_i1e_float/1 17.6 17.6 41025735 BM_eigen_i1e_float/8 35.5 35.5 19597891 BM_eigen_i1e_float/64 240 240 2924237 BM_eigen_i1e_float/512 1424 1424 485953 BM_eigen_i1e_float/4k 10722 10723 65162 BM_eigen_i1e_float/32k 86286 86297 8048 BM_eigen_i1e_float/256k 691821 691868 1000 BM_eigen_i1e_float/1M 2777336 2777747 256 This shows anywhere from a 50% to 75% improvement on these operations. I've also benchmarked without any of these flags turned on, and got similar performance to before (if not better). Also tested packetmath.cpp + special_functions to ensure no regressions. | ||
* | bug #1736: fix compilation issue with A(all,{1,2}).col(j) by implementing ↵ | Gael Guennebaud | 2019-09-11 |
| | | | | true compile-time "if" for block_evaluator<>::coeff(i)/coeffRef(i) | ||
* | bug #1741: fix self-adjoint*matrix, triangular*matrix, and ↵ | Gael Guennebaud | 2019-09-11 |
| | | | | triangular^1*matrix with a destination having a non-trivial inner-stride | ||
* | bug #1741: fix SelfAdjointView::rankUpdate and product to triangular part ↵ | Gael Guennebaud | 2019-09-10 |
| | | | | for destination with non-trivial inner stride | ||
* | bug #1741: fix C.noalias() = A*C; with C.innerStride()!=1 | Gael Guennebaud | 2019-09-10 |
| | |||
* | PR 681: Add ndtri function, the inverse of the normal distribution function. | Srinivas Vasudevan | 2019-08-12 |
| | |||
* | Add more tests for corner cases of log1p and expm1. Add handling of infinite ↵ | Rasmus Munk Larsen | 2019-08-28 |
| | | | | arguments to log1p such that log1p(inf) = inf. | ||
* | Revert changes to std_falback::log1p that broke handling of arguments less ↵ | Rasmus Munk Larsen | 2019-08-27 |
| | | | | than -1. Fix packet op accordingly. | ||
* | Clean up float16 a.k.a. Eigen::half support in Eigen. Move the definition of ↵ | Rasmus Munk Larsen | 2019-08-27 |
| | | | | half to Core/arch/Default and move arch-specific packet ops to their respective sub-directories. | ||
* | Implement vectorized versions of log1p and expm1 in Eigen using Kahan's ↵ | Rasmus Munk Larsen | 2019-08-12 |
| | | | | | | | | | | | | formulas, and change the scalar implementations to properly handle infinite arguments. Depending on instruction set, significant speedups are observed for the vectorized path: log1p wall time is reduced 60-93% (2.5x - 15x speedup) expm1 wall time is reduced 0-85% (1x - 7x speedup) The scalar path is slower by 20-30% due to the extra branch needed to handle +infinity correctly. Full benchmarks measured on Intel(R) Xeon(R) Gold 6154 here: https://bitbucket.org/snippets/rmlarsen/MXBkpM | ||
* | Guard against repeated definition of EIGEN_MPL2_ONLY | Rasmus Munk Larsen | 2019-08-07 |
| | |||
* | [SYCL] This PR adds the minimum modifications to Eigen core required to run ↵ | Mehdi Goli | 2019-06-27 |
| | | | | | | | | Eigen unsupported modules on devices supporting SYCL. * Adding SYCL memory model * Enabling/Disabling SYCL backend in Core * Supporting Vectorization | ||
* | Various fixes for packet ops. | Rasmus Munk Larsen | 2019-06-20 |
| | | | | | | 1. Fix buggy pcmp_eq and unit test for half types. 2. Add unit test for pselect and add specializations for SSE 4.1, AVX512, and half types. 3. Get rid of FIXME: Implement faster pnegate for half by XOR'ing with a sign bit mask. | ||
* | Merged in Artem-B/eigen (pull request PR-654) | Rasmus Larsen | 2019-05-31 |
|\ | | | | | | | | | | | Minor build improvements Approved-by: Rasmus Larsen <rmlarsen@google.com> | ||
* | | Clean up CUDA/NVCC version macros and their use in Eigen, and a few other ↵ | Rasmus Munk Larsen | 2019-05-31 |
| | | | | | | | | CUDA build failures. | ||
| * | Minor build improvements | tra | 2019-05-31 |
|/ | | | | | | | | * Allow specifying multiple GPU architectures. E.g.: cmake -DEIGEN_CUDA_COMPUTE_ARCH="60;70" * Pass CUDA SDK path to clang. Without it it will default to /usr/local/cuda which may not be the right location, if cmake was invoked with -DCUDA_TOOLKIT_ROOT_DIR=/some/other/CUDA/path | ||
* | bug #1707: Fix deprecation warnings, or disable warnings when testing ↵ | Christoph Hertzberg | 2019-05-10 |
| | | | | deprecated functions | ||
* | Add masked_store_available to unpacket_traits | Eugene Zhulenev | 2019-05-02 |
| | |||
* | Add masked pstoreu to AVX and AVX512 PacketMath | Eugene Zhulenev | 2019-05-02 |
| | |||
* | Adding lowlevel APIs for optimized RHS packet load in TensorFlow | Anuj Rawat | 2019-04-20 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | SpatialConvolution Low-level APIs are added in order to optimized packet load in gemm_pack_rhs in TensorFlow SpatialConvolution. The optimization is for scenario when a packet is split across 2 adjacent columns. In this case we read it as two 'partial' packets and then merge these into 1. Currently this only works for Packet16f (AVX512) and Packet8f (AVX2). We plan to add this for other packet types (such as Packet8d) also. This optimization shows significant speedup in SpatialConvolution with certain parameters. Some examples are below. Benchmark parameters are specified as: Batch size, Input dim, Depth, Num of filters, Filter dim Speedup numbers are specified for number of threads 1, 2, 4, 8, 16. AVX512: Parameters | Speedup (Num of threads: 1, 2, 4, 8, 16) ----------------------------|------------------------------------------ 128, 24x24, 3, 64, 5x5 |2.18X, 2.13X, 1.73X, 1.64X, 1.66X 128, 24x24, 1, 64, 8x8 |2.00X, 1.98X, 1.93X, 1.91X, 1.91X 32, 24x24, 3, 64, 5x5 |2.26X, 2.14X, 2.17X, 2.22X, 2.33X 128, 24x24, 3, 64, 3x3 |1.51X, 1.45X, 1.45X, 1.67X, 1.57X 32, 14x14, 24, 64, 5x5 |1.21X, 1.19X, 1.16X, 1.70X, 1.17X 128, 128x128, 3, 96, 11x11 |2.17X, 2.18X, 2.19X, 2.20X, 2.18X AVX2: Parameters | Speedup (Num of threads: 1, 2, 4, 8, 16) ----------------------------|------------------------------------------ 128, 24x24, 3, 64, 5x5 | 1.66X, 1.65X, 1.61X, 1.56X, 1.49X 32, 24x24, 3, 64, 5x5 | 1.71X, 1.63X, 1.77X, 1.58X, 1.68X 128, 24x24, 1, 64, 5x5 | 1.44X, 1.40X, 1.38X, 1.37X, 1.33X 128, 24x24, 3, 64, 3x3 | 1.68X, 1.63X, 1.58X, 1.56X, 1.62X 128, 128x128, 3, 96, 11x11 | 1.36X, 1.36X, 1.37X, 1.37X, 1.37X In the higher level benchmark cifar10, we observe a runtime improvement of around 6% for AVX512 on Intel Skylake server (8 cores). On lower level PackRhs micro-benchmarks specified in TensorFlow tensorflow/core/kernels/eigen_spatial_convolutions_test.cc, we observe the following runtime numbers: AVX512: Parameters | Runtime without patch (ns) | Runtime with patch (ns) | Speedup ---------------------------------------------------------------|----------------------------|-------------------------|--------- BM_RHS_NAME(PackRhs, 128, 24, 24, 3, 64, 5, 5, 1, 1, 256, 56) | 41350 | 15073 | 2.74X BM_RHS_NAME(PackRhs, 32, 64, 64, 32, 64, 5, 5, 1, 1, 256, 56) | 7277 | 7341 | 0.99X BM_RHS_NAME(PackRhs, 32, 64, 64, 32, 64, 5, 5, 2, 2, 256, 56) | 8675 | 8681 | 1.00X BM_RHS_NAME(PackRhs, 32, 64, 64, 30, 64, 5, 5, 1, 1, 256, 56) | 24155 | 16079 | 1.50X BM_RHS_NAME(PackRhs, 32, 64, 64, 30, 64, 5, 5, 2, 2, 256, 56) | 25052 | 17152 | 1.46X BM_RHS_NAME(PackRhs, 32, 256, 256, 4, 16, 8, 8, 1, 1, 256, 56) | 18269 | 18345 | 1.00X BM_RHS_NAME(PackRhs, 32, 256, 256, 4, 16, 8, 8, 2, 4, 256, 56) | 19468 | 19872 | 0.98X BM_RHS_NAME(PackRhs, 32, 64, 64, 4, 16, 3, 3, 1, 1, 36, 432) | 156060 | 42432 | 3.68X BM_RHS_NAME(PackRhs, 32, 64, 64, 4, 16, 3, 3, 2, 2, 36, 432) | 132701 | 36944 | 3.59X AVX2: Parameters | Runtime without patch (ns) | Runtime with patch (ns) | Speedup ---------------------------------------------------------------|----------------------------|-------------------------|--------- BM_RHS_NAME(PackRhs, 128, 24, 24, 3, 64, 5, 5, 1, 1, 256, 56) | 26233 | 12393 | 2.12X BM_RHS_NAME(PackRhs, 32, 64, 64, 32, 64, 5, 5, 1, 1, 256, 56) | 6091 | 6062 | 1.00X BM_RHS_NAME(PackRhs, 32, 64, 64, 32, 64, 5, 5, 2, 2, 256, 56) | 7427 | 7408 | 1.00X BM_RHS_NAME(PackRhs, 32, 64, 64, 30, 64, 5, 5, 1, 1, 256, 56) | 23453 | 20826 | 1.13X BM_RHS_NAME(PackRhs, 32, 64, 64, 30, 64, 5, 5, 2, 2, 256, 56) | 23167 | 22091 | 1.09X BM_RHS_NAME(PackRhs, 32, 256, 256, 4, 16, 8, 8, 1, 1, 256, 56) | 23422 | 23682 | 0.99X BM_RHS_NAME(PackRhs, 32, 256, 256, 4, 16, 8, 8, 2, 4, 256, 56) | 23165 | 23663 | 0.98X BM_RHS_NAME(PackRhs, 32, 64, 64, 4, 16, 3, 3, 1, 1, 36, 432) | 72689 | 44969 | 1.62X BM_RHS_NAME(PackRhs, 32, 64, 64, 4, 16, 3, 3, 2, 2, 36, 432) | 61732 | 39779 | 1.55X All benchmarks on Intel Skylake server with 8 cores. | ||
* | fix unit test in c++03: c++03 does not allow passing local or anonymous enum ↵ | Gael Guennebaud | 2019-03-18 |
| | | | | as template param | ||
* | bug #1692: enable enum as sizes of Matrix and Array | Gael Guennebaud | 2019-03-17 |
| | |||
* | Remove undefined std::complex<int> | David Tellenbach | 2019-03-14 |
| | |||
* | PR 593: Add variadtic ctor for DiagonalMatrix with unit tests | David Tellenbach | 2019-03-14 |
| | |||
* | revert debug stuff | Gael Guennebaud | 2019-03-14 |
| | |||
* | bug #1684: partially workaround clang's 6/7 bug #40815 | Gael Guennebaud | 2019-03-13 |
| | |||
* | Enable construction of Ref<VectorType> from a runtime vector. | Gael Guennebaud | 2019-03-03 |
| | |||
* | bug #1684: add simplified regression test for respective clang's bug (this ↵ | Gael Guennebaud | 2019-02-22 |
| | | | | also reveal the same bug in Apples's clang) | ||
* | Add fully generic Vector<Type,Size> and RowVector<Type,Size> type aliases. | Gael Guennebaud | 2019-02-20 |
| | |||
* | Protect c++11 type alias with Eigen's macro, and add respective unit test. | Gael Guennebaud | 2019-02-20 |
| | |||
* | Fix compilation with gcc and remove TR1 stuff. | Gael Guennebaud | 2019-02-20 |
| | |||
* | bug #1409: make EIGEN_MAKE_ALIGNED_OPERATOR_NEW* macros empty in c++17 mode: | Gael Guennebaud | 2019-02-20 |
| | | | | | - this helps clang 5 and 6 to support alignas in STL's containers. - this makes the public API of our (and users) classes cleaner | ||
* | bug #899: make sparseqr unit test more stable by 1) trying with larger ↵ | Gael Guennebaud | 2019-02-19 |
| | | | | threshold and 2) relax rank computation for rank-deficient problems. | ||
* | Fix C++17 compilation | Gael Guennebaud | 2019-02-19 |
| | |||
* | fix unit compilation in c++17: std::ptr_fun has been removed. | Gael Guennebaud | 2019-02-19 |
| | |||
* | bug #1046: add unit tests for correct propagation of alignment through ↵ | Gael Guennebaud | 2019-02-19 |
| | | | | std::alignment_of | ||
* | Add unit test for LinSpaced and complex numbers. | Gael Guennebaud | 2019-02-18 |
| |