Commit message (Collapse) | Author | Age | |
---|---|---|---|
* | Implement a generic vectorized version of Smith's algorithms for complex ↵ | Rasmus Munk Larsen | 2021-07-01 |
| | | | | division. | ||
* | Get rid of code duplication for conj_helper. For packets where ↵ | Rasmus Munk Larsen | 2021-06-24 |
| | | | | LhsType=RhsType a single generic implementation suffices. For scalars, the generic implementation of pconj automatically forwards to numext::conj, so much of the existing specialization can be avoided. For mixed types we still need specializations. | ||
* | Fix address of temporary object errors in clang11. | Chip Kerchner | 2021-04-02 |
| | | | | This fixes the problem with taking the address of temporary objects which clang11 treats as errors. | ||
* | Fixed performance issues for complex VSX and P10 MMA in gebp_kernel (level 3). | Chip Kerchner | 2021-03-25 |
| | |||
* | Fix sqrt, ldexp and frexp compilation errors. | Chip Kerchner | 2021-01-25 |
| | |||
* | Fix pscatter and pgather for Altivec Complex double | Pedro Caldeira | 2020-06-16 |
| | |||
* | Remove unused packet op "palign". | Rasmus Munk Larsen | 2020-05-07 |
| | | | | Clean up a compiler warning in c++03 mode in AVX512/Complex.h. | ||
* | Remove unused packet op "preduxp". | Rasmus Munk Larsen | 2020-04-23 |
| | |||
* | Add masked_store_available to unpacket_traits | Eugene Zhulenev | 2019-05-02 |
| | |||
* | Adding lowlevel APIs for optimized RHS packet load in TensorFlow | Anuj Rawat | 2019-04-20 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | SpatialConvolution Low-level APIs are added in order to optimized packet load in gemm_pack_rhs in TensorFlow SpatialConvolution. The optimization is for scenario when a packet is split across 2 adjacent columns. In this case we read it as two 'partial' packets and then merge these into 1. Currently this only works for Packet16f (AVX512) and Packet8f (AVX2). We plan to add this for other packet types (such as Packet8d) also. This optimization shows significant speedup in SpatialConvolution with certain parameters. Some examples are below. Benchmark parameters are specified as: Batch size, Input dim, Depth, Num of filters, Filter dim Speedup numbers are specified for number of threads 1, 2, 4, 8, 16. AVX512: Parameters | Speedup (Num of threads: 1, 2, 4, 8, 16) ----------------------------|------------------------------------------ 128, 24x24, 3, 64, 5x5 |2.18X, 2.13X, 1.73X, 1.64X, 1.66X 128, 24x24, 1, 64, 8x8 |2.00X, 1.98X, 1.93X, 1.91X, 1.91X 32, 24x24, 3, 64, 5x5 |2.26X, 2.14X, 2.17X, 2.22X, 2.33X 128, 24x24, 3, 64, 3x3 |1.51X, 1.45X, 1.45X, 1.67X, 1.57X 32, 14x14, 24, 64, 5x5 |1.21X, 1.19X, 1.16X, 1.70X, 1.17X 128, 128x128, 3, 96, 11x11 |2.17X, 2.18X, 2.19X, 2.20X, 2.18X AVX2: Parameters | Speedup (Num of threads: 1, 2, 4, 8, 16) ----------------------------|------------------------------------------ 128, 24x24, 3, 64, 5x5 | 1.66X, 1.65X, 1.61X, 1.56X, 1.49X 32, 24x24, 3, 64, 5x5 | 1.71X, 1.63X, 1.77X, 1.58X, 1.68X 128, 24x24, 1, 64, 5x5 | 1.44X, 1.40X, 1.38X, 1.37X, 1.33X 128, 24x24, 3, 64, 3x3 | 1.68X, 1.63X, 1.58X, 1.56X, 1.62X 128, 128x128, 3, 96, 11x11 | 1.36X, 1.36X, 1.37X, 1.37X, 1.37X In the higher level benchmark cifar10, we observe a runtime improvement of around 6% for AVX512 on Intel Skylake server (8 cores). On lower level PackRhs micro-benchmarks specified in TensorFlow tensorflow/core/kernels/eigen_spatial_convolutions_test.cc, we observe the following runtime numbers: AVX512: Parameters | Runtime without patch (ns) | Runtime with patch (ns) | Speedup ---------------------------------------------------------------|----------------------------|-------------------------|--------- BM_RHS_NAME(PackRhs, 128, 24, 24, 3, 64, 5, 5, 1, 1, 256, 56) | 41350 | 15073 | 2.74X BM_RHS_NAME(PackRhs, 32, 64, 64, 32, 64, 5, 5, 1, 1, 256, 56) | 7277 | 7341 | 0.99X BM_RHS_NAME(PackRhs, 32, 64, 64, 32, 64, 5, 5, 2, 2, 256, 56) | 8675 | 8681 | 1.00X BM_RHS_NAME(PackRhs, 32, 64, 64, 30, 64, 5, 5, 1, 1, 256, 56) | 24155 | 16079 | 1.50X BM_RHS_NAME(PackRhs, 32, 64, 64, 30, 64, 5, 5, 2, 2, 256, 56) | 25052 | 17152 | 1.46X BM_RHS_NAME(PackRhs, 32, 256, 256, 4, 16, 8, 8, 1, 1, 256, 56) | 18269 | 18345 | 1.00X BM_RHS_NAME(PackRhs, 32, 256, 256, 4, 16, 8, 8, 2, 4, 256, 56) | 19468 | 19872 | 0.98X BM_RHS_NAME(PackRhs, 32, 64, 64, 4, 16, 3, 3, 1, 1, 36, 432) | 156060 | 42432 | 3.68X BM_RHS_NAME(PackRhs, 32, 64, 64, 4, 16, 3, 3, 2, 2, 36, 432) | 132701 | 36944 | 3.59X AVX2: Parameters | Runtime without patch (ns) | Runtime with patch (ns) | Speedup ---------------------------------------------------------------|----------------------------|-------------------------|--------- BM_RHS_NAME(PackRhs, 128, 24, 24, 3, 64, 5, 5, 1, 1, 256, 56) | 26233 | 12393 | 2.12X BM_RHS_NAME(PackRhs, 32, 64, 64, 32, 64, 5, 5, 1, 1, 256, 56) | 6091 | 6062 | 1.00X BM_RHS_NAME(PackRhs, 32, 64, 64, 32, 64, 5, 5, 2, 2, 256, 56) | 7427 | 7408 | 1.00X BM_RHS_NAME(PackRhs, 32, 64, 64, 30, 64, 5, 5, 1, 1, 256, 56) | 23453 | 20826 | 1.13X BM_RHS_NAME(PackRhs, 32, 64, 64, 30, 64, 5, 5, 2, 2, 256, 56) | 23167 | 22091 | 1.09X BM_RHS_NAME(PackRhs, 32, 256, 256, 4, 16, 8, 8, 1, 1, 256, 56) | 23422 | 23682 | 0.99X BM_RHS_NAME(PackRhs, 32, 256, 256, 4, 16, 8, 8, 2, 4, 256, 56) | 23165 | 23663 | 0.98X BM_RHS_NAME(PackRhs, 32, 64, 64, 4, 16, 3, 3, 1, 1, 36, 432) | 72689 | 44969 | 1.62X BM_RHS_NAME(PackRhs, 32, 64, 64, 4, 16, 3, 3, 2, 2, 36, 432) | 61732 | 39779 | 1.55X All benchmarks on Intel Skylake server with 8 cores. | ||
* | Collapsed revision from PR-619 | William D. Irons | 2019-03-26 |
| | | | | | | | * Add support for pcmp_eq in AltiVec/Complex.h * Fixed implementation of pcmp_eq for double The new logic is based on the logic from NEON for double. | ||
* | Fix conflicts and merge | Gael Guennebaud | 2019-01-30 |
|\ | |||
* | | bug #1652: fix position of EIGEN_ALIGN16 attributes in Neon and Altivec | Gael Guennebaud | 2019-01-14 |
| | | |||
| * | Introducing "vectorized" byte on unpacket_traits structs | Gustavo Lima Chaves | 2018-12-19 |
|/ | | | | | | | | | | | | | | | | | | | | | This is a preparation to a change on gebp_traits, where a new template argument will be introduced to dictate the packet size, so it won't be bound to the current/max packet size only anymore. By having packet types defined early on gebp_traits, one has now to act on packet types, not scalars anymore, for the enum values defined on that class. One approach for reaching the vectorizable/size properties one needs there could be getting the packet's scalar again with unpacket_traits<>, then the size/Vectorizable enum entries from packet_traits<>. It turns out guards like "#ifndef EIGEN_VECTORIZE_AVX512" at AVX/PacketMath.h will hide smaller packet variations of packet_traits<> for some types (and it makes sense to keep that). In other words, one can't go back to the scalar and create a new PacketType, as this will always lead to the maximum packet type for the architecture. The less costly/invasive solution for that, thus, is to add the vectorizable info on every unpacket_traits struct as well. | ||
* | bug #1436: fix compilation of Jacobi rotations with ARM NEON, some ↵ | Gael Guennebaud | 2017-06-15 |
| | | | | specializations of internal::conj_helper were missing. | ||
* | Add std:: namespace prefix to all (hopefully) instances if size_t/ptrdfiff_t | Gael Guennebaud | 2017-01-23 |
| | |||
* | bug #1360: fix sign issue with pmull on altivec | Gael Guennebaud | 2016-12-18 |
| | |||
* | minor fixes for big endian altivec/vsx | Konstantinos Margaritis | 2016-07-10 |
| | |||
* | fix compilation with clang 3.9, fix performance with pset1, use vector ↵ | Konstantinos Margaritis | 2016-06-23 |
| | | | | operators instead of intrinsics in some cases | ||
* | mostly cleanups and modernizing code | Konstantinos Margaritis | 2016-06-19 |
| | |||
* | minor fix, add to copyright | Konstantinos Margaritis | 2016-04-28 |
| | |||
* | Let unpacket_traits<> exposes the required alignment and make use of it ↵ | Gael Guennebaud | 2015-08-07 |
| | | | | everywhere | ||
* | Fix bug in pdiv<Packet1cd> which swaps 32-bit halves of a pair of | Doug Kwan | 2015-03-11 |
| | | | | doubles instead of swapping the doubles. | ||
* | The usage of DenseIndex is deprecated, so let's replace DenseIndex by Index | Gael Guennebaud | 2015-02-16 |
| | |||
* | fixed to make big-endian VSX work as well | Konstantinos Margaritis | 2014-10-01 |
| | |||
* | fix compile error on big endian altivec | Konstantinos Margaritis | 2014-09-21 |
| | |||
* | VSX port passes packetmath_[1-5] tests! | Konstantinos Margaritis | 2014-09-20 |
| | |||
* | 32-bit floats/ints, 64-bit doubles pass packetmath tests, complex 32/64-bit ↵ | Konstantinos Margaritis | 2014-09-19 |
| | | | | remaining | ||
* | First time it compiles, but fails to pass the tests. | Konstantinos Margaritis | 2014-09-09 |
| | |||
* | Initial VSX commit | Konstantinos Margaritis | 2014-08-29 |
| | |||
* | Simplification of some Altivec constants, reuse existing constants and avoid ↵ | Konstantinos Margaritis | 2014-07-22 |
| | | | | loading from RAM esp in the case of p16uc_COMPLEX_TRANSPOSE* | ||
* | Fix many long to int implicit conversions | Gael Guennebaud | 2014-07-08 |
| | |||
* | Implement pbroadcast4 on altivec | Gael Guennebaud | 2014-04-25 |
| | |||
* | Enable vectorization of pack_rhs with a column-major RHS. | Gael Guennebaud | 2014-04-25 |
| | | | | Rename and generalize Kernel<*> to PacketBlock<*,N>. | ||
* | Implement ptranspose on altivec and fix pgather/pscatter | Gael Guennebaud | 2014-04-24 |
| | |||
* | Add Altivec implementation of pgather/pscatter (not tested) | Gael Guennebaud | 2014-04-23 |
| | |||
* | Add a mechanism to recursively access to half-size packet types | Gael Guennebaud | 2014-03-28 |
| | |||
* | Automatic relicensing to MPL2 using Keirs script. Manual fixup follows. | Benoit Jacob | 2012-07-13 |
| | |||
* | Get rid of include directives inside namespace blocks (bug #339). | Jitse Niesen | 2012-04-15 |
| | |||
* | fix static inline versus inline static issues (the former is the correct order) | Gael Guennebaud | 2012-01-31 |
| | |||
* | better fix for gcc 4.6.0 / ptrdiff_t, as suggested by Benoit | Thomas Capricelli | 2011-05-05 |
| | |||
* | Fix compilation with gcc-4.6.0, patch provided by Anton Gladky ↵ | Thomas Capricelli | 2011-05-05 |
| | | | | | | <gladky.anton@gmail.com>, working on debian packaging. | ||
* | fix pset1 for complex | Gael Guennebaud | 2011-02-23 |
| | |||
* | implement ploaddup for altivec and add respective unit test | Gael Guennebaud | 2011-02-23 |
| | |||
* | fix a couple of issues with pcplxflip | Gael Guennebaud | 2011-02-23 |
| | |||
* | implement pcplxflip for altivec | Gael Guennebaud | 2011-02-23 |
| | |||
* | Altivec: fix infinite loop (ei_ -> internal:: change) | Gael Guennebaud | 2011-02-23 |
| | |||
* | add missing AlignedOnScalar | Gael Guennebaud | 2011-02-22 |
| | |||
* | altivec compilation fix | Gael Guennebaud | 2011-02-22 |
| | |||
* | more compilation fixes for altivec | Gael Guennebaud | 2011-02-21 |
| |