diff options
| author |  Gael Guennebaud <g.gael@free.fr> | 2015-06-08 15:40:51 +0200 |
|---|---|---|
| committer | Gael Guennebaud <g.gael@free.fr> | 2015-06-08 15:40:51 +0200 |
| commit | 274b1f5d7ec2efff07712fec5157f1740ee78d4c (patch) | |
| tree | 7d641ac76888150ce6c0bc11a392f49402bc7c5d /Eigen | |
| parent | cbe3a1a83e811773687c0efaac080e388611252b (diff) | |
Fix homogeneous() for 1x1 matrix: in this case, homogeneous follows the storage order guaranteeing that v.transpose().homogeneous() == v.homogeneous().transpose()
Diffstat (limited to 'Eigen')
| -rw-r--r-- | Eigen/src/Core/MatrixBase.h | 3 |
1 files changed, 2 insertions, 1 deletions
diff --git a/Eigen/src/Core/MatrixBase.h b/Eigen/src/Core/MatrixBase.h index 5482b237e..81b900eee 100644 --- a/Eigen/src/Core/MatrixBase.h +++ b/Eigen/src/Core/MatrixBase.h @@ -410,7 +410,8 @@ template<typename Derived> class MatrixBase ScalarMultipleReturnType operator*(const UniformScaling<Scalar>& s) const; // put this as separate enum value to work around possible GCC 4.3 bug (?) - enum { HomogeneousReturnTypeDirection = ColsAtCompileTime==1?Vertical:Horizontal }; + enum { HomogeneousReturnTypeDirection = ColsAtCompileTime==1&&RowsAtCompileTime==1 ? ((internal::traits<Derived>::Flags&RowMajorBit)==RowMajorBit ? Horizontal : Vertical) + : ColsAtCompileTime==1 ? Vertical : Horizontal }; typedef Homogeneous<Derived, HomogeneousReturnTypeDirection> HomogeneousReturnType; HomogeneousReturnType homogeneous() const; |