diff options
| author |  Antonio Sanchez <cantonios@google.com> | 2021-07-01 13:41:04 -0700 |
|---|---|---|
| committer | Antonio Sanchez <cantonios@google.com> | 2021-07-01 13:41:04 -0700 |
| commit | 154f00e9eacaec5667215784c7601b55024e2f61 (patch) | |
| tree | f17d88a4084f103aac544f0799b60bd16f9cca6d /Eigen/src/LU/PartialPivLU.h | |
| parent | eb047759030558acf0764d5d2f913f4f84cf85a8 (diff) | |
Fix inverse nullptr/asan errors for LU.
For empty or single-column matrices, the current `PartialPivLU`
currently dereferences a `nullptr` or accesses memory out-of-bounds.
Here we adjust the checks to avoid this.
Diffstat (limited to 'Eigen/src/LU/PartialPivLU.h')
| -rw-r--r-- | Eigen/src/LU/PartialPivLU.h | 7 |
1 files changed, 6 insertions, 1 deletions
diff --git a/Eigen/src/LU/PartialPivLU.h b/Eigen/src/LU/PartialPivLU.h index 46ffdd320..34aed7249 100644 --- a/Eigen/src/LU/PartialPivLU.h +++ b/Eigen/src/LU/PartialPivLU.h @@ -504,8 +504,13 @@ struct partial_lu_impl template<typename MatrixType, typename TranspositionType> void partial_lu_inplace(MatrixType& lu, TranspositionType& row_transpositions, typename TranspositionType::StorageIndex& nb_transpositions) { + // Special-case of zero matrix. + if (lu.rows() == 0 || lu.cols() == 0) { + nb_transpositions = 0; + return; + } eigen_assert(lu.cols() == row_transpositions.size()); - eigen_assert((&row_transpositions.coeffRef(1)-&row_transpositions.coeffRef(0)) == 1); + eigen_assert(row_transpositions.size() < 2 || (&row_transpositions.coeffRef(1)-&row_transpositions.coeffRef(0)) == 1); partial_lu_impl < typename MatrixType::Scalar, MatrixType::Flags&RowMajorBit?RowMajor:ColMajor, |