tdf#32834 : improve the precision of MDETERM calculation
MDETERM uses lcl_LUP_decompose() which as the name suggests does LUP decomposition. This patch allows additive cancellations to occur in a cleaner way while doing the *only* additive operation in LUP decomposition, that is while computing Shur complement. This patch does not change the high level semantics of the algorithm. Also note that this change makes improvement only in the case where matrix elements entered by the user are *integers*. The change allows MDETERM to evaluate to exact 0.0 for singular integer matrices. The steps to calculate Shur complement are : for i = k+1 to n aik = aik / akk; for j = k+1 to n aij = aij - akj*aik This is now modified as : for i = k+1 to n for j = k+1 to n aij = ( aij*akk - akj*aik ) / akk Without this change MDETERM() for certain singular matrices used to evaluate to a tiny non zero value, which also caused MINVERSE() to generate a wrong output. An example of such a matrix is : 1 2 3 4 5 6 7 8 9 Change-Id: Idd4211ddceab1b758fd05bfd57f7eecd5d4fd1a0 Reviewed-on: https://gerrit.libreoffice.org/19534Reviewed-by:Eike Rathke <erack@redhat.com> Tested-by:
Eike Rathke <erack@redhat.com>
Showing
Please
register
or
sign in
to comment