|
`"pca": true` on the matrix request now folds the correlation matrix into
principal components: eigenvalues descending, explained variance ratios,
loadings per kept trait, and the eigenvalues' sum reported so a caller can
check the decomposition is sane instead of trusting it.
The eigendecomposition is cyclic Jacobi -- symmetric matrices, no LAPACK, no
new dependency -- capped at 300 traits because Jacobi is O(n^3) per sweep and
this is for a collection, not a dataset.
Two decisions from ai-memory #103, both visible in the reply:
* **Traits with any null cell are dropped and named**, with the reason. A
component from a matrix with holes is not defined; GN2 silently mutates its
shared-sample list instead, and nobody can reproduce that component.
* **The sign is fixed**: within a component the largest absolute loading is
positive. Eigenvectors are defined up to sign, so without a rule two correct
implementations disagree and no parity test could be exact.
Verified against numpy.linalg.eigh on the same r the reply carries
(scripts/pca-parity.py, BXDPublish 10001/10002/10003/10005/10007):
eigenvalues: worst relative difference 3.2e-14
loadings: worst absolute difference 9.99e-16
trace: 5.000000000000 for 5 traits
Unit tests cover the identity (unit eigenvalues), the closed form for 2x2
(1 +/- rho), the trace, a non-symmetric matrix being refused, and the cap. The
monitor gained `compute-correlation-pca`, whose judge checks that the
eigenvalues sum to the trait count and the ratios to 1.
|