Golub Kahan Bidiagonalization and Corresponding Tests

[etontackett]

Add Golub Kahan Bidiagonalization, Givens Rotations, and corresponding tests.

Description

This pull request introduces bidiagonalization.jl, which implements Golub Kahan Bidiagonalization using a sequence of Givens rotations. This implementation includes functions for computing Givens rotations coefficients, applying transformations to the rows and columns, and performing bidiagonalization while accumulating the orthogonal matrices H and K. The algorithm transforms A into an upper bidiagonal form satisfying H^T * A * K = B, and applies the transformations to the constant vector and matrix L. Tests were added to cover rectangular matrices and several basic numerical cases to ensure the implementation behaves correctly. Additional tests were added to verify orthogonality and other structural properties.

Motivation and Context

This change adds the implementation of a direct method for solving ridge regression problems. In ridge regression, we aim to solve the regularized least squares problem. The approach used to solve this is to reduce the A matrix into bidiagonal form by applying right and left orthogonal transformations. Golub Kahan Bidiagonalization is a method used for performing this reduction. Once A is in bidiagonal form, the resulting system becomes much easier to solve and analyze.

Types of changes

• CI
• Docs
• Feature
• Fix
• Performance
• Refactor
• Style
• Test
• Other (use sparingly):

Checklists:

Code and Comments
If this PR includes modifications to the code base, please select all that apply.

• My code follows the code style of this project.
• I have updated all package dependencies (if any).
• I have included all relevant files to realize the functionality of the PR.
• I have exported relevant functionality (if any).

API Documentation

• For every exported function (if any), I have included a detailed docstring.
• I have checked the spelling and grammar of all docstring updates through an external tool.
• I have checked that the docstring's function signature is correctly formatted and has all arguments.
• I have checked that the docstring's list of arguments, fields, or return values match the function.
• I have compiled the docs locally and read through all docstring updates to check for errors.

Manual Documentation

• I have checked the spelling and grammar of all manual updates through an external tool.
• Any code included in the docstring is tested using doc tests to ensure consistency.
• I have compiled the docs locally and read through all manual updates to check for errors.

Testing

• I have added unit tests to cover my changes. (For Macros, be sure to check
  @code_lowered and
  @code_typed)
• All new and existing tests passed.
• I have achieved sufficient code coverage.