ml-op-suite

A small machine learning library built from scratch in Python. Linear and kernel models, custom gradient-based optimizers, and PCA, no scikit-learn under the hood for the actual algorithms.

What's here

• Linear and logistic regression, plus kernelized classifiers (linear, polynomial, Gaussian RBF)
• Loss functions with L0, L1, and L2 regularization, and a softmax loss for multiclass
• Optimizers: gradient descent, gradient descent with line search, proximal gradient for L1, and stochastic gradient descent with swappable learning rate schedules (constant, inverse, inverse squared, inverse sqrt)
• PCA encoder for dimensionality reduction
• Gradient correctness checks via finite differences
• A small CLI in code/main.py that runs experiments and writes plots to figs/

Install

Needs Python 3.9+.

git clone https://github.com/Ry4nW/ml-op-suite.git
cd ml-op-suite
pip install -r requirements.txt

Usage

Experiments are run by number from inside code/:

cd code
python main.py 1        # logistic regression, plain vs linear kernel
python main.py 1.1      # polynomial and RBF kernel comparison
python main.py 1.2      # grid search over kernel width and regularization
python main.py 3.2      # PCA on the animals dataset
python main.py 4        # least squares with line search gradient descent
python main.py 4.1      # SGD across different batch sizes
python main.py 4.3      # SGD across different learning rate schedules
python main.py all      # run everything

Layout

code/
  linear_models.py          linear, logistic, and kernel classifiers
  fun_obj.py                loss functions and their gradients
  optimizers.py              gradient descent, line search, proximal L1, SGD
  kernels.py                 linear, polynomial, and RBF kernels
  encoders.py                PCA
  learning_rate_getters.py  learning rate schedules for SGD
  utils.py                  data loading, plotting, gradient checks
  main.py                   CLI entry point
data/                       pickled datasets used by the experiments
figs/                       output plots