ELE 888 — Intelligent Systems (lab projects)

This repository contains practical assignments for ELE 888 (TMU Intelligent Systems): pattern classification, neural networks, and unsupervised learning in Python.

Author: Arian Fooladray · Term: Winter 2026 (where noted)

Prerequisites

• Python 3.10+ recommended
• Common packages: numpy, matplotlib
• Lab 2, 1 (sklearn): scikit-learn
• Lab 4 (images): imageio and Pillow — pip install numpy matplotlib scikit-learn imageio pillow

All commands below are written for a shell with the repository root as the working directory unless cd is shown.

---

Lab overview and results (summary)

Lab	Topic	What you get	Representative results*
1	Bayesian decision theory (Iris, 1D Gaussian)	Posteriors, Bayes decisions, cost-sensitive thresholds, PDF plots	Full analysis for sepal length & width; mu1 / mu2 thresholds; classification tables and risk for listed test x; saved PNGs per feature
2	Linear discriminant functions (Iris)	MSE/Adaline (batch GD), 2D decision plot, 3-class OVR + confusion	Binary (4D): 100% test on 0v1, 0v2, 1v2; 3-class OVR: ~80% test (typical; Versicolour vs Virginica is hardest)
3	2-2-1 MLP, XOR, batch backprop, tanh	Convergence, learning curve, input/hidden decision surfaces	XOR learned; report example: converged in 2251 epochs (exact epoch count depends on random init)
4	K-means (Lloyd) on RGB; image quantization	J vs iteration, means over stages, RGB scatter, labeled images, Xie–Beni (c=5)	Part 1: c=2 with fixed inits; Part 2: c=5, two random runs; figures under lab4/figures/

*Exact numbers for Labs 1–2 and 4 re-run are deterministic where seeds are fixed; Lab 3 epoch count can vary with weight initialization.

---

Lab 1 — Bayesian decision theory (Iris, binary 1D)

Script: lab1/lab1_bayes_classifier.py
Idea: Classify Setosa vs Versicolour using a single feature at a time and Gaussian class-conditional likelihoods. Computes priors, posteriors, discriminant, optimal thresholds (equal and asymmetric costs), and risk under a high-penalty cost matrix.

Run

cd lab1
python lab1_bayes_classifier.py

Closes or saves the matplotlib windows to finish; plots are written to the current directory, e.g. lab1_sepal_length_(cm)_analysis.png, lab1_sepal_width_(cm)_analysis.png.

Outputs

• Console: priors, means/stds, thresholds mu1 (equal cost) and mu2 (high penalty for misclassifying Versicolour as Setosa), classification table for the lab’s test feature values, risk table, and a short feature comparison.
• Figures: class-conditional densities, thresholds, and data ticks at the base.

---

Lab 2 — Linear discriminant functions (Iris)

Script: lab2/lab2_linear_discriminant.py
Idea: Linear discriminants $g(\mathbf{x})=\mathbf{w}^T[1,\mathbf{x}]^T$ trained with batch gradient descent on MSE (Adaline), $\pm1$ targets; standardized features; 3-class via one-vs-rest and $\arg\max$ scores.

Run

cd lab2
python lab2_linear_discriminant.py

Headless (no GUI; writes PNGs only):

python lab2_linear_discriminant.py --save-only

Outputs

• Console: train/test accuracies for three binary problems (4 features), 2D petal-only experiment, and 3-class OVR; confusion matrix for the test set.
• Figures: lab2/figures/ — learning curves, 2D decision region, OVR learning curves, confusion heatmap.

Report (optional)

cd lab2
pdflatex lab2_report.tex

---

Lab 3 — Multilayer network: XOR (2-2-1, backprop, tanh)

Script: lab3/lab3_xor.py
Idea: Four XOR patterns in ${-1,1}^2$, one hidden layer, batch backprop, stop when $J<0.001$ (with eta=0.1).

Run

cd lab3
python lab3_xor.py

Headless + save figures to lab3/figures/:

python lab3_xor.py --save-only

Outputs

• Console: final weights, epoch count, LaTeX table snippet for the report.
• Figures (with --save-only or after closing windows): learning curve, decision surfaces in input space and in hidden-unit space (see lab3_report.tex).

Report (optional)

cd lab3
pdflatex lab3_report.tex

---

Lab 4 — K-means: RGB clustering & image reconstruction

Script: lab4/lab4_kmeans.py
Idea: Lloyd k-means on pixels in $\mathbb{R}^3$; Part 1: c=2 with specified initial means; Part 2: c=5, two random seeds; Xie–Beni for comparing runs.

Input image

Place house.tiff in lab4/, or create a stand-in:

cd lab4
python create_house_image.py

Run

cd lab4
python lab4_kmeans.py

Headless:

python lab4_kmeans.py --save-only

Outputs

• Console: progress, final J, mean evolution, Xie–Beni for the two c=5 runs.
• Figures: lab4/figures/ — J vs iteration, means at stages, RGB scatters, reconstructed (quantized) images.

Report (optional)

cd lab4
pdflatex lab4_report.tex

---

Repository layout

ele888/
  README.md                 ← this file
  lab1/lab1_bayes_classifier.py
  lab2/lab2_linear_discriminant.py, lab2_report.tex, figures/
  lab3/lab3_xor.py, lab3_report.tex, figures/ (after run)
  lab4/lab4_kmeans.py, create_house_image.py, lab4_report.tex, figures/

Each labN/ may also keep a local README_LAB3.md / DEMO_*.md with extra demo notes; the table above is the short version.