Research · Engineering Notes
Recent engineering notes
Technical write-ups from the Modelence lab — architecture experiments, fine-tuning methodology, and production inference engineering.
2026.03
Sub-1B Parameter Models Matching 7B Benchmark Performance
Architecture
2026.02
Adaptive LoRA Rank Selection via Gradient Spectral Analysis
Fine-Tuning
2026.01
KV-Cache Compression with Zero Quality Degradation at 4× Reduction
Inference
2025.12
Fault-Tolerant Distributed Training on Heterogeneous GPU Clusters
Infrastructure