Training Vision Models for LLM-Driven Weight-Space Programming