Sharing my Maintainable Software Podcast episode!
I joined Robby Russell on the Maintainable Software Podcast recently, for a conversation about what it actually takes to maintain AI products!
The framing question of the podcast is what does maintainability mean when your software can change without a deployment?
Unlike deterministic software, an AI system can degrade because the data changes, the model changes, or its inputs drift, even when the underlying code stays exactly the same. That’s a different kind of maintenance problem, and it needs different tools.
We got into the details of how teams build confidence in these systems:
- evals
- human review
- production monitoring
- thoughtful experimentation
We also talked about the ongoing maintenance cost of AI features, and why starting with the simplest model that solves the problem is often the better engineering decision.
Robby was a fantastic host. Give it a listen, and let me know what you think of the conversation.