Working with LLM provider APIs is like trying to build a sandcastle during high tide. Just when you think you’ve got everything perfect, a wave of updates comes crashing in and reshapes your entire landscape overnight! In this tech talk, we’re shedding some light on how we tackled this issue via online integration tests for our in-house GenAI application, ensuring that we catch & fix errors in production before our users notice!