Juan's ordering problem is usually a spec problem underneath it. You can't hand a step to an agent to check its own work against until you've written down what a good output actually looks like, and that's exactly the writing most people skip, because they're still fuzzy on it themselves. The steps that get automated first are just the ones where that spec already existed before AI ever showed up.
From what I have read, I think a clear spec explains what “good” means, but it does not stop self-confirmation. Even with the right rules, an agent can stick to its first idea and read the evidence in a biased way. To prevent this a separate checker is still needed to make sure the result really matches the spec and is supported by the evidence.
Yeah that tracks with what we've seen too, but the separation has to be real because if the checker is shown the first agent's reasoning along with the evidence, it'll just anchor on that reasoning instead of working the answer out itself. Just give it the spec and the raw evidence, not the explanation for why the first agent thinks it's right, or the bias doesn't go away, it just moves one step downstream.
Thanks for sharing this, Joel! It helps. The same here. Love the interviewing part but the rest could be automated to a certain level. I don’t think if anyone is still manually creating the timestamps though. :) Would you have any suggestions to how to come up with the right wording for the thumbnails and titles? It has such a huge impact on the discoverability.
I automate whatever is easiest first, which is never what was actually in my way. The craft line fixes the order. Stealing it.
Juan's ordering problem is usually a spec problem underneath it. You can't hand a step to an agent to check its own work against until you've written down what a good output actually looks like, and that's exactly the writing most people skip, because they're still fuzzy on it themselves. The steps that get automated first are just the ones where that spec already existed before AI ever showed up.
From what I have read, I think a clear spec explains what “good” means, but it does not stop self-confirmation. Even with the right rules, an agent can stick to its first idea and read the evidence in a biased way. To prevent this a separate checker is still needed to make sure the result really matches the spec and is supported by the evidence.
Yeah that tracks with what we've seen too, but the separation has to be real because if the checker is shown the first agent's reasoning along with the evidence, it'll just anchor on that reasoning instead of working the answer out itself. Just give it the spec and the raw evidence, not the explanation for why the first agent thinks it's right, or the bias doesn't go away, it just moves one step downstream.
Love this, two AI experts sharing experience based knowledge.
😂 trying….sometimes hallucinating along the agents.
Hahaha 😂
Good point! 👏
Glad to hear!!
it's clear to me that the transcript is gold
Yes!
Fantastic workflow to mine the video podcast transcript for professional posting, Joel. Great stuff.
Thanks for sharing this, Joel! It helps. The same here. Love the interviewing part but the rest could be automated to a certain level. I don’t think if anyone is still manually creating the timestamps though. :) Would you have any suggestions to how to come up with the right wording for the thumbnails and titles? It has such a huge impact on the discoverability.
The craft line is the whole framework in two words, everything else here is just execution.