A couple of weeks back, I spent two days in Manchester attending an event organised by Amazon Web Services (AWS) which was all about – yes, you guessed it – AI.
Nothing to do with L&D. Actually, quite techie-focused with the opportunity to gets hands-on practice using some of the new tools they have launched to help businesses build out AI solutions.
It isn’t quite what I need right now but it was interesting to use and to see the interface for a suite of tools that are focused on building out a software that has AI at its core.
It was reassuring to discover that many of the generic AI issues that I have been focused on and grappling with in these early stages of PerformaGo are exactly the kinds of things that AWS’s tools aim to address. I realised I am on the right track and not alone in seeing where the pain points lie, when it comes to developing an AI-focused solution.
One of those pain points that has been very much on my mind recently is something the AI world calls determinism. (Another new word to add to my AI tech vocabulary list). I’ve become focused on this through painful experience. Discovering the frustrations of inconsistent GPTs output.
For the uninitiated (welcome to the wonderful world of AI-speak, folks) deterministic GPT output is output that is stable and consistent. Non-deterministic is when the output varies. In other words, you ask the same question multiple times but you won’t get an identical answer each time you ask.
I’ve written about this before in a previous post. From a technical point of view, there are controls and parameters that you can put in place to help manage this; but even with these, you can’t absolutely guarantee the level of determinism that you might really want.
The question of deterministic versus non-deterministic output was central to the AWS event. With good reason. The event was primarily about ‘Agentic AI’. This is AI agents used to perform operational tasks in a business, rather than having human employees complete said task. Clearly, accuracy and completeness are critical to the success (and adoption) of this kind of technology.
The need for highly deterministic output is going to be higher with agentic AI than AI used for performance support; but the need for highly deterministic output absolutely exists in some performance support contexts.
So, is there a solution? Before I answer that I need to make an important point and add a couple of caveats.
What that was very clear from the event is that what AI can do is moving very rapidly. The point was made more than once during the event that things we can achieve today were just not possible 12 months ago.
Which brings me to the caveats. First, I only know what I know. I’m not the world’s expert. I’m learning as I go. Second, if you are reading this 12 months from now things could be quite different. What I’m writing today could be hopelessly out of date.
Back to my, ‘is there a solution?’ question. The short answer is; ‘yes’. But the solution isn’t necessarily elegant or super-straightforward.
The easy option is to place any output you want to be displayed verbatim within your prompt. Include clear instructions that this output must be displayed under a given set of circumstances or at a particular moment. In my experience, so far, this has worked 100% of the time. An easy and reliable solution if the output is short and infrequent.
A second solution is to create very tightly designed and written prompts which incorporate a master control document. Think of this as the definitive SOP manual for your GPT. My friend Scott Schang has become a master of this genre – more on this is another post.
But if you have more complex requirements, you are likely going to need a database (this is in addition to a knowledge base) that can be referenced and drawn from by the GPT. I understand that some people might recoil at the thought. Perhaps it sounds way too techie and complex.
One of the challenges ahead in designing and building PerformaGo will be making that kind of functionality easy and accessible for L&D people who are not big on tech.
One final point for this week. During the event, I had some interesting thoughts on the overlap between agentic AI and AI used for performance support. Keep an eye out for a post all about that sometime soon.
Until next time…
