0:00
Welcome to the technical writing success podcast from Kurt Robbins,
0:03
where we help you get smarter than your competition.
0:06
Higher Kurt to coach you or your employees in AI
0:09
to avoid a pink slip or having your competition ease your lunch.
0:13
This is episode 189.
0:15
I'm your host, Fred Jones, and I'm joined today
0:17
by a resident expert, Daphne Blake.
0:19
Hey, everyone, it's really great to be here today.
0:22
This episode reviews an informative article
0:24
from Senior Technical Writer, Maria Sovenkova,
0:26
and Berlin entitled, How I Handle AI Drift in Real Work.
0:31
That was published on February 10th.
0:32
Yeah, and setting the stage for you today,
0:34
our mission is actually pretty simple,
0:36
but it's incredibly vital.
0:38
We are going to normalize the confusion of working with AI.
0:41
Because it gets confusing fast.
0:44
I mean, if you use these tools daily,
0:46
you definitely know that feeling of sudden betrayal
0:49
when a perfectly good AI workflow just abruptly breaks down.
0:53
You just feel stuck.
0:55
So instead of feeling frustrated,
0:57
you are going to learn how to use critical thinking
0:59
to spot what's known as AI Drift, recover your footing,
1:03
and calmly guide the AI back on track.
1:06
Right, and to kick this off, imagine
1:08
you're leading a highly productive meeting with your team.
1:11
Everything is clicking.
1:13
You have the whiteboard markers out.
1:15
The project roadmap is coming together flawlessly.
1:17
The dream scenario, basically.
1:19
The momentum is just palpable.
1:21
But then midway through, you casually
1:23
ask a random question about the lunch catering.
1:26
You ask, hey, are we getting Pistrami or Turkey today?
1:30
I can see where this is going.
1:32
Yeah, suddenly, the entire team just drops the roadmap
1:35
and spends the next hour aggressively
1:38
debating sandwich meat, dietary restrictions,
1:40
local delus, completely forgetting
1:42
about the project at hand.
1:43
They're just totally derailed.
1:46
And that sudden frustrating derailment
1:48
is exactly what we are tackling today.
1:49
It perfectly encapsulates the first major concept
1:53
from Maria's article, which she calls intent drift.
1:55
Right, intent drift.
1:57
Yeah, it's that moment you look at the screen
1:58
and realize the model has entirely misunderstood
2:01
what you wanted it to do.
2:03
And then suddenly, it's just off in the weeds.
2:06
What's fascinating here is the technical reality
2:08
behind that behavior.
2:09
We actually call it large language model attention inertia.
2:13
Attention inertia, that's a great term.
2:17
To understand why your workflow derails,
2:21
we have to look at how these models process information
2:24
at a, well, a mechanical level.
2:27
When you're working with an LLM,
2:28
it's constantly scanning the context of your conversation.
2:31
Like all the previous stuff you typed.
2:34
The actual words or tokens you've previously exchanged,
2:37
it does this to statistically predict
2:39
what word it should generate next.
2:41
So it's constantly weighing the past conversation
2:44
to figure out the future response.
2:46
When you have a working prompt flow,
2:48
the context is entirely focused on your technical task.
2:51
The mathematical weights assigned to the words in your prompt
2:54
are they're heavily stacked toward generating
2:59
Like writing a software manual, for example.
3:02
But intent drift usually happens when
3:04
you interrupt that flow with a meta question.
3:07
Say you just get curious and ask the AI
3:09
how it just generated that last paragraph
3:11
or why it made a certain grammatical choice.
3:15
So that is the equivalent of asking about the lung catering.
3:19
You're introducing a completely different vocabulary
3:21
And because the AI relies on those contextual weights,
3:25
it essentially assumes a mode switch.
3:27
It recalculates the probability of the next tokens
3:31
based on your new input.
3:32
So thinks the whole goal of the conversation has changed?
3:35
It mathematically determines, okay,
3:37
we are no longer writing a technical manual.
3:39
We are now having a philosophical discussion
3:41
about computational linguistics.
3:43
The digression literally alters the mathematical landscape
3:46
of the context window.
3:48
And once invited in, that metatoc carries weight.
3:51
When you try to go back to your original prompts,
3:53
the model is still partially focused
3:55
on the underlying philosophy of the task,
3:57
not the actual execution of it.
3:59
So the original prompts suddenly stop working
4:02
because the AI's attention has drifted
4:04
and it cannot simply unsee the tangent you just introduced.
4:08
Which leads us right into a very specific trap
4:12
that Maria highlights in her article.
4:13
It's the helpful trap.
4:15
Oh, I definitely know the one.
4:16
The model finishes a task and then it tax
4:19
on that exceedingly polite little question at the end.
4:22
Like, would you like me to also format this
4:25
into a table for you?
4:26
Or should we explore other frameworks?
4:29
It's so tempting to just say yes.
4:33
And a reader in the comments,
4:35
Boon-Rinworgobum pointed out that they fall victim
4:38
to this exact clause all the time,
4:40
getting infuriated when they notice the drift
4:42
that inevitably follows.
4:43
Yeah, the AI is offering a service
4:46
and it just feels natural to accept it.
4:48
But wait, let me play devil's advocate here for a second.
4:51
Isn't AI supposed to be our ultimate brainstorming partner?
4:55
If it has a related idea that could improve the document,
4:59
why should you say no and intentionally limit its creativity?
5:02
Aren't we stunting the very potential of the tool
5:05
by keeping it on such a tight leash?
5:06
I get that argument, I do.
5:08
But that assumes the AI has infinite focus.
5:11
The reality of working with these models
5:13
is that exploration comes with a heavy cost
5:15
to the context window.
5:18
When you say yes to that helpful little suggestion,
5:20
you are explicitly opening a brand new intent branch.
5:24
You are dividing the mathematical focus of the model.
5:27
Yeah, every new branch introduces competing probabilities
5:31
into the token generation process.
5:34
You're basically diluting the clarity
5:36
of your original instructions.
5:37
So even when it feels like the AI is being a proactive assistant,
5:41
it's actually muddying the waters.
5:43
You're sacrificing precision for breath.
5:45
That is exactly the trade-off.
5:46
And this brings us to Murray's golden rule,
5:48
which is profound in its simplicity.
5:51
Staying boring for five more minutes
5:53
often saves you an hour of troubleshooting later.
5:55
Staying boring, that phrase really challenges
5:58
how we are marketed these tools.
6:00
We're told to explore and iterate endlessly,
6:02
but Maria is arguing for rigid boring discipline.
6:06
It requires a massive amount of restraint.
6:09
Another commenter, Innocent Ujada,
6:11
summarizes perfectly by noting that self-restraint
6:14
is now a highly valuable asset when taking advantage of AI.
6:18
Just holding back from asking that shiny new question.
6:21
You have to learn to restrain your own curiosity
6:24
If you want to explore how the model works
6:27
or see where a creative tangent might lead,
6:29
you absolutely should do that, but do it later.
6:32
Like in a totally different chat window.
6:35
Open a completely different thread.
6:36
Use a different tool or just start a new session.
6:39
Protect the mathematical integrity
6:41
of your working prompt flow at all costs.
6:43
Keep the sandwich meat discussion in the break room.
6:46
Do not let it into the board room.
6:48
Do not interrupt a working flow with curiosity,
6:50
no matter how tempting it is,
6:52
because you will fracture the AI's intent.
6:55
Let's take a brief break for a special message
6:57
from our producer, Kurt Robbins.
6:59
We'll be right here.
7:00
Hi, this is Kurt Robbins.
7:02
First, thanks for listening.
7:03
I truly appreciate your support.
7:05
I want to let you know that I'm currently accepting new clients.
7:08
My rates are affordable, and I have more than 25 years
7:11
of experience working for enterprise companies
7:14
like Microsoft, Northrop Grumman, Oracle, PNC Bank, FedEx,
7:18
USAA, and Wells Fargo, among many others.
7:22
If you want to improve your IT documentation and communications,
7:26
I deliver fast, know how to use AI to improve efficiency
7:29
and accuracy, and love going the extra mile
7:31
to satisfy my clients.
7:33
Thank you for subscribing and listening.
7:35
Back to you, Daphne and Fred.
7:38
Welcome back to the Technical Writing Success Podcast,
7:40
where we help you get smarter than your competition
7:43
by coaching you in AI.
7:44
So we just unpacked how your own curiosity
7:47
can break an AI's intent, you know, what it is trying to do.
7:51
But that attention inertia we discuss
7:53
doesn't just ruin a specific prompt.
7:55
It goes deeper than that.
7:57
When the cognitive load gets too heavy,
7:59
it can actually break the AI's entire understanding
8:02
of who it is supposed to be.
8:04
Which brings us to the second major failure mode, Maria,
8:09
This is the moment you look at the screen
8:11
and wonder, did the model completely forget its own identity?
8:14
It's a very surreal experience when it happens.
8:18
Maria gives us a highly specific real-world scenario
8:22
of this happening in cybersecurity technical writing
8:24
and it really highlights the limits of these tools
8:27
when they face immense complexity.
8:28
The scenario she outlines involves remediation writing.
8:31
For context, remediation writing is a very strict discipline.
8:34
You're writing precise step-by-step instructions
8:36
on how to fix a security flaw.
8:38
So it's very constrained procedural work.
8:41
And Maria notes that when dealing with edge cases,
8:44
the AI just hits a wall.
8:45
Specifically, she mentions situations
8:47
where a vulnerability spans multiple common vulnerabilities
8:51
and exposures or CVEs and spans multiple platforms
8:56
Hold on, let's pause there.
8:57
For anyone listening who isn't elbow-deep in cybersecurity
9:00
every day, we need to translate some of this.
9:03
What exactly is the AI trying to balance
9:06
when we talk about multiple CVEs across multiple platforms?
9:10
That's a great point.
9:11
Think of a CVE or common vulnerability and exposure
9:14
as a globally recognized catalog number
9:16
for a specific security flaw.
9:18
It's like a standardized dictionary of broken locks.
9:22
So Maria is asking the AI to write a manual
9:24
on how to fix a situation where a hacker could exploit
9:27
three different broken locks on a Windows server,
9:30
a Linux database, and a cloud network all simultaneously.
9:33
That is a massive amount of highly-specific data to juggle.
9:37
And this complexity triggers a fascinating mechanism
9:41
We tend to anthropomorphize AI and say it gets overwhelmed
9:44
or frustrated, but what's actually happening
9:46
is a collapse of statistical probability.
9:48
A collapse of probability?
9:51
Well, when the rigid constraints of acting
9:53
as a remediation writer become too mathematically complex
9:57
to balance, because it has to track all those different platforms
10:01
and CDE rules at once, the AI
10:03
seeks a path of lesser mathematical resistance.
10:06
It defaults to easier token generation.
10:09
Instead of doing the tedious, highly constrained work,
10:13
it quietly abandons the job and shifts
10:15
into a broader, more generalized persona.
10:18
Suddenly, it starts acting like a cybersecurity researcher.
10:21
It literally changes jobs on you.
10:23
You're asking for step-by-step instructions
10:25
on how to patch a server.
10:26
And the AI suddenly starts enthusiastically
10:29
lecturing you about exploit chains, indicators
10:31
of compromise, proofs of concept, and threat context.
10:34
It's so frustrating when you're on a deadline.
10:37
It's giving you amazing, technically correct information,
10:40
but it's completely useless for the procedural task at hand.
10:42
And again, let's clarify those terms for the listener.
10:45
When the AI shifts into researcher mode,
10:47
it starts talking about exploit chains,
10:49
which is the sequence of moves a hacker makes.
10:53
It talks about indicators of compromise or IOCs,
10:55
which are essentially the digital fingerprints left behind
10:58
by a burglar and proofs of concept,
11:01
which are like theoretical videos showing how the lock
11:04
All of which is fascinating, but none of which actually fixes
11:08
Why does it do this?
11:10
Because the LLM was trained on vast swaths of the internet
11:13
where security researchers talk openly
11:16
and extensively about theory.
11:18
Generating tokens about theory is statistically easy
11:21
Ah, so it's the path of least resistant.
11:24
Generating highly constrained cross-platform remediation steps
11:28
is statistically hard.
11:30
So when the constraints stack up,
11:32
the model defaults to the easier, broader role.
11:35
Commenter Shafali Arunan hit the nail on the head
11:37
regarding this behavior.
11:40
She noted that the information you get from AI
11:42
is heavily biased to favor the role it adopts.
11:45
When it slips into that researcher mode,
11:47
every single word it generates is biased
11:49
toward exploration and theory,
11:51
rather than strict resolution.
11:53
So you're sitting there.
11:53
You're on a deadline.
11:55
And your AI assistant has suddenly decided
11:57
it wants to be a theoretical hacker instead
11:59
of writing the manual.
12:00
The instinct is to get frustrated.
12:02
You want to type in all caps, no stop, do the steps.
12:05
Which is the worst thing you can do.
12:07
Yeah, but Maria offers a brilliant,
12:10
almost poetic psychological fix for this,
12:12
borrowed from her ballet teacher.
12:14
I absolutely love this imagery for technical work.
12:16
Her ballet teacher tells students,
12:18
picture sparkles raining down on you.
12:21
It sounds a bit abstract,
12:23
but it addresses the human element of AI drift perfectly.
12:27
When you miss a step in ballet or you lose your balance,
12:29
getting angry or tense just makes the next move clumsier.
12:35
Visualizing sparkles raining down
12:36
is a psychological cue to immediately reset,
12:39
release the tension, regain your balance,
12:41
and smoothly continue the choreography.
12:44
Maria argues we need to apply this exact mindset
12:49
When the AI swaps roles and starts lecturing you
12:51
about digital fingerprints, do not get angry.
12:54
You it as a neutral cue to reset yourself.
12:56
That manages the human frustration, for sure.
12:59
But practically speaking,
13:00
how you execute the technical reset is just as crucial.
13:04
You want to choreograph this fix without any drama.
13:08
Because if you yell at the machine,
13:10
you are just introducing more intent drift.
13:13
I always think of fixing role drift
13:14
like bumping a record needle back into the correct groove.
13:18
That's a great way to look at it.
13:19
If a record skips, you don't stand there
13:21
and yell at the record player.
13:22
You don't ask the record player why it skipped.
13:25
You just calmly, physically lift the needle
13:28
and place it back in the constraints of the groove.
13:30
That is a perfect technical analogy.
13:32
You do not want to prompt the AI by saying,
13:34
why are you acting like a researcher?
13:36
That just traps you in the philosophical mud again.
13:38
It creates another intent branch.
13:40
You simply acknowledge the confusion
13:42
and reestablish the structural boundaries immediately.
13:46
You look at the researcher tangent, it just produced,
13:48
and you calmly prompt.
13:50
This is remediation option one, let's get back to work.
13:54
You take its output, box it into a structured sequence,
13:58
and firmly pull the model back to the task.
14:00
You are acting as the guardrails.
14:03
You're providing the mathematical constraints
14:05
to the model temporarily lost.
14:07
And you can actually prevent the statistical collapse
14:09
before it happens by being proactive.
14:11
When you know you are handing the AI a massive cognitive load
14:16
like those multiple vulnerabilities
14:17
across multiple platforms, you need to sequence the work.
14:21
Break it down for it.
14:23
Tell the AI in advance.
14:25
We are handling a complex edge case.
14:26
We will break this into three stages.
14:29
By explicitly naming the edge case
14:31
and breaking down the complexity into smaller manageable token
14:34
windows, you keep the AI grounded in its correct role.
14:39
You lower the mathematical resistance of the correct path.
14:42
It all connects back to your daily grind.
14:44
Mastering AI, keeping your edge, getting smarter
14:47
than your competition, it isn't just
14:49
about crafting that one perfect magical prompt
14:51
and expecting the machine to do the rest.
14:54
It is an ongoing, highly active process.
14:57
It's about learning the subtle art of noticing
14:59
when the AI wanders off script, understanding
15:01
the underlying mechanics of why it drifted
15:04
and knowing exactly how to firmly calmly pull it back
15:08
It really changes how you view the tool.
15:10
We're constantly fed this narrative that AI is eventually
15:14
going to manage us, that it will be this flawless omniscient
15:17
overseer of our daily workflows.
15:20
Right, the all-knowing machine.
15:21
But reading through Marie's experiences
15:23
and looking at how these models actually
15:24
calculate probability in the trenches right now,
15:27
it's quite the opposite.
15:29
Interacting with AI today is very much
15:31
like managing an incredibly brilliant, vastly knowledgeable,
15:35
but astonishingly easily distracted intern.
15:38
An intern who constantly wants to talk about sandwich meat
15:40
or theoretical hacking when there are urgent manuals to write.
15:45
Which leaves us with a provocative thought
15:47
to consider as you go back to your own workflows this week.
15:50
If AI is our brilliant but distractible intern,
15:54
perhaps the ultimate career-saving skill of the future
15:57
won't be writing complex code or even engineering
16:00
the perfect prompt.
16:01
What do you think it will be?
16:02
Will the most valuable skill actually
16:04
be radical human focus?
16:07
The sheer unyielding ability to set strict boundaries,
16:11
maintain your own attention in a sea of tangents,
16:14
and forcefully guide these powerful systems back
16:16
to the task at hand.
16:18
The technology is advancing rapidly,
16:20
but your ability to focus might just be the one thing
16:22
the machine cannot replicate.
16:24
A powerful thought to leave on.
16:26
Thank you for listening to the technical writing success
16:28
podcast from Kurt Robbins, where we
16:30
help you get smarter than your competition.
16:32
Hire us to coach you or your employees in AI
16:35
to future-proof your career or company.
16:37
Subscribe now to never miss a career-saving episode.