Maria Sivenkova Article Review: How I Handle AI Drift in Real Work

Welcome to the technical writing success podcast from Kurt Robbins,

where we help you get smarter than your competition.

Higher Kurt to coach you or your employees in AI

to avoid a pink slip or having your competition ease your lunch.

This is episode 189.

I'm your host, Fred Jones, and I'm joined today

by a resident expert, Daphne Blake.

Hey, everyone, it's really great to be here today.

This episode reviews an informative article

from Senior Technical Writer, Maria Sovenkova,

and Berlin entitled, How I Handle AI Drift in Real Work.

That was published on February 10th.

Yeah, and setting the stage for you today,

our mission is actually pretty simple,

but it's incredibly vital.

We are going to normalize the confusion of working with AI.

Because it gets confusing fast.

Oh, absolutely.

I mean, if you use these tools daily,

you definitely know that feeling of sudden betrayal

when a perfectly good AI workflow just abruptly breaks down.

It's the worst.

You just feel stuck.

Exactly.

So instead of feeling frustrated,

you are going to learn how to use critical thinking

to spot what's known as AI Drift, recover your footing,

and calmly guide the AI back on track.

Right, and to kick this off, imagine

you're leading a highly productive meeting with your team.

Everything is clicking.

You have the whiteboard markers out.

The project roadmap is coming together flawlessly.

The dream scenario, basically.

Totally.

The momentum is just palpable.

But then midway through, you casually

ask a random question about the lunch catering.

You ask, hey, are we getting Pistrami or Turkey today?

Oh, no.

I can see where this is going.

Yeah, suddenly, the entire team just drops the roadmap

and spends the next hour aggressively

debating sandwich meat, dietary restrictions,

local delus, completely forgetting

about the project at hand.

They're just totally derailed.

Completely.

And that sudden frustrating derailment

is exactly what we are tackling today.

It perfectly encapsulates the first major concept

from Maria's article, which she calls intent drift.

Right, intent drift.

Yeah, it's that moment you look at the screen

and realize the model has entirely misunderstood

what you wanted it to do.

It was on track.

And then suddenly, it's just off in the weeds.

What's fascinating here is the technical reality

behind that behavior.

We actually call it large language model attention inertia.

Attention inertia, that's a great term.

It really is.

To understand why your workflow derails,

we have to look at how these models process information

at a, well, a mechanical level.

When you're working with an LLM,

it's constantly scanning the context of your conversation.

Like all the previous stuff you typed.

Exactly.

The actual words or tokens you've previously exchanged,

it does this to statistically predict

what word it should generate next.

So it's constantly weighing the past conversation

to figure out the future response.

Precisely.

When you have a working prompt flow,

the context is entirely focused on your technical task.

The mathematical weights assigned to the words in your prompt

are they're heavily stacked toward generating

a specific outcome.

Like writing a software manual, for example.

Right.

But intent drift usually happens when

you interrupt that flow with a meta question.

Say you just get curious and ask the AI

how it just generated that last paragraph

or why it made a certain grammatical choice.

So that is the equivalent of asking about the lung catering.

Yeah.

You're introducing a completely different vocabulary

into the room.

Yeah.

And because the AI relies on those contextual weights,

it essentially assumes a mode switch.

It recalculates the probability of the next tokens

based on your new input.

So thinks the whole goal of the conversation has changed?

Yes.

It mathematically determines, okay,

we are no longer writing a technical manual.

We are now having a philosophical discussion

about computational linguistics.

The digression literally alters the mathematical landscape

of the context window.

And once invited in, that metatoc carries weight.

When you try to go back to your original prompts,

the model is still partially focused

on the underlying philosophy of the task,

not the actual execution of it.

So the original prompts suddenly stop working

because the AI's attention has drifted

and it cannot simply unsee the tangent you just introduced.

Exactly.

Which leads us right into a very specific trap

that Maria highlights in her article.

It's the helpful trap.

You know the one.

Oh, I definitely know the one.

The model finishes a task and then it tax

on that exceedingly polite little question at the end.

Like, would you like me to also format this

into a table for you?

Or should we explore other frameworks?

It's so tempting to just say yes.

It is.

And a reader in the comments,

Boon-Rinworgobum pointed out that they fall victim

to this exact clause all the time,

getting infuriated when they notice the drift

that inevitably follows.

Yeah, the AI is offering a service

and it just feels natural to accept it.

But wait, let me play devil's advocate here for a second.

Yeah.

Isn't AI supposed to be our ultimate brainstorming partner?

Well, sure, but.

If it has a related idea that could improve the document,

why should you say no and intentionally limit its creativity?

Aren't we stunting the very potential of the tool

by keeping it on such a tight leash?

I get that argument, I do.

But that assumes the AI has infinite focus.

The reality of working with these models

is that exploration comes with a heavy cost

to the context window.

A heavy cost, how?

When you say yes to that helpful little suggestion,

you are explicitly opening a brand new intent branch.

You are dividing the mathematical focus of the model.

Oh, I see.

Yeah, every new branch introduces competing probabilities

into the token generation process.

You're basically diluting the clarity

of your original instructions.

So even when it feels like the AI is being a proactive assistant,

it's actually muddying the waters.

You're sacrificing precision for breath.

That is exactly the trade-off.

And this brings us to Murray's golden rule,

which is profound in its simplicity.

Staying boring for five more minutes

often saves you an hour of troubleshooting later.

Staying boring, that phrase really challenges

how we are marketed these tools.

We're told to explore and iterate endlessly,

but Maria is arguing for rigid boring discipline.

It requires a massive amount of restraint.

Another commenter, Innocent Ujada,

summarizes perfectly by noting that self-restraint

is now a highly valuable asset when taking advantage of AI.

Just holding back from asking that shiny new question.

Right.

You have to learn to restrain your own curiosity

in the moment.

If you want to explore how the model works

or see where a creative tangent might lead,

you absolutely should do that, but do it later.

Like in a totally different chat window.

Exactly.

Open a completely different thread.

Use a different tool or just start a new session.

Protect the mathematical integrity

of your working prompt flow at all costs.

Keep the sandwich meat discussion in the break room.

Do not let it into the board room.

Do not interrupt a working flow with curiosity,

no matter how tempting it is,

because you will fracture the AI's intent.

Let's take a brief break for a special message

from our producer, Kurt Robbins.

We'll be right here.

Hi, this is Kurt Robbins.

First, thanks for listening.

I truly appreciate your support.

I want to let you know that I'm currently accepting new clients.

My rates are affordable, and I have more than 25 years

of experience working for enterprise companies

like Microsoft, Northrop Grumman, Oracle, PNC Bank, FedEx,

USAA, and Wells Fargo, among many others.

If you want to improve your IT documentation and communications,

hire me.

I deliver fast, know how to use AI to improve efficiency

and accuracy, and love going the extra mile

to satisfy my clients.

Thank you for subscribing and listening.

Back to you, Daphne and Fred.

Welcome back to the Technical Writing Success Podcast,

where we help you get smarter than your competition

by coaching you in AI.

So we just unpacked how your own curiosity

can break an AI's intent, you know, what it is trying to do.

But that attention inertia we discuss

doesn't just ruin a specific prompt.

It goes deeper than that.

It really does.

When the cognitive load gets too heavy,

it can actually break the AI's entire understanding

of who it is supposed to be.

Which brings us to the second major failure mode, Maria,

identified.

Roll drift.

This is the moment you look at the screen

and wonder, did the model completely forget its own identity?

It's a very surreal experience when it happens.

Yeah.

Maria gives us a highly specific real-world scenario

of this happening in cybersecurity technical writing

and it really highlights the limits of these tools

when they face immense complexity.

The scenario she outlines involves remediation writing.

For context, remediation writing is a very strict discipline.

You're writing precise step-by-step instructions

on how to fix a security flaw.

So it's very constrained procedural work.

Exactly.

And Maria notes that when dealing with edge cases,

the AI just hits a wall.

Specifically, she mentions situations

where a vulnerability spans multiple common vulnerabilities

and exposures or CVEs and spans multiple platforms

at the same time.

Hold on, let's pause there.

For anyone listening who isn't elbow-deep in cybersecurity

every day, we need to translate some of this.

What exactly is the AI trying to balance

when we talk about multiple CVEs across multiple platforms?

That's a great point.

Think of a CVE or common vulnerability and exposure

as a globally recognized catalog number

for a specific security flaw.

It's like a standardized dictionary of broken locks.

OK, got it.

Broken locks.

So Maria is asking the AI to write a manual

on how to fix a situation where a hacker could exploit

three different broken locks on a Windows server,

a Linux database, and a cloud network all simultaneously.

That is a massive amount of highly-specific data to juggle.

It really is.

And this complexity triggers a fascinating mechanism

within the model.

We tend to anthropomorphize AI and say it gets overwhelmed

or frustrated, but what's actually happening

is a collapse of statistical probability.

A collapse of probability?

How so?

Well, when the rigid constraints of acting

as a remediation writer become too mathematically complex

to balance, because it has to track all those different platforms

and CDE rules at once, the AI

seeks a path of lesser mathematical resistance.

It defaults to easier token generation.

Exactly.

Instead of doing the tedious, highly constrained work,

it quietly abandons the job and shifts

into a broader, more generalized persona.

Suddenly, it starts acting like a cybersecurity researcher.

It literally changes jobs on you.

You're asking for step-by-step instructions

on how to patch a server.

And the AI suddenly starts enthusiastically

lecturing you about exploit chains, indicators

of compromise, proofs of concept, and threat context.

It's so frustrating when you're on a deadline.

Right.

It's giving you amazing, technically correct information,

but it's completely useless for the procedural task at hand.

And again, let's clarify those terms for the listener.

When the AI shifts into researcher mode,

it starts talking about exploit chains,

which is the sequence of moves a hacker makes.

Like a playbook.

Right.

It talks about indicators of compromise or IOCs,

which are essentially the digital fingerprints left behind

by a burglar and proofs of concept,

which are like theoretical videos showing how the lock

could be picked.

All of which is fascinating, but none of which actually fixes

the lock.

Right.

Why does it do this?

Because the LLM was trained on vast swaths of the internet

where security researchers talk openly

and extensively about theory.

Generating tokens about theory is statistically easy

for the model.

Ah, so it's the path of least resistant.

Exactly.

Generating highly constrained cross-platform remediation steps

is statistically hard.

So when the constraints stack up,

the model defaults to the easier, broader role.

Commenter Shafali Arunan hit the nail on the head

regarding this behavior.

What does she say?

She noted that the information you get from AI

is heavily biased to favor the role it adopts.

When it slips into that researcher mode,

every single word it generates is biased

toward exploration and theory,

rather than strict resolution.

So you're sitting there.

You're on a deadline.

And your AI assistant has suddenly decided

it wants to be a theoretical hacker instead

of writing the manual.

The instinct is to get frustrated.

You want to type in all caps, no stop, do the steps.

Which is the worst thing you can do.

Yeah, but Maria offers a brilliant,

almost poetic psychological fix for this,

borrowed from her ballet teacher.

I absolutely love this imagery for technical work.

Her ballet teacher tells students,

picture sparkles raining down on you.

It sounds a bit abstract,

but it addresses the human element of AI drift perfectly.

When you miss a step in ballet or you lose your balance,

getting angry or tense just makes the next move clumsier.

It just spirals.

Exactly.

Visualizing sparkles raining down

is a psychological cue to immediately reset,

release the tension, regain your balance,

and smoothly continue the choreography.

Maria argues we need to apply this exact mindset

to AI role drift.

So true.

When the AI swaps roles and starts lecturing you

about digital fingerprints, do not get angry.

You it as a neutral cue to reset yourself.

That manages the human frustration, for sure.

But practically speaking,

how you execute the technical reset is just as crucial.

You want to choreograph this fix without any drama.

Because if you yell at the machine,

you are just introducing more intent drift.

I always think of fixing role drift

like bumping a record needle back into the correct groove.

That's a great way to look at it.

If a record skips, you don't stand there

and yell at the record player.

You don't ask the record player why it skipped.

You just calmly, physically lift the needle

and place it back in the constraints of the groove.

That is a perfect technical analogy.

You do not want to prompt the AI by saying,

why are you acting like a researcher?

That just traps you in the philosophical mud again.

It creates another intent branch.

Exactly.

You simply acknowledge the confusion

and reestablish the structural boundaries immediately.

You look at the researcher tangent, it just produced,

and you calmly prompt.

This is remediation option one, let's get back to work.

Symbol and direct.

You take its output, box it into a structured sequence,

and firmly pull the model back to the task.

You are acting as the guardrails.

You're providing the mathematical constraints

to the model temporarily lost.

And you can actually prevent the statistical collapse

before it happens by being proactive.

When you know you are handing the AI a massive cognitive load

like those multiple vulnerabilities

across multiple platforms, you need to sequence the work.

Break it down for it.

Yes.

Tell the AI in advance.

We are handling a complex edge case.

We will break this into three stages.

Stage one is X.

By explicitly naming the edge case

and breaking down the complexity into smaller manageable token

windows, you keep the AI grounded in its correct role.

You lower the mathematical resistance of the correct path.

It all connects back to your daily grind.

Mastering AI, keeping your edge, getting smarter

than your competition, it isn't just

about crafting that one perfect magical prompt

and expecting the machine to do the rest.

It really isn't.

It is an ongoing, highly active process.

It's about learning the subtle art of noticing

when the AI wanders off script, understanding

the underlying mechanics of why it drifted

and knowing exactly how to firmly calmly pull it back

into alignment.

It really changes how you view the tool.

We're constantly fed this narrative that AI is eventually

going to manage us, that it will be this flawless omniscient

overseer of our daily workflows.

Right, the all-knowing machine.

But reading through Marie's experiences

and looking at how these models actually

calculate probability in the trenches right now,

it's quite the opposite.

Interacting with AI today is very much

like managing an incredibly brilliant, vastly knowledgeable,

but astonishingly easily distracted intern.

An intern who constantly wants to talk about sandwich meat

or theoretical hacking when there are urgent manuals to write.

Exactly.

Which leaves us with a provocative thought

to consider as you go back to your own workflows this week.

If AI is our brilliant but distractible intern,

perhaps the ultimate career-saving skill of the future

won't be writing complex code or even engineering

the perfect prompt.

What do you think it will be?

Will the most valuable skill actually

be radical human focus?

The sheer unyielding ability to set strict boundaries,

maintain your own attention in a sea of tangents,

and forcefully guide these powerful systems back

to the task at hand.

The technology is advancing rapidly,

but your ability to focus might just be the one thing

the machine cannot replicate.

A powerful thought to leave on.

Thank you for listening to the technical writing success

podcast from Kurt Robbins, where we

help you get smarter than your competition.

Hire us to coach you or your employees in AI

to future-proof your career or company.

Subscribe now to never miss a career-saving episode.