Building intent-based access control without reading minds

In Minority Report, the Department of Precrime promised certainty. Three precognitives saw the future, the system picked the consensus, and police arrested people for murders they hadn't yet committed. The whole machine depended on one buried secret: sometimes the precogs disagreed, and the dissenting vision, the minority report, was quietly deleted. The system held together as long as the precogs agreed, but broke when it removed its own doubt.

A new wave of security products is reaching for the same capability. The pitch goes something like this: put a small AI model on every laptop, let it watch what people do across their apps, browsers, and AI tools the way a colleague sitting behind them would, understand what each person is trying to do, determine if they are malicious, and intervene before the damage is done. The same monitoring keeps a complete record of work, so an analyst can read what happened instead of piecing it together from scattered logs. The person might be an employee pasting sensitive data into an unapproved AI assistant, an outsider logged in with stolen credentials, or someone following a scammer's instructions to hand over remote control of their laptop. The watching part is now straightforward with local models. The design question is what a system should do with inference about intent that won’t always be correct.

Strip away the architecture and the pitch makes three promises: catch a risky action in time to stop it, tell it apart from normal work without burying analysts in alerts, and make whatever gets flagged quick to investigate. Useful intent-based access control can be built today by combining contextual inference with explicit policy and enforcement at sensitive actions. The research supports reconstructing what happened after the fact, scoring how well an action fits a person's job, and adding friction at the moment of risk. Recognizing someone's immediate goal while they're still pursuing it is less settled, and reliably separating malicious from benign motives in ordinary work is a challenge. So working systems test something more concrete: how well an action fits the person's job, how unusual it is for them, or which objective on a short list it matches. Each test trades false alarms for misses differently, and each leaves a gap an imitator can hide in, which is why the design around the test matters as much as the test itself. We’ll start with what a local model can read from the screen, move through the tests from most to least concrete, take the case where the intent belongs to someone else, and finish with how the tests combine into access control.

What a local model can read from the screen

Reconstructing what someone did on their screen already works surprisingly well. Cohen et al. split the job in two. A small model first summarizes each step of a short UI trajectory, using screenshots and action records to capture what was on screen and what the user did, and a second fine-tuned model infers the goal from those summaries. With Gemini Flash 8B as the small model, that decomposed pipeline inferred the user's goal about as well as Gemini 1.5 Pro, a model that cost more than 30 times as much per token. Jiang et al.'s task-induction method handles an earlier step, sorting interleaved computer-use traces back into their original tasks with near-perfect agreement.

Both results are retrospective. Each settles its answer after the last interaction, which suits post-incident response but doesn't establish how reliably a goal can be identified while someone is still pursuing it.

Established endpoint vendors and newer products alike are building around small, specialized models that make decisions in real time, in some cases directly on the device. Many rely mainly on structured events such as clipboard copies, file access, browser content, data lineage, and a baseline of each person's normal work. Some capture screenshots around an incident as evidence rather than continuously interpreting the screen, and that’s a reasonable choice: evidence that makes an alert quick to check lowers the cost of every false alarm. Logins, file access, process launches, and payment events arrive already labeled, so there's little left to summarize, and none of the studies reviewed here measures the added security value of reading the screen on top of those event streams.

The harder problem is inferring the goal mid-session from only the evidence available so far. GUIDE, a CVPR 2026 benchmark built from 67.5 hours of novice users working in apps like Photoshop and Excel, took a first step. It cut clips averaging about 25 seconds off at 25%, 50%, and 75% and asked models to pick the user's immediate goal from four options. With a quarter of the clip, the best model in this partial-clip experiment picked correctly about half the time, rising to about two thirds with the full clip, where random guessing gets 25%. Even with the overall task and possible immediate goals supplied, models frequently chose the wrong one. The researchers used users’ spoken explanations to establish the answers, but some intent was not recoverable from the screen alone.

Even a perfect, instant description settles nothing by itself. "Copying the customer list into a personal email draft" fits a sales rep finishing work at home as well as a sales rep leaving for a competitor. To act, a system has to determine if that is one of a myriad of benign goals or an example of rare malicious intent.

Job fit as the primary signal

The most concrete test asks whether an action fits its context. Google's Facade, which has monitored employee access to documents, databases, and internal services since 2018, is trained on unlabeled historical activity rather than known attacks and scores each access by how well it matches what the system knows about the person and the resource, such as their role, team, and recent work. Nobody has to write a menu of goals, and its authors report an event-level false-positive rate below 0.01% in their evaluation.

A small review queue comes with modest recall. In a red-team exercise, Google employees spent two weeks reaching for sensitive data outside their normal work using their own credentials. In a simulated daily review process, Facade retrieved three to four of the 15 attackers across five trained models at a budget of ten audits a day, a cost its authors call remarkably low for a company of more than 100,000 employees.

Its authors are explicit about the blind spot. Facade can't detect malicious actions that are indistinguishable from everyday work, which makes it unsuitable for catching a rogue employee reading files that are relevant to their job.

Deviation needs a review budget

Most insider-threat research takes a less concrete route: learn each person's normal behavior and flag what departs from it. However, real deployments show how much ordinary deviation there is. When Erola et al. ran an insider-threat tool inside three multinational companies, deliberately broad configurations produced about 356 anomalous alerts a day at one organization and 3,713 at another. The team detected a staged insider scenario, but could not establish how much of the ordinary flagged behavior was genuinely malicious. The authors found that real behavior deviates far more than the synthetic data most research relies on.

Synthetic benchmarks yield flattering results. A review of 131 studies built on the CERT insider-threat datasets found that nearly half use a release with 70 malicious users among 1,000, and few report how many alerts an analyst would review to find them. 

Lateral-movement detectors show the same pattern. When Larroche re-ran Euler and Argus with all remote logins retained in the Los Alamos National Laboratory test period, both still scored about 98% AUC, which measures how often a random attack outranks a random normal login. However, with only about two labeled malicious events per million, average precision fell below 0.1% for both, compared with the 5% and 32% reported on the original, more heavily filtered test sets.

Deviation can also be manipulated. Goyal et al. padded attacks with ordinary activity sampled from the target system, and reported 100% evasion against five detectors in their main public-dataset experiments, without changing the malicious payload.

Named objectives for known stories

Candidate-based goal recognition methods rank goals that someone already wrote down, and building that menu for an open-ended desktop session, where the user might be doing anything from editing a quarterly report to exfiltrating a database, is a problem they don't take on. Commercial products sidestep it by shrinking the menu to a few risk labels or policy categories. Screen-reading systems like Cohen's skip the menu by writing the intent in their own words, which creates a different problem. A security tool would still have to decide whether a free-text guess like "export the customer list" is malicious.

A detour can mean a hidden goal, or someone who is lost, rushed, or trying a fix that didn't work. Telling those apart starts with noticing the confusion, and current vision models often don't. In GUIDE's screen recordings of real novice users, the reported confusion matrix correctly classified only 7% of frustration clips and 4% of debugging clips. Most of those moments were labeled as confident work or deliberate exploration, reading repeated clicks, undos, and hesitation as purposeful progress. A security tool that reads struggle as purpose could see more intent in a detour than is really there.

Products go one step further and assign threat categories: malicious, careless, or compromised. The promise is a proportional response, coaching for the careless employee and an investigation for the malicious one, but those labels mix intent, behavior, and who controls the account. Distinguishing malicious motives from benign ones is harder than recognizing the immediate goal, and the evidence that would settle it may not be on the endpoint when the action happens. Facade deliberately avoids these labels. Its threat model treats a rogue employee and an employee whose stolen credentials someone else is using as the same case, because both look unusual for that person. 

Named-objective detectors turn motives and risk factors into stories. A recent review of insider-threat psychology discusses financial pressure, workplace grievances, and ideology, alongside a separate category of accidental or negligent harm. In DARPA's insider-threat program, Senator et al. built a detector for each of several such stories, including the saboteur, the fraudster, the careless user, and the employee walking off with intellectual property, and ran them over two months of monitored computer activity from about 5,500 employees, with red-team scenarios inserted into real users' records. The best of them, built for the leader of a small group stealing intellectual property, placed 6 red-team user-days in its October top 50, but in September it ignored 33 of 98 red-team user-days because they didn't fit the story. A named objective is a bet that the attacker will follow a story written in advance. Behavior outside that story can be missed, and an adaptive attacker has a reason to exploit the gap. This study did not test that deliberate evasion.

Friction when the intent belongs to someone else

Some intent doesn't belong to the person at the keyboard at all. A customer coached by a scammer over the phone can log in from their own device, pass authentication, and authorize the payment themselves, leaving familiar signs of account takeover absent. Dhama's theoretical study formalizes a limiting case: if scam and legitimate payments have identical observable feature distributions, a transaction-only detector cannot distinguish between them. The paper argues for adding session context and acknowledges clues in amounts, recipients, and timing.

Scam detection may be where reading the screen adds the most. A scam in progress can leave traces a payment record never sees, such as a remote-access tool running during a banking session or a chat window dictating the payee. ORACLE reads app usage and summarized screen content to warn before a scam completes, but it was tested on synthesized trajectories, so whether it catches live scams at a false-alarm rate a bank could live with is still open.

Friction at the moment of payment can help even without detection. In a field test on Alipay, the better-performing redesigned warning reduced the fraud-case rate among payments already flagged as risky from 0.24% to 0.13%. Both redesigned warnings retained reported pass rates above 90%.

However, scammers do adapt to friction. The UK's payment regulator reported fraudsters coaching victims to ignore warnings that a payee's name didn't match the account, sometimes by explaining that a new "safe account" had been set up that couldn't match yet.

Keep the minority report

For prevention, knowing the goal only helps if you learn it while there's still time to act. Products that promise to stop an action before it completes mostly act at the final step, such as the upload, with a warning, a request for justification, masking, or a block.

When the test at that step can't settle the question, the system can still choose to ask. Ren et al.'s KnowNo shows one way to decide when to ask. A language model reads a text description of the scene and proposes possible next actions for a robot, and KnowNo keeps every option that scores at or above a cutoff calibrated on held-out example tasks. If exactly one option survives, the robot acts. Otherwise it seeks human help, halting if the correct action is absent from the set. 

That's the minority report. When more than one option survives, the system is telling you it has a dissenting prediction. The movie deleted that signal to maintain the illusion of certainty. The research keeps it and uses it to decide when a human should weigh in. Jev's Paradox argued that confidence becomes operationally useful once it drives a decision policy.

Asking is already shipping in commercial products, several of which show a justification prompt or coaching message when intent is unclear. A question costs far less than a premature block, so the system can act on uncertain cases without paying the full price of a false alarm.

How the tests combine into access control

Each test in this piece fails differently, which is the argument for stacking them. Explicit policy sits at the sensitive action, such as the upload, the share, the payment, or the remote-access session, and decides what is permitted at all. Job-fit scoring of the kind Facade uses ranks what's left, because its reported false-positive rate is below 0.01% and it needs no menu of goals. Deviation from a person's baseline becomes an input to that score, so an unusual but plausible day raises a rank without producing an alert of its own. Named stories cover the few patterns worth writing down in advance, such as a departing employee staging files or a remote-access tool running while someone approves a payment, where a narrow detector gets high lift on the pattern it was written for and leaves the rest to the other layers.

When the combined evidence still leaves more than one plausible reading, the system keeps the minority report. A calibrated set of interpretations decides between acting, asking, and escalating. One surviving interpretation lets policy proceed or block. Several surviving interpretations trigger a justification prompt, step-up verification, or a short hold, and the session evidence travels with that decision to an analyst. Answers to those prompts go to the analyst as evidence, since the person most motivated to explain an action is the one the system can least take at their word.

The scoreboard has to reflect base rates. The numbers that matter are attackers caught at a fixed number of reviews per day, average precision against the full benign background, and how often legitimate work gets interrupted, because those are the costs a team actually feels. Red-team exercises should include imitation, padding malicious actions with ordinary activity the way Goyal et al. did, so the evaluation measures how the system handles an attacker who has read its filters.

Deployment can begin with a narrow workflow, such as sending customer data to an external AI service. Contextual scoring runs in observation mode first, measuring attacks caught within a fixed review budget and how often legitimate work would be interrupted. Clear policy violations can support immediate enforcement. Responses that depend on inferred intent expand as evaluation shows which interventions prevent harm at an acceptable cost to ordinary work.

The imitation failures this piece describes are documented blind spots, but how often they occur in real deployments and how well these tests hold up against adaptive attackers remain largely unmeasured. For security, deception is the norm, and it hides among base rates where approximately everything is benign.

Precrime fell to Lamar Burgess, the man who built it. He knew the system threw out repeated visions as echoes, so he staged a murder to look like one. He is the imitator every section of this piece warned about: an adversary who learns what the test treats as normal, whether that's a job-relevant file, an ordinary session, or a story the detector wasn't built for, and hides inside it. The best model keeps the minority report. When the test can't settle the question, it asks the user, waits, introduces friction, or flags the ambiguity for an analyst, on the assumption that the people who know its filters best are the ones most likely to use them.

For a human user, a security tool usually has to infer the task as well as judge the action. Whereas with an AI agent, the goal arrives in writing, but so does the attack, and an injected instruction can make sabotage look like progress. That makes agents the place where intent-based access control can be most precise, because every action can be checked against the stated task. The next piece looks at what it takes to hold an agent to that task.

Research and blogging assistance provided by Hermes, an AI Assistant. Its deviations from the outline were confusion, not concealment, but it understands why the distinction is hard to make from the outside.