Accelerating UX research with synthetic users

How I helped turn AI-powered personas into a scalable research capability at SentinelOne

Company: SentinelOne
Role: Senior Staff Content Designer, synthetic user initiative lead for content testing and enablement
Timeline: December 2025–May 2026
Focus: AI research, product strategy, UX content, experimentation, operational enablement

At a glance

  • 2-week development sprint to build and evaluate synthetic users

  • 3 LLM platforms evaluated: ChatGPT, Gemini, and Claude

  • 20-question comparative evaluation using general, workflow, and open-ended questions

  • Research-backed AI personas created for cybersecurity users

  • Reusable playbooks, prompts, and templates developed for product teams

  • Real UX issues identified through testing of product prototypes

The challenge: Moving fast without sacrificing user insight

At SentinelOne, we designed enterprise cybersecurity software for highly specialized users, including security operations center (SOC) analysts, security engineers, and administrators.

Understanding these users was essential to designing clear, effective experiences. But traditional user research required time for recruitment, scheduling, testing, and synthesis.

Not every product decision could wait for a formal study.

Teams needed a way to get earlier feedback on content, information architecture, and workflows without replacing the rigor of human research.

I saw an opportunity to explore whether AI-powered synthetic users could help close that gap.

The question wasn't whether AI could convincingly pretend to be a person. It was whether we could ground an AI persona in real research strongly enough to generate useful, testable feedback.

My role

I helped lead the synthetic user initiative, with ownership of content testing strategy, prompt evaluation, playbooks, and enablement.

I worked with UX Research, Product Management, UX Design, and external AI consultants to develop and evaluate synthetic users.

My responsibilities included:

  • Developing and testing AI-powered user personas

  • Evaluating different LLMs and prompting approaches

  • Establishing methods for testing UX content and product workflows

  • Defining when synthetic users should and shouldn't be used

  • Creating reusable research playbooks, prompts, and templates

  • Facilitating workshops and supporting adoption

  • Applying the methodology to actual product designs

My goal was to make this capability useful beyond the initial experiment.

1. Building research-backed AI personas

We began with a two-week development sprint in December 2025.

The team explored how to build synthetic users using existing research, persona data, and realistic cybersecurity workflows.

Rather than relying on generic AI-generated personalities, we provided detailed context about users' responsibilities, experience levels, goals, and working environments.

We experimented with personas including:

  • Tier 1 SOC analysts

  • Security engineers

  • SOC managers

  • Engineering managers

One of the agents I worked on was Alex General, a Tier 1 SOC analyst.

The goal was to create agents that could evaluate designs from the perspective of a particular user type.

What we learned

More persona data didn't automatically produce better feedback.

When we combined information from multiple security engineers into a single persona, responses sometimes became less focused.

A more specific persona grounded in a particular user and use case could produce clearer, more useful responses.

This reinforced an important principle: the quality and relevance of the source context mattered more than the amount of information supplied.

2. Testing the technology, not just trusting the output

We evaluated synthetic user behavior across ChatGPT, Gemini, and Claude.

Our experiments included technical documentation, interface screenshots, prototype walkthroughs, and recorded videos.

In one comparative test, we evaluated responses to the same 20 questions across three categories:

  • General questions

  • Workflow questions

  • Open-ended questions

We compared baseline responses with those generated by a full-context synthetic user grounded in additional research.

The full-context approach produced better-quality responses across all three question types.

We also found that changing the response format to structured bullet points produced more detailed, useful feedback that better aligned with existing internal research.

Exploring multimodal workflows

We also tested whether AI could interpret product workflows from recorded prototype walkthroughs.

Early experiments revealed limitations. Some models struggled to summarize workflows accurately or required multiple tools to complete the process.

Gemini performed particularly well in a longer workflow test, producing a more scannable summary and allowing us to use a single tool for both workflow interpretation and feedback.

These experiments helped us understand where synthetic users could be valuable and where additional context, prompt refinement, or human evaluation was still necessary.

3. Turning experimentation into a repeatable process

The initial experiments demonstrated potential, but a promising prototype wasn't enough.

For synthetic users to become useful across the organization, teams needed guidance on how to use them responsibly and consistently.

I developed a set of resources to support self-service testing:

Synthetic Users Playbook

Guidance explaining how to select or create an agent, prepare design materials, formulate questions, interpret feedback, and understand limitations.

Prompting framework

Reusable question structures for evaluating content clarity, terminology, information hierarchy, workflows, and potential friction.

Testing templates

Standardized ways to capture questions, responses, findings, and follow-up actions.

Research outcomes templates

A repeatable structure for communicating what was tested, what the synthetic users identified, and what the team should investigate next.

Workshops and documentation

Training and centralized resources that helped designers, researchers, writers, and product managers begin using synthetic users independently.

I also helped establish clear boundaries for their use.

Synthetic users were intended to provide early directional feedback, surface hypotheses, and identify potential issues—not replace human research or validate user behavior conclusively.

4. Applying synthetic users to real product design

We moved beyond isolated experiments and applied synthetic users to actual product workflows.

One example involved SentinelOne's Data Loss Prevention (DLP) experience.

We tested a Figma prototype of a rule-creation workflow using an administrator synthetic user.

The exercise surfaced an important usability issue: the workflow assumed users would configure classifications separately before creating a rule.

The synthetic user feedback suggested that administrators might expect to create or add a classification directly within the rule-creation process.

This led to a proposed design improvement: adding an in-flow classification action so users wouldn't need to leave their current task.

The finding demonstrated how synthetic users could help teams identify assumptions embedded in product workflows.

It also showed the value of using AI feedback to generate specific design questions that could be investigated and validated further.

5. Scaling the practice across teams

By early 2026, the initiative had expanded beyond the original sprint.

I helped develop and publish the playbooks, prompts, workshop materials, and documentation needed to make synthetic user testing accessible to other teams.

The available tools included Gemini-based synthetic user agents and a workflow summary agent for interpreting images and video.

Designers could select a persona, prepare a prototype or documentation, ask structured questions, and capture findings using shared templates.

The approach was incorporated into UX and Product resources and shared through workshops and organizational updates.

Teams also began applying synthetic users to additional product experiences, including security analyst workflows.

Results and impact

Established a new AI-assisted research capability

We moved from experimental synthetic user prototypes to a documented, reusable workflow that product teams could access independently.

Demonstrated the importance of research grounding

Comparative testing showed that full-context synthetic users produced more useful responses than less-contextualized approaches.

Identified actionable product issues

Testing of the DLP rule-creation workflow surfaced a navigation and task-continuity issue that informed a proposed design improvement.

Enabled self-service experimentation

Playbooks, templates, workshops, and centralized documentation gave designers and product teams a consistent starting point for running their own evaluations.

Extended research beyond traditional study cycles

Synthetic users provided an additional way to explore design questions early, helping teams identify areas for further investigation before investing in formal user studies.

What I learned

This project changed how I think about AI's role in product development.

The most useful applications of AI don't simply generate more output. They help teams ask better questions, explore alternatives, and identify assumptions earlier.

I also learned that AI-generated research feedback is only as valuable as the context, evaluation methods, and judgment behind it.

A synthetic user isn't a replacement for a real person. It's a tool for making the design process more exploratory and informed.

My takeaway: AI can make research practices more accessible and scalable when teams combine strong source data, disciplined experimentation, clear limitations, and human judgment.

View next project