How LLMs (Large Language Model) choose next action

How do an AI model like ChatGPT and Google Gemini decide their next move in a conversation or workflow? This question sits at the heart of understanding how LLMs choose next action: a process that determines the accuracy, relevance, and overall effectiveness of AI-driven interactions.

A Large Language Model (LLM) operates by predicting the next word, phrase, or instruction based on the context it receives. While this may sound simple, it involves complex probability calculations, transformer-based architectures for context management, and adjustments informed by historical data. For B2B, eCommerce, and SEO strategies, understanding this process means gaining greater control over chatbot performance, product recommendations, and personalized content delivery.

In this article, we will break down the mechanisms behind how LLMs determine the “next action,” explore the prompting techniques that influence these decisions, and highlight insights from practitioners like Luca Fancello that can help you design AI workflows with precision and measurable business impact.

Understanding the core mechanism of Large Language Model decision-making

At their core, Large Language Models are probability engines. An LLM such as GPT-5 predicts a distribution over the next token given the current context, then selects or samples the next token according to a decoding strategy. This loop repeats step by step to produce coherent outputs. Tokens are the basic units the model reads and writes, covering whole words, subwords, spaces, and punctuation.

This prediction process is enabled by the Transformer architecture, whose attention layers learn which parts of the prompt matter most when inferring the next action. In practice, that means the model can consider long-range dependencies without recurrence and still stay efficient at scale.

You can think of the decision flow in four steps:

1. Tokenization: Split the input into tokens that the model can process.
2. Context mapping: Attention layers compute relationships among tokens.
3. Probability distribution: The model assigns log-probabilities to all candidate next tokens.
4. Decoding: Choose the next token via greedy selection or sampling; common controls include temperature and top_p nucleus sampling.

Decoding details matters. Lower temperature pushes outputs toward more deterministic answers, while higher temperature increases variability. Nucleus sampling limits choices to the smallest set of tokens whose cumulative probability reaches a threshold top_p, a method introduced to reduce bland or repetitive text during generation.

Finally, GPT-5 itself is documented by OpenAI as the current flagship model, with a system card describing how it routes tasks and when it engages deeper reasoning. Referencing those materials helps anchor any discussion of next-action behavior in an up-to-date model specification.

Expert perspectives on how Large Language Models choose next action

A key reference for understanding how LLMs choose next action when it comes to citing or crawling websites is Jérôme Salomon’s LinkedIn post. As a senior technical SEO expert at Oncrawl, he broke down the process of how ChatGPT Search evaluates and selects which sources to crawl and include in its answers.

The ChatGPT search process

Based on Jérôme’s research and direct conversations with the support team, the steps behind ChatGPT Search can be summarised as follows:

1. Prompt transformation: the user prompt is converted into one or more queries.
2. Search engine response: Bing sends back a list of results, including URLs, titles, snippets, and metadata.
3. Source selection: ChatGPT chooses relevant results and sends its bot to crawl the content in real time.
4. Citation: the crawled content is then included in the final answer with inline citations.

This means the model’s “next action” is not only predicting the next token but also deciding which external sources to trust and bring into the response.

The four criteria for source selection

ChatGPT relies on four main criteria when deciding which sites to crawl:

• Title: How descriptive and aligned it is with the query.
• Snippet/meta description: the clarity and accuracy of the preview text.
• Freshness: whether the content is up to date.
• Domain credibility: the authority of the site (Important: “domain authority” here isn’t a technical score like Moz’s – it’s simply the model’s perception of trust based on its training data.)

Together, these factors shape the model’s next action in terms of external citations. Titles and meta descriptions remain crucial signals, echoing traditional SEO fundamentals but reframed in a generative AI context.

Actionable tips for brands

For businesses aiming to appear as a cited source in ChatGPT Search or similar generative systems, it is recommended to:

• Write descriptive, specific, and accurate titles.
• Ensure your meta descriptions reflect the actual content.
• Prioritise clarity over clickbait.

By aligning site metadata with these selection signals, brands can improve the likelihood that an LLM will choose their content as the next action in a search-driven interaction. This analysis focuses on ChatGPT Search. Other LLM-powered engines, such as Google Gemini, Bing Copilot, or Perplexity AI may apply similar but distinct selection processes depending on their training data, retrieval pipelines, and citation policies.

Prompting strategies that influence next action

The way you prompt an LLM directly shapes how it chooses the next action. Beyond the underlying transformer mechanics, the structure and clarity of the prompt act as guardrails that guide decision quality.

Zero-shot, one-shot, and few-shot prompting

Different prompting styles change how much context the model has before making a decision:

• Zero-shot: the model is asked to act without examples. Useful for general instructions, but may lead to generic or uncertain actions.
  
• One-shot: the prompt includes a single example to anchor the response. This helps the model align more closely with the desired action format.
  
• Few-shot: the prompt includes multiple examples, giving the model a stronger pattern to follow. Few-shot prompting is particularly effective in narrowing the model’s next action within complex workflows.

Research has shown that few-shot prompting can significantly improve accuracy, especially for reasoning tasks or structured outputs. For businesses, this means you can reduce variability in customer-facing chatbots or automated marketing flows by seeding prompts with examples that match your tone and objectives.

The role of historical and contextual data

Context history is another major factor in determining the model’s next action. The longer the conversation window, the more signals the model can weigh before acting. However, too much context can cause hallucinations or exceed the model’s token limits.

One study on long-context LLMs highlighted that relevant historical context improves output precision, but irrelevant or excessive context can dilute accuracy. This makes context curation a critical skill: knowing which parts of history to retain and which to drop.

In practice:

• For support bots, include recent conversation history plus the ticket status, but avoid dumping entire customer records.
• For SEO and content automation, provide structured metadata such as product attributes or geo-targeted keywords to help the LLM align next actions with search intent.
• For eCommerce recommendations, feed in session-specific data (cart contents, browsing category) rather than full order history to keep the model focused.

Designing prompts for decision quality

Clear prompts consistently outperform long or vague ones. The most effective prompts usually contain:

• a single clear objective
• a defined set of allowed actions
• concise inputs (structured if possible)
• explicit decision rules for selecting the next action

OpenAI’s prompt engineering guide underscores this principle: precision in wording often leads to higher instruction-following accuracy than adding extra detail without structure.

Practical applications for B2B, eCommerce, and SEO

Understanding how LLMs choose next action is not only useful for technical AI design but also for business applications where precision directly impacts revenue and visibility.

Smarter chatbots and support flows

In B2B and eCommerce, customer support bots often face branching decisions: provide an FAQ answer, escalate to human support, or trigger a workflow. Structuring prompts with explicit decision rules ensures the LLM selects the most efficient next action. For example, adding a simple instruction such as “only escalate if no knowledge base match is found” can reduce unnecessary handovers and improve response time.

Personalised marketing automation

Marketing teams using platforms like Shopify or Klaviyo can design LLM-powered campaigns that adapt to customer signals in real time. By feeding in session data, product preferences, and engagement history, the model can choose the next action: send a reminder email, generate a discount code, or recommend a complementary product. This bridges predictive AI with actionable retention strategies.

Content optimisation for generative search

Generative AI is reshaping how content gets discovered. Since ChatGPT and other AI engines select sources based on signals like titles, snippets, freshness, and credibility, aligning metadata with these criteria helps brands become “the chosen source.” This is where Generative Engine Optimization (GEO) becomes critical.

GEO ensures that content is optimised not just for traditional search engines but also for generative engines that pull, evaluate, and cite content dynamically. By designing content with clear, accurate metadata and structured context, businesses can guide the LLM’s next action toward selecting their site as the authoritative answer.

Strategic SEO advantage

For SEO teams, the implication is clear: prompt design and metadata are now inseparable from search strategy. Crafting page titles and descriptions that reflect genuine value, maintaining freshness through frequent updates, and building domain authority all feed into the LLM’s next action logic. In other words, optimising for LLMs is about being chosen in generative search responses.

Ethical and geographical considerations

When analysing how LLMs choose next action, it is important to distinguish between two perspectives. Section 5 addressed Generative Engine Optimization (GEO) as a strategy to shape how LLMs select and cite content. Here, “geo” refers to geographical factors that influence how models behave across regions and cultures.

Bias and fairness

LLMs inherit patterns from the datasets they are trained on. This can lead to biased decisions when choosing next actions, such as favouring dominant languages or larger brands while underrepresenting smaller, local players. For companies expanding globally, this raises the need to evaluate and adjust AI outputs to ensure fairness and inclusivity.

Transparency and accountability

Because LLMs rarely expose the reasoning behind a next action, businesses face challenges in regulated industries like healthcare, finance, or government services. Logging decision flows, monitoring action paths, and implementing human review become essential for compliance and building user trust.

Localisation and cultural nuance

Next actions that work in one geography may fail in another. A recommendation system in the US might suggest products in imperial units, while in Europe the same model must adapt to metric standards. Cultural nuance matters too: tone, idioms, and even imagery can shape whether an action feels relevant or inappropriate. For global eCommerce, prompt design and metadata should reflect local preferences to increase acceptance.

Balancing automation with human oversight

Although automation scales decision making, relying solely on AI increases the risk of errors or cultural missteps. Introducing human checkpoints, such as approval steps for sensitive outputs or region-specific QA, ensures that next actions remain ethical, contextually aware, and brand-safe.

Note: in this section, geo refers to geographical localisation, not to Generative Engine Optimization (GEO) which was covered earlier. Both perspectives are essential: one shapes how content is selected by AI engines, the other ensures that those actions remain relevant and responsible in diverse markets.

Turning LLM decisions into business impact

Understanding how LLMs choose their next action reveals more than just technical detail. It shows how probability, prompting, and context shape the way AI interacts with users, selects information, and drives business outcomes. From Jérôme Salomon’s breakdown of ChatGPT Search signals to the role of metadata in Generative Engine Optimization, the lesson is clear: brands that structure their content and workflows with precision are more likely to be chosen, by both algorithms and customers.

For eCommerce and B2B organisations, this means designing prompts with explicit rules, curating context carefully, and aligning metadata with the signals that generative engines value. It also means adapting those actions ethically and geographically, ensuring that AI-driven decisions remain relevant across cultures and compliant with regulations.

Flatline helps businesses bridge this technical and strategic gap. As a Shopify Plus Partner and automation expert, we design workflows and AI-powered strategies that connect customer journeys with measurable business impact.

Looking to align your AI and eCommerce strategies with how LLMs really work? Get in touch with our team and start building systems that are not only automated but intelligently chosen.