Share via

Inconsistent and Hallucinatory Responses from Copilot Studio Agent

Vivek Milind Shimpi 5 Reputation points
2025-12-10T11:31:59.68+00:00

Dear Microsoft Team,

I would like to raise a concern regarding an issue we are facing with Copilot Studio while building an HR bot using the Agent functionality. We have uploaded a detailed HR policy document of over 140 pages, and the bot is expected to provide answers strictly based on the content of this document.

However, we are experiencing inconsistent and unreliable behavior:

  1. Different answers for different users: When two users ask the same question, the bot sometimes provides a correct answer to one user and an incorrect or unrelated answer to another.
  2. Hallucinated and ambiguous responses: For example, when asking: “How many paid holidays are available for XYZ location employees?” —the bot should ideally respond with: “The specific number of paid holidays for employees working at the XYZ location in our organization is not directly mentioned in the available policy details.” Instead, the bot adds incorrect and ambiguous information such as: “However, the policy states that ABC Office Locations observe 9 paid holidays. If XYZ is considered an ABC Location, then employees there would be eligible for 9 paid holidays.” This additional statement is not accurate and is not supported by the document. Such hallucinations create confusion and reduce trust in the system.
  3. Inconsistent output over time: The same question sometimes yields the correct answer, and at other times, the bot provides a completely different or hallucinated response. This inconsistency is affecting our testing and workflow.

We request your support in investigating and resolving this issue. Our intention is to rely on Copilot Studio for accurate, document-based responses, but the current behavior is creating challenges in ensuring reliability for end users.

 

Microsoft Copilot | Other

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Karan Shewale 2,385 Reputation points Microsoft External Staff
    2025-12-11T09:12:46.3533333+00:00

    Hi Vivek,

    The behavior you are experiencing with Copilot Studio—where the agent provides inconsistent answers or hallucinates information—is a known challenge when working with large or complex documents in document-based AI agents. Copilot Studio agents use a combination of document embeddings and generative AI to answer questions. If the agent cannot find explicit answers in the uploaded content, it may generate responses that seem plausible but are not directly supported by the document, which causes hallucinations or inconsistent results across users or repeated queries.

    To reduce these issues, consider the following steps:

    Check Document Quality and Structure: Ensure the uploaded HR policy document is well-structured, clean, and clearly segmented. Long, unstructured documents increase the likelihood of inconsistent embeddings.

    Use Proper Chunking: Large documents should be split into smaller chunks when ingested. Copilot Studio agents perform better when each chunk contains self-contained context.

    Configure Retrieval Settings: Use stricter retrieval or context limits so the agent only considers relevant sections of the document when answering questions.

    Test with Ground Truth Questions: Validate the agent’s responses against specific questions with known answers. Adjust chunking or indexing if hallucinations persist.

    Review Agent Logs and Embeddings: Check the agent’s monitoring and analytics to see which sections are being used for answers. Misalignment in embeddings often leads to hallucinated outputs.

    Consider Fine-Tuning / Prompt Guidance: If the bot needs very strict adherence to the uploaded document, provide explicit prompt instructions like: “Answer only using the content from the uploaded HR policy. If the answer is not present, respond: ‘The policy does not provide this information.’”

    Currently, hallucinations and inconsistent responses are expected behavior for generative AI agents when the answer is ambiguous or not explicitly documented. Following the above best practices can significantly improve reliability and consistency.

    References:

    1. https://v4.hkg1.meaqua.org/en-gb/answers/questions/5620401/microsoft-copilot-agent-struggle-with-hallucinatio
    2. https://v4.hkg1.meaqua.org/en-us/microsoft-copilot-studio/guidance/
    3. https://v4.hkg1.meaqua.org/en-us/microsoft-copilot-studio/fundamentals-what-is-copilot-studio
    4. https://v4.hkg1.meaqua.org/en-us/microsoft-copilot-studio/nlu-boost-node

    Thank you.

    Karan Shewale.

    If this response resolves your issue, please Accept the answer and, if helpful, click the “Upvote” button. Your feedback helps us improve and assist others more effectively.

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.