Customer-facing assistants are attractive because support is expensive and repetitive questions dominate. The risks are not hypothetical: businesses have been held to commitments their chatbots made, and assistants have disclosed information they should not have.

What it says may bind you

Courts and regulators have shown little sympathy for the argument that an automated agent on your website does not speak for you. Discounts it offers, policies it describes, and assurances it gives can create real obligations. Constrain what it is permitted to state about pricing, refunds, timelines, and eligibility.

Ground it in your own content

An assistant answering from its general knowledge will confidently describe policies you do not have. Retrieval from your own documented content — help articles, policies, product pages — with instructions to decline when the answer is not there, is a substantially safer architecture than open generation, and it is also more accurate.

Set hard boundaries on actions

  • No processing of payments or refunds without a human decision.
  • No account changes, password resets, or access grants.
  • No disclosure of account details beyond what the authenticated session already permits.
  • A clear escalation path to a person, offered rather than hidden.

Handle data properly

Customers will paste personal information, card numbers, and case details into a chat box regardless of what you tell them. Decide where transcripts are stored, for how long, who can read them, and whether they leave your jurisdiction. If a third-party provider is involved, that is a processor relationship with contractual and disclosure obligations attached.

Disclose that it is automated

Several jurisdictions now require it, and beyond compliance it sets expectations correctly. People tolerate an automated system that is honest about being one and offers a human alternative; they react badly to discovering they were misled about who they were speaking to.

Consider the fraud angle specifically

For a business like ours, where distressed people arrive after a loss, an assistant that sounds authoritative about recovery prospects is actively dangerous. Any automated response touching outcomes must be conservative, disclaimed, and escalated to a person — a principle we apply to our own systems.