What does AI say about my business when someone asks in Welsh?

An assistant that searches the web builds its answer from the pages it finds for the question it was given. Ask in English and the likely pages are English ones. Ask in Welsh and the words being searched for are different, so the pages found may be different too: your Welsh pages if they say enough, a directory if they do not, or nothing at all.

So a business can be described well in one language and badly in the other, with nothing on the site obviously broken. The common causes are ordinary: Welsh pages that carry a welcome but not the facts, opening hours that were updated on the English page only, or a Welsh page that never says what the business actually does.

You cannot see any of that from the English answer. The only way to know what a Welsh question gets is to ask one.

Pick the assistants your customers are likely to use. ChatGPT, Google’s Gemini and Microsoft Copilot are a sensible start: OpenAI, Google and Microsoft each say their assistant can search the web for an answer and show where it came from. Start a new conversation for every question, so an earlier one does not shape the next.

Ask the same four questions in each language, in the words a customer would use. Write them down once and reuse them exactly each month, because a changed question gives a changed answer and you will not know which change you are looking at.

An invented example: a bike shop in Bethesda asks “What can you tell me about [shop name] in Bethesda?” and “Beth wyddoch chi am [enw’r siop] ym Methesda?”, then “Where can I get a bike serviced near Bethesda?” and “Ble alla i gael gwasanaethu beic ger Bethesda?”. Note the Welsh place name changes after “ym”; ask the way people actually write it. Four questions, in two languages, with three assistants is twenty-four answers: about fifteen minutes.

Keep one row per question, per language, per assistant, in a spreadsheet or on paper. The value is in comparing rows from month to month, not in any single answer.

Open the sources every time. ChatGPT lists the pages a searched answer used, and Gemini shows a Sources button when it has links for a response; Google’s help says that when the button is missing, Gemini gave no links for that answer. The sources tell you which page of yours, or of somebody else’s, the answer was built from, and that is where any fix starts.

OpenAI also warns that search results and citations can be incomplete, outdated or wrong, and that ChatGPT uses your approximate location for local results. Run the test from the same place each month, and include the town in the question, so that a change in the answer is more likely to be a change in what the web says.

Not named when asked by name, in either language: the assistant found nothing that clearly describes you. Start with your own home page, saying in plain text what you do, where and for whom, and with your listings, which are often what an answer falls back on.

Named in English but not in Welsh: your Welsh pages are not saying enough, or are not being recognised as the Welsh version of the English ones. Check that each Welsh page carries the same facts as its English partner, and that the two pages name each other. Google’s own guidance describes how a page tells it about its versions in other languages.

Right in one language and wrong in the other: one version of a page is out of date. A wrong fact in both languages, or one that matches an old directory entry, usually means your details differ from place to place; our guide on consistent business details covers finding and fixing them.

Told that you offer no Welsh service when you do: nothing on the site says so in words. A line on the contact page in both languages, “Croeso i chi gysylltu yn Gymraeg” and “You are welcome to contact us in Welsh”, gives an assistant something to quote.

Run the test once a month on the same day, and again a few weeks after any change you make to the site, so you can tell whether the change was noticed. Answers move from week to week for reasons that have nothing to do with you; a single bad month is a reason to look, not to rewrite.

Keep the questions, the assistants and the place fixed, and change one thing on the site at a time. That is the only way to know which fix made the difference.

If you want to see what a machine can read on a page before you ask an assistant anything, the free assessment on this site reads one address in either language and names what is missing.