Associations and foundations sit on a lot of knowledge: statutes, decisions, course material, handbooks, years of answers to the same member questions. An AI assistant that can answer from that knowledge is an obvious idea. The obvious implementation — a public chatbot with the documents pasted in — is the wrong one for most organizations.
It sends members’ questions, and often personal details, to an external provider. And it answers fluently even when it does not know, which is worse than not answering at all. Here is the architecture we use instead.
The architecture in five parts
- 1.Documents are split into meaningful sections — a paragraph of the statutes, a lesson, a decision — each with its source.
- 2.Each section is turned into an embedding, a numerical fingerprint of its meaning, and stored in a vector index inside the organization’s own database.
- 3.A question is matched against that index; the most relevant sections are retrieved.
- 4.A language model writes the answer from those sections only, with creativity set to zero.
- 5.The answer is returned with citations, so every statement can be checked in seconds.
This pattern is called retrieval-augmented generation. What makes it private is where each part runs.
Running it on your own server
For the Verein für holistische Gesundheit, every part runs on the association’s own server: the documents, the vector index, the embedding model and a compact open-weights language model. No question and no document leaves the association’s infrastructure. More than 600 sections of the association’s material are indexed.
Guardrails for the domain
Every organization has things its assistant must never say. In a health-related association, the assistant must not give medical advice or make health claims; nowhere should it reveal personal contact details. These rules are enforced in the instructions, in what is indexed, and in tests — not left to good luck.
The acceptance catalogue
Before an assistant is switched on for members, it has to pass an exam. We collect real questions — including tricky ones that should be refused — and define what a correct answer must contain and must not contain. The association’s assistant answered its 29-question catalogue without a single error before going live, and the catalogue is re-run after every change.
Transparency under the EU AI Act
People must know when they are talking to an AI. The assistant says so, its answers carry their sources, and the system is documented. Building these duties in from day one is far easier than adding them later.
Where such an assistant helps most
- Members asking about rules, fees, procedures and deadlines laid down in statutes and decisions.
- Course participants looking for the passage that explains a topic.
- Boards and staff searching past decisions and minutes.
- Onboarding new volunteers, who otherwise ask the same questions as everyone before them.
Checklist
- 1.Decide where data may live — and choose the hosting accordingly.
- 2.Index your own documents with their sources; nothing else.
- 3.Answer only from retrieved passages, with citations, at zero creativity.
- 4.Write down what the assistant must never say, and test it.
- 5.Pass an acceptance catalogue of real questions before go-live; re-run it after every change.
- 6.Tell users they are talking to an AI.