Arabic RAG: how to build a knowledge assistant that cites its sources

Here is one way an Arabic RAG project can fail in its first week (an illustrative case). A clerk at a Jordanian water utility types "اجراءات الاستقاله" (resignation procedures) into the new assistant. Nothing comes back, although the HR manual covers resignation in full. Three things went wrong. The manual is a scanned PDF, and the OCR misread several words. The clerk typed the alef without its hamza and the final taa marbuta as a ha. And the keyword index matches spellings letter by letter. All three problems sit in the pipeline around the language model, and that is where to fix them.

Our position: fix document quality and Arabic spelling variation before you choose a model. Decide how the assistant refuses before you decide how it answers. The five steps below are in the order we recommend, each with a test. The last sections cover when not to build one at all.

Step 1: Check OCR quality on your scanned Arabic documents

Retrieval cannot find text that was never extracted. Separate born-digital files, whose text can be read directly, from scans, which need optical character recognition (OCR). Arabic scans bring their own trouble: connected letters, dots lost on faint copies, stamps and mixed English codes.

The open-source Tesseract engine has an Arabic model (ara). Its documentation on data files describes three model sets. The tessdata_fast set is the fastest and least accurate, and Linux distributions ship it. The tessdata_best set is the slowest and most accurate, and tessdata sits between them. Run Arabic and English together and test the best set on your own pages before accepting the default.

A simple gate: pick 20 to 30 representative pages and type the correct text of a few paragraphs by hand. Then measure the character error rate, the share of characters the OCR got wrong. Agree the threshold before you look at the results. Pages that fail are re-scanned or corrected, not indexed silently.

Illustrative example: a mid-sized Saudi insurer with old scanned claims manuals may find its benefit tables come out jumbled. The fix is to extract tables separately, each row with its column headers; no model setting repairs a broken table.

Step 2: Normalise Arabic spelling, but only in the search copies

Users tend to type Arabic faster and looser than documents are written. Normalisation maps spelling variants to one form so that a query and a document meet. Two rules hold throughout. Users see and citations quote the original text. Every query passes through the same rules as the copy it searches.

We recommend a heavier profile for keyword search and a lighter one for embeddings. The lighter one removes diacritics and tatweel but leaves taa marbuta and alef maqsura as written, because embedding models learn from normal writing. Test both on your evaluation set. Dialect words and Arabizi (Arabic in Latin letters, such as "kifak") need a separate step. It rewrites the question in standard Arabic before search.

Step 3: Split documents into passages and index them for hybrid search

Chunking that respects Arabic sentences

Generic splitters tend to cut documents on English punctuation or fixed character counts. Add the Arabic question mark (؟), comma (،) and semicolon (؛) to the boundaries. Follow the document's own structure: one article of a regulation, one FAQ question, one contract clause. Carry the document title and section heading with each chunk, so a passage on annual leave stays tied to the HR policy. Start at around 1,200 characters with one sentence of overlap. Change the size only when the evaluation set shows a better result.

Keyword and vector search, merged

Vector search uses embeddings, a numeric representation of meaning, to find passages that say the same thing in other words. With a multilingual model, this works across Arabic and English. Keyword search finds exact terms that vectors can blur, such as policy numbers and names. Arabic RAG needs both. In PostgreSQL, the pgvector extension stores embeddings and searches them with HNSW or IVFFlat indexes and a cosine distance operator. Its README suggests combining it with PostgreSQL full-text search. The two result lists are then merged with Reciprocal Rank Fusion or a cross-encoder.

On the keyword side, the PostgreSQL 12 release notes record that Snowball stemming for Arabic was added to full-text search. Stemming can help with formal Arabic and hurt names and dialect words. Compare the arabic configuration with the simple dictionary on your normalised text. The simple dictionary does no stemming: it lowercases tokens and drops stop words only if given a stop-word file. Filter by the user's groups inside the search query, so a passage they may not see is never retrieved.

Choose the embedding model by measurement. BAAI's bge-m3 model card, for example, describes support for more than 100 languages and inputs of up to 8,192 tokens. It lists dense, sparse and multi-vector retrieval, under the MIT licence. That is a reason to test it against another candidate on your own questions, not proof that it suits your documents. Where the models run is a hosting decision. Our guide to sovereign AI: private cloud or managed API covers the options.

Step 4: Make every answer cite a passage, or refuse

Instruct the model to answer only from the retrieved passages and to cite each claim by document, section and page. When the passages are silent, it replies "The documents I can search do not cover this" and points to a person, such as the HR inbox. Then check in code that every citation points to a retrieved passage. An answer without a valid citation is replaced by the refusal.

An assistant that guesses about leave entitlements or contract terms does more harm than one that says it does not know. Treat document text as data, never as instructions. A scanned memo that says "ignore your previous instructions" must not change the assistant's behaviour. Mark sources clearly in the prompt (a mitigation, not a guarantee) and put such a test in your evaluation set.

Illustrative example: a Gulf free-zone authority has some regulations only in English. Its assistant should cite the English text for an Arabic question and say so, rather than translate silently.

Step 5: Test your Arabic RAG assistant with questions split by dialect

Write questions with known answers before tuning anything. Staff may ask in Modern Standard Arabic, in Gulf, Levantine or Egyptian dialect, in Arabizi and in English. The set we recommend before go-live:

• At least 100 questions in the words real staff use, each labelled with its variety and the passage that answers it.
• About 10 questions the documents cannot answer, to test refusal.
• Questions whose answer sits in a table, a number or a scanned page.
• A few prompt-injection attempts written in Arabic and in Arabizi.

Measure three things for each variety. Recall at 5: does the right passage appear in the top five results? Citation precision: does each citation support its sentence? Refusal accuracy: does the assistant refuse when it should? Set pass thresholds before the first run, and report the weakest variety as well as the average. Rerun the set after every change to OCR, normalisation, chunking, the embedding model or the prompt. Our guide to Arabic LLM evaluation covers dialect test sets in depth.

When not to build an Arabic RAG assistant

Check these before you commit a budget:

If two or more of these apply, we recommend spending the first month on the documents and leaving the assistant for later.

How to start

Start an Arabic RAG project with one document collection staff ask about every week. Add one person who can judge whether an answer is right. Measure retrieval on a sample before anyone designs a chat window. Our Transformation Lab builds retrieval platforms under Custom AI Applications & Modernisation (TT3). The legacy systems page shows where this fits next to replacing spreadsheets and old systems. Document preparation and test questions are covered by Agent Knowledge Packs (AA3) and Agent Evaluation Sets (AA4). Not sure the use case is right? The AI Exposure Scan is a free 45-minute conversation that ends in a written next step.

Related lab: Transformation Lab ↗