Miftāḥ – Offline Qur’an Search with Exact Letter Positions, Morphology and Topic Clusters

Miftāḥ — offline Qur'an search and analysis engine

Miftāḥ — مِفتاح

> 🚀 An offline Qur’an search engine that treats position as the answer: 325,386 letters, each with an address in its word, its verse and the whole muṣḥaf — tashkeel-aware, with morphology, root graphs and topic clusters, and an engine written in Python’s standard library alone.

GPL-3.0 Python 3.14 stdlib only 61 tests github.com/dailybarid/miftah

Miftāḥ means key. The project started as one line in a file called IDEA.md, all lowercase, typo included: “an davnced quran search engine”. That was the whole brief — not another Qur’an website, but the search you want when you are studying a passage and need something precise.

🔍 Why ordinary search fails on Arabic

Ordinary text search works because text is stable. Arabic script is not. The same word appears in several legitimate shapes: the ʿUthmānī rasm of the muṣḥaf, a variant that writes the dagger alif as a full alif, and the plain Imlāʾī spelling most of us type. Add vocalisation and a word like «الصلاة» has a dozen surface forms — all the same word to a reader, none of them equal as strings.

Two consequences follow, and most tools handle only the first. A query has to reach the same place whichever spelling the reader typed; and a reader who did type the marks means them, so الصلاة and ٱلصَّلَوٰةَ are not the same request. There is a third question this project is built around: when search returns “2:255”, is that an answer? For someone studying the text, usually not. They want to know where in the verse — which word, which letter — and how that letter’s position relates to the whole muṣḥaf.

🔢 The numbers

Every figure below is asserted in the test suite or reproducible from the built database, and every source file’s SHA-256 is pinned in the build report.

  • 6,236 āyāt · 77,433 words · 325,386 letters
  • 290,931 diacritics and small Qur’anic marks · 695,149 characters · 36 distinct letters
  • 1,642 roots · 4,832 lemmas · 49,967 words carrying a root (64.5%)
  • 14,657 distinct normalised word forms · 128,223 morphological segments · 15 sajdas
  • 61 unit tests · 25 HTTP API endpoints · 0 third-party libraries in the engine

✨ What you actually get

  • Exact position for every hit: sūra, āya, word, letter-in-word, letter-in-āya, letter-in-the-whole-muṣḥaf, juzʾ, ḥizb, rubʿ, manzil, rukūʿ, page — plus abjad value and part of speech.
  • Tashkeel-aware matching: marks the reader typed must exist on the same letter; a mark-free query is an ordinary match.
  • Letter-level modes: inside a word, whole word, word start, word end, consecutive letters anywhere, letters contiguous across word boundaries, and multi-word phrases with a proximity window.
  • Every word is a tag: all occurrences, forms, per-sūra distribution, first and last occurrence, gaps, and the root.
  • Graphs and clusters: root co-occurrence network, per-āya word→root graph, radial mind map, and 75 topic axes.
  • Fully offline: fonts and D3 vendored, corpus committed, database built locally. The only network call is the optional recitation audio.

🧠 Under the hood: an index built from letters

The unusual choice at the heart of the project is that the index is letter-level, not word-level. Every word is stored in three orthographic forms, and each form carries maps from letter position to word and from index-in-form to index-in-text, so when a query matches the Imlāʾī form the engine still reports the ʿUthmānī position. That is what makes the rest possible.

Because every letter has an address, the engine can do things a word-level index cannot. Letter numbering does not restart at each āya: it runs through the whole muṣḥaf, from the first letter of 1:1 to the last letter of the revelation at position 325,386. And letters that touch across a word boundary are visible: type محمدرسول and it finds the place where «مُّحَمَّدٌ» ends and «رَّسُولُ» begins — 48:29, and nowhere else.

Morphology comes from the Quranic Arabic Corpus (University of Leeds). Aligning an external analysis onto the ʿUthmānī text is where such projects usually leak quality — a silent off-by-one and half the roots land on the wrong word — so the alignment is verified per verse and unmatched_words must be 0. Where the two corpora genuinely disagree (four āyāt are merge points — 2:181, 8:6, 13:37 and 37:130 — such as بَعْدَمَا), the difference is recorded in the build report instead of being smoothed over.

⚡ Decision: tashkeel as a promise

“Respect tashkeel” is implemented strictly: every mark the reader typed must exist on that same letter. It is a good illustration of why reference numbers matter — the word «الصلاة» occurs 58 times, and with full vocalisation as typed it matches 47. Both numbers are asserted in the tests, so a regression in the tashkeel logic cannot pass unnoticed.

⚡ Decision: greedy modularity, not label propagation

Roots that appear together across many verses are semantically related. Build a weighted graph — edge weight = shared verses — and cluster it, and you get something close to the classical themes of the text. The first attempt used label propagation, the obvious choice. It collapsed: one cluster swallowing 836 roots, the entire graph as a single “topic”.

The working algorithm is greedy modularity (CNM) with two safeguards that make collapse impossible: edges are pruned first (minimum co-occurrence, a PMI threshold, and each root’s strongest 25 neighbours), and a merge that would exceed --max-size is simply refused. Defaults give 75 topic axes covering 50.9% of the corpus’s roots, the largest holding 34 roots, in 0.2 seconds — with nothing but the standard library. Axes are labelled from their internally strongest roots rather than the corpus’s most frequent ones; otherwise every axis is labelled «الله · قول» and they all look alike.

Drawing those relationships produced three bugs that all looked identical on screen (“the nodes are overlapping”): a force layout ticked only on requestAnimationFrame never ran at all in a background tab; a missing weight field made every coordinate NaN; and a 50-word chain of linked words cannot be force-laid-out at all, because physics stretches a path graph into a line thousands of pixels long. The fixes were a synchronous layout pass, guards at the data boundary, and a deterministic layered layout for verses. The lesson generalises: when a visualisation looks wrong, measure it — counting overlapping circles and text boxes at three window widths beats squinting at a screenshot.

🖼️ Screenshots

The interface is Arabic and RTL by design; the numerals are Latin because that is how the owner reads them.

📦 Install

git clone https://github.com/dailybarid/miftah.git
cd miftah
./run.sh                 # builds the database on first run (~20 s), serves on 127.0.0.1:8770

The only dependencies are fastapi and uvicorn; the build script, the search engine, the morphology lookups, the statistics and the clustering all run on Python’s standard library. Interactive API docs at /api/docs.

⚠️ What it does not do, honestly

  • Recitation audio needs the network. Everything else is offline, but the five reciters stream from everyayah.com. Nothing loads until you press play.
  • “325,386 letters” excludes the small Qur’anic marks (U+06E5, U+06E6, U+06E2). They are classified separately and counted inside the 290,931 marks figure, so the two numbers are consistent by construction — but they are not the same measurement.
  • Two accuracy notes are properties of the sources, not bugs: four āyāt are merge points where one corpus word spans two ʿUthmānī words (2:181, 8:6, 13:37, 37:130), and the basmala embedded in the first āya of 112 sūras is excluded from statistics — which is why «بسم الله الرحمن الرحيم» matches exactly twice, 1:1 and 27:30.
  • The corpus data is not mine to relicense. The Tanzil texts are CC BY-ND 3.0 and the morphology is GPL, so the repository carries a full attribution notice.

📜 License

Miftāḥ is free software under the GNU General Public License v3.0 or later. The Qur’anic text is redistributed unmodified with attribution to Tanzil (CC BY-ND 3.0); the morphology is from the Quranic Arabic Corpus (GPL); fonts and D3 keep their own licences — the KFGQPC muṣḥaf font ships unmodified under its own end-user agreement, which permits free use, copy and distribution but forbids sale and modification. The full detail is in NOTICE.md in the repository.

💡 Author

Khaled Mimoune — I read what I build. Code, screenshots and the verified reference numbers: github.com/dailybarid/miftah. If you study the text, the thing worth trying is a letter query with tashkeel: type the marks you mean and watch the engine insist they sit on the same letter.

share:


Discover more from Khaled Mimoune

Subscribe to get the latest posts sent to your email.

Comments

Leave a Reply