# Tariq al-Haqq archive — backend upload spec, addendum: focused collections

**For:** the backend developer of the Dar el Haq app
**Extends:** backend upload spec 2.0 (parts with their indexes). Nothing in 2.0 changes.
**Content language:** Arabic only for now.

---

## 1. What this adds

Twelve "focused collections". Each one is a curated list of passages taken word for word from the magazine (prayers on the Prophet, dhikr, poems, seasons, biographies, and so on). Every passage points back to the article and page it came from, using the same `part_id` and `article_id` the archive already uses.

A collection is read-only content for the app: a list screen per collection, one row per entry, and a "where it appears" link from each entry to the article reader.

## 2. Zip layout

```
manifest.json
collections/salawat.json
collections/dhikr.json
collections/mudhakara.json
collections/akhlaq.json
collections/mawalid_hadarat.json
collections/qasaid.json
collections/awrad.json
collections/mawasim.json
collections/ziyarat.json
collections/adiya.json
collections/tarajim.json
collections/taarif.json
extras/youtube-candidates.json      not a collection, see section 7
extras/suggestions.json             not a collection, see section 7
```

`manifest.json`:

```json
{
  "schemaVersion": "collections-1.0",
  "collections": [
    {"id": "salawat", "title": "الصلوات على النبي", "order": 1, "part": 1,
     "file": "collections/salawat.json", "sha256": "…", "entries": 232, "shape": "entries"}
  ],
  "extras": [{"id": "suggestions", "file": "extras/suggestions.json", "sha256": "…"}]
}
```

- `order` in the manifest is the display order (1 to 12). Use it, not the `order` inside the collection file.
- `part` is only our delivery batch number (1 to 5). It has no meaning in the app.
- `shape` is `entries` for eleven collections and `parts` for `taarif` (section 5).
- Import from the combined zip `collections-all-import.zip`. The five per-batch zips have the same layout and exist only so the client can review in batches.
- A batch zip lists fewer collections, its `schemaVersion` may read `collection-1.0`, and its file names may differ from the collection id (for example `collections/tarajim-wal-maraji.json` for id `tarajim`); always take the id from the manifest.

## 3. The collection file

```json
{
  "schema": "collection-1.0",
  "collection": {"id": "dhikr", "title": "الذكر", "icon": "dhikr", "order": 2, "intro": "…", "counts": {…}},
  "entries": [ … ]
}
```

- `icon` is a key, not an image. The app maps it to its own artwork.
- `intro` is one short Arabic line for the top of the screen.
- `counts` is informational. Do not rely on its keys; they differ per collection.
- Some files carry extra top-level lists: `salawat` has `sayings` and `honorific_forms`, `mawasim` has `seasons`. Store them as JSON on the collection row (section 6).

### Entry: fields every entry has

| field | type | meaning |
|---|---|---|
| `id` | string | Stable and unique inside the collection (`sal-004`, `poem-112`, `ms-031`). Ids may have gaps. |
| `order` | int | Position inside the collection. |
| `refs` | array | Where the passage is printed. Never empty. See below. |
| `occurrences` | int | `refs.length`. |

### Entry: text

- Most collections: `text` (string, the passage as printed).
- `qasaid`: `lines` (array of `{"sadr": "…", "ajuz": "…"}`), plus `text` holding the same lines joined. Show `sadr` and `ajuz` as the two halves of one verse line. `ajuz` can be empty when the archive holds the line as one string; then show `sadr` alone on the line.
- `tarajim`: entries with `section: "tarajim"` have `title`, `byline` and `paragraphs` (array of strings); entries with `section: "maraji"` have `text` (the book title as printed).
- `text_normalised_for_matching` is the text without diacritics and with letter forms folded. Use it for search and for nothing else.

### Entry: optional fields

Store what you do not model as columns in one JSON column. The ones the app will use first:

| field | where | meaning |
|---|---|---|
| `kind`, `section`, `family`, `tags` | most | Grouping labels in Arabic. Use `kind` (or `section` then `kind`) for section headings. |
| `is_core`, `core_reason` | salawat, dhikr, awrad, adiya, akhlaq, mudhakara | Our proposal for a "main selection". The client may change it. |
| `variant_of_ids` | most | Ids of entries in the same collection whose wording is close but not identical. Show as "similar wording". |
| `benefit`, `when_to_say` | dhikr, adiya, salawat | Arrays of `{"text": "…", "ref": {…}}`: sentences of the magazine that state the benefit or the time. |
| `lead_in` | dhikr, adiya, qasaid | The words printed just before the passage. |
| `poet`, `poet_as_printed`, `title`, `line_count` | qasaid | Poem header. |
| `season_id`, `season`, `also_seasons`, `hijri` | mawasim | `hijri` is `{"month": 9, "day": 27, "day_basis": "…"}`; `day` can be null (whole month). This is our calendar key for linking a season to a Hijri date; it is not magazine text. |
| `stated_hijri_dates` | mawasim, mawalid_hadarat, ziyarat | Dates the magazine itself prints inside the passage. |
| `excerpt`, `excerpt_only` | part 4 collections, tarajim | The entry is part of a longer paragraph or article. |
| `also_in_collections` | mawalid_hadarat | The same passage is also an entry in another collection. |
| `quran`, `quran_refs_on_page` | akhlaq, mudhakara, adiya | Surah and ayah of a Qur'anic passage. |

### Ref

```json
{"part_id": "year-11", "year": 11, "hijri": 1381, "issue": 8, "page": 239, "printed_page": "١٥",
 "article_id": "year-11-a238", "article_title": "وصية والد لبنيه"}
```

- `part_id` and `article_id` are the ids of spec 2.0. `page` is the archive page number `n`; `printed_page` is what is printed on the paper.
- Optional: `page_end` (passage runs over pages), `as_printed` / `printed_as` / `printed_as_lines` (this place prints the same wording with a different spelling), `block`, `char_from`, `char_to` (exact position inside the article block).

## 4. Upsert rules

1. **One collection per transaction.** A collection that fails validation fails alone; the others still import.
2. **Replace, do not merge.** A collection file is always complete. On import, upsert the collection row by `id`, then delete its entries and refs and insert the new ones in the same transaction. An entry that is missing from a new upload was removed on purpose.
3. **Entry key is `(collection_id, id)`.** Entry ids are stable between uploads, so bookmarks and share links can use them. Ids are not unique across collections.
4. **Skip unchanged collections** when the manifest `sha256` equals the stored one.
5. **Refs must resolve.** Every `part_id` and `article_id` must exist in the archive tables. If the archive part is not imported yet, report it and keep the ref (do not drop the entry).
6. **A batch zip touches only the collections it lists.** Collections not in the manifest stay as they are.
7. **`?dryRun=1`**, background job, streaming and zip-slip checks: as in spec 2.0, section 6.
8. Uploading the same zip twice changes nothing.

The client reviews each collection and may ask for entries to be removed or shortened. That arrives as a new upload of the same collection id.

## 5. The `taarif` file (introduction to the Sheikh)

This one is a short document in sections, not a flat list:

```json
{"schema": "collection-1.0", "collection": {"id": "taarif", …},
 "parts": [{"id": "nasab", "title": "…", "intro": "…",
            "passages": [{"id": "ta-001", "paragraphs": ["…", "…"], "byline": "…", "refs": [{…}], "words": 127}]}]}
```

Import each passage as an entry of collection `taarif`, with the section id and title stored on the entry and `order` taken from the position in the file. Keep the paragraph breaks.

## 6. Suggested tables

```sql
collections (
  id          text primary key,        -- salawat
  title       text not null,
  icon        text,
  sort_order  int,                     -- manifest order
  intro       text,
  extra       jsonb,                   -- counts, seasons, sayings, honorific_forms
  sha256      text,
  updated_at  timestamptz
)

collection_entries (
  collection_id text references collections(id) on delete cascade,
  id            text,                  -- sal-004
  sort_order    int,
  kind          text,
  section       text,
  text          text,                  -- joined text; null for taarif
  search_text   text,                  -- text_normalised_for_matching
  is_core       boolean default false,
  body          jsonb,                 -- lines / paragraphs / every optional field
  primary key (collection_id, id)
)

collection_entry_refs (
  collection_id text,
  entry_id      text,
  seq           int,
  part_id       text,                  -- year-11
  article_id    text,                  -- year-11-a238
  issue         int,
  page          int,
  page_end      int,
  printed_page  text,
  extra         jsonb,
  primary key (collection_id, entry_id, seq),
  foreign key (collection_id, entry_id) references collection_entries on delete cascade
)
```

## 7. Upload API

```
POST /api/admin/collections/imports          body: the zip, or {"url": "<signed link>"}
GET  /api/admin/collections/imports/{jobId}  → {status, collections: [{id, status, entries, refs, unresolvedRefs, errors[]}]}
PUT  /api/admin/collections/{id}             one collection file, replaces that collection
```

Read endpoints the app needs:

```
GET /api/collections                         id, title, icon, order, intro, entry count
GET /api/collections/{id}/entries?kind=&core=1&q=&page=
GET /api/collections/{id}/entries/{entryId}  with refs
```

## 8. Not collections

- `extras/youtube-candidates.json`: links proposed for the "about the Sheikh" screen. Nothing in it is approved yet. Do not import it; the approved list will come separately.
- `extras/suggestions.json`: ideas for further collections, with sizes. For the client, not for the app.

## 9. Sizes

The combined zip is about 2 MB compressed, 11 MB unpacked, 5,064 entries in 12 collections. The largest file is `qasaid` (5.7 MB, 1,239 entries, 14,684 verse lines).
