Predecessor flow — how the Express implementation actually works

Extracted 2026-09-10 by a full read of ~/projects/sap-apps/arena-plm (reference-only). File:line cites refer to that repo, mostly backend/src/arena-plm.js. This record informs the rebuild's design; the durable behavioral tombstones harvested from it live in KNOWLEDGE.md, and the requirements it produced are in PRD.md (ARN-010, sharpened ARN-002/004/006).

Shape of the system: not event-driven — an operator-driven staging workbench. Arena is the read-only source, SAP is the write target and the durable ledger. The webhook and poller exist but are effectively inert (§1). Real syncs are triggered by a human clicking Approve in the UI5 worklist.

1. Ingestion — three paths, only one live

Webhook (POST /api/arena/webhook, routes/arena-webhook.js) — mounted before the XSUAA gate, Basic auth (timing-safe), raw-text body of any content-type (delivery shape never pinned down), always answers 200 so Arena never retries. WEBHOOK_CAPTURE_ONLY logs+acks; otherwise pollOutboundIntegration({dryRun}) with dryRun defaulting true. The consumer never syncs: arena-plm.js:1252-1256 is a literal TODO — non-dry runs only mark event items reconciled in Arena. The webhook path writes nothing to SAP.

Outbound-integration pull queue (arena-plm.js:1197-1263) — Arena holds the cursor: GET /outboundintegrations/{guid}/events?itemsReconciled=false, per-event …/items, then PUT …/items/{itemGuid} {reconciled:true} (idempotent). No local state.

Change poller (routes/poll.js → arena-plm.js:1117-1191) — manual only (no scheduler exists in the repo). Watermark precedence: explicit since → newest A_ChangeMaster.ChangeNumberValidFromDate where startswith(ReasonForChangeText,'ARENA ') → defaults.arenaChangeCutoff; refuses non-dry runs with neither (POLL/NO_CUTOFF). Pages the whole Arena /changes feed client-side (no working server-side date filter), filters effectiveDateTime > since. Dedup via one bulk read of every ARENA -marked change master out of SAP into a Map. Real runs create change masters only.

2. Staging & approval

SQLite via node:sqlite on the ephemeral CF filesystem (db.js): approvals(id, item_number, change_type PRODUCT|CHANGE_MASTER|BOM|DOCUMENTS, status PENDING|APPROVED|REJECTED, arena_data, sap_preview, sap_result, error, reject_reason, …) — one row per (item × change type); queue lost on restart.

Seeding (POST /api/approvals/seed): previewFromArena() per item — all four syncs dry-run under Promise.allSettled (concurrency 6). No dedup on (item, changeType): re-seeding duplicates rows.
Approve: bulk-approve returns immediately; work runs in a detached async IIFE guarded by a module-level in-memory postJob singleton (progress source AND concurrency guard; forces instances: 1; on restart, in-flight jobs leave rows PENDING forever). chain-approve is synchronous per item, halts on first failure. Reject is a pure DB flip.
On approval only item_number is passed — the stored sap_preview is never replayed; every sync re-fetches Arena and recomputes. What was approved is not necessarily what gets written (→ rebuild ARN-010).
REJECTED is overloaded: human rejection AND SAP write failure land in the same status, distinguishable only by error vs reject_reason; no retry path from REJECTED except re-seeding (→ rebuild ARN-002 states).

3. Sync execution — the SAP write chain

Order from config/sync-chain-order.json: PRODUCT → CHANGE_MASTER → BOM → DOCUMENTS (CM's material object record needs the material; BOM's EngineeringChangeDocForEdit needs the CM). Services as in PRD interface contracts (V4 product, V2 product fallback, BOM write ;v=0002 / read v1, change master ;v=0002, CV attachment).

PRODUCT (:1336-1663) — exists-check (V4 GET, 404 ⇒ create path). Create: ProductType strictly from Arena attr "SAP Product Type" (missing ⇒ 422 SYNC/NO_PRODUCT_TYPE — deliberately "not a material"); UOM via config + hardcoded ISO map; plants from attr "SAP Plant" through plantMapping; weight g→×1000, dims mm→×1000; description truncated 40. Deep insert with _ProductDescription, _ProductPlant only when no AvailabilityCheckType; else per-plant V2 A_ProductPlant POSTs; ProcurementType requires MRPType in the same ProductPlantSupplyPlanning PATCH. Best-effort CM link keyed on startswith(ChangeNumberDescription,'ECO:') — a different key than everything else (landmine). Update path: ProductType/BaseUnit immutable; ProductDescription PATCH by full key (nav PATCH → 501), POST fallback on 404/405.

CHANGE_MASTER (:1794-1873) — EFFECTIVE revision with change.number via /revisions (/items/{guid}/changes 404s — does not exist). Double-condition dedup: ReasonForChangeText eq 'ARENA <eco>' AND a material object record for this item. Create: A_ChangeMaster with ChangeNumberDescription: "ECO: title"[0..40], /Date(ms)/ valid-from, ReasonForChangeText: "ARENA <eco>"; then object-type assignments Material + MaterialBOM (IsActiveForChangeNumber: true mandatory — else 201 + silently discarded), then the material object record. Errors returned, not thrown.

BOM (:1665-1792) — first mapped plant wins; lines quantity > 0, item numbers (i+1)*10 zero-padded 4; category via bomlineByCategory default 'L'. Opportunistic ECM lookup (bare catch — silently drops ECM and creates the BOM plain if the object-type assignment fails: traceability quietly lost). Component pre-check: if all components missing in SAP → early return, no orphan header (SAP's real error is an opaque consistency-check message). Header POST with EngineeringChangeDocForEdit (never EngineeringChangeDocument — creatable=false, silently dropped), validity end /Date(253402214400000)/. Items POSTed serially; each item re-reads the header on the v1 read service; ECM BOMs must carry HeaderChangeDocument. BOMItemIsSalesRelevant:'X' hardcoded. Partial failure is a return value. Not idempotent — re-approval creates a second BOM header.

DOCUMENTS (:1875-1913) — URL links only, no bytes: filters files to storageMethodName === 'WEB' && file.location (blob-stored files silently skipped), then V2 function import CreateUrlAsAttachment with hardcoded BusinessObjectTypeName BUS1001006, MIMEType 'text/url', description " Rev "[0..40]. The editable arena-doc-category-to-sap-doctype.json config is ignored by code. Not idempotent — re-approval duplicates attachments.

Cross-cutting: CSRF token+cookie cached per service, invalidated only on 403 with exactly one retry; PRODUCT services reject bare-root token fetch (seed from Product?$top=1 / A_Product?$top=1). Arena 401 → re-login, one retry. No backoff, no retry queue, no transactions. Idempotency exists only for CHANGE_MASTER (double-check) and weakly for PRODUCT (exists→PATCH).

4. Arena reads

Session POST /login {email, password, workspaceId} → arena_session_id header, cached 24 min vs 25-min expiry, single-flight promise lock, one 401 retry. Endpoints used by sync: /items?number= (resolves EFFECTIVE), /items/{guid}, /items/{guid}/revisions, /items/{guid}/bom, /items/{guid}/files, /changes/{guid}, /changes?limit&offset, /changes/{guid}/items, /settings/items/categories, outbound-integration trio. Write-back (POST /changes/statuschanges) is parked behind ARENA_WRITEBACK_ENABLED.

Load-bearing subtlety — WORKING-revision overlay (:508-552): the seven SAP classification attributes (SAP Object Type / Product Type / Product Category / Sub-Category / Product Family / Business Unit / Plant) are only writable on the WORKING revision (Arena error 3224), while /items?number= returns the EFFECTIVE revision — so those seven are merged from WORKING over EFFECTIVE. The overlay is best-effort: a transient Arena failure silently loses "SAP Product Type" and turns a valid sync into NO_PRODUCT_TYPE.

5. Config actually read vs dead

Live (read by sync code, hot-reloaded, git-backed via external sap-apps-config repo): defaults.json, plant-mapping.json, uom-arena-to-sap.json, material-group-by-product-type.json, availability-check-by-product-type.json, lifecycle-to-xplant-status.json, bomline-category-by-arena-category.json, sync-chain-order.json.

Dead (loaded, zero read sites — confirms the KNOWLEDGE.md landmine): arena-category-to-sap.json, procurement-type.json (code uses a hardcoded map keyed on the first char of Arena "Item Type"), catalog-to-planning-strategy.json. Ignored-but-editable: arena-doc-category-to-sap-doctype.json. Hardcoded despite the "no hardcoding" comment: ISO UOM map, status map, ARENA marker, procurement map, service paths, working-attrs list, BOM validity end, sales-relevant flag, Language 'EN', 40-char truncations, BUS1001006.

6. Fragilities the rebuild must not inherit

In-memory postJob singleton = progress + concurrency guard; restart orphans PENDING rows; forces instances: 1.
Fire-and-forget approval worker; no resume, no cancellation.
Seed dedup absent; BOM and DOCUMENTS syncs non-idempotent.
REJECTED overload (human vs SAP failure).
Silent ECM degradation (bare catches around ECO lookup, object-type assignment, revision overlay).
Two disagreeing CM lookup keys (ChangeNumberDescription startswith vs ReasonForChangeText eq; 40-char truncation can split them).
The ARENA <eco> free-text marker is the entire ledger — dedup key, watermark, and status all hang on an editable 40-char field.
O(history) paging on every poll (Arena full feed; up to 100k SAP change masters).
Arena credentials base64 in SQLite on the ephemeral filesystem; the whole approvals queue dies on restart/redeploy.
Approved preview is decorative — execution re-fetches and recomputes.

9.8 KiB Raw Permalink Blame History Unescape Escape