173 lines
9.8 KiB
Markdown
173 lines
9.8 KiB
Markdown
# Predecessor flow — how the Express implementation actually works
|
||
|
||
Extracted 2026-09-10 by a full read of `~/projects/sap-apps/arena-plm`
|
||
(reference-only). File:line cites refer to that repo, mostly
|
||
`backend/src/arena-plm.js`. This record informs the rebuild's design; the
|
||
durable behavioral tombstones harvested from it live in `KNOWLEDGE.md`, and
|
||
the requirements it produced are in `PRD.md` (ARN-010, sharpened ARN-002/004/006).
|
||
|
||
**Shape of the system:** not event-driven — an **operator-driven staging
|
||
workbench**. Arena is the read-only source, SAP is the write target *and* the
|
||
durable ledger. The webhook and poller exist but are effectively inert (§1).
|
||
Real syncs are triggered by a human clicking Approve in the UI5 worklist.
|
||
|
||
## 1. Ingestion — three paths, only one live
|
||
|
||
**Webhook (`POST /api/arena/webhook`, routes/arena-webhook.js)** — mounted
|
||
before the XSUAA gate, Basic auth (timing-safe), raw-text body of any
|
||
content-type (delivery shape never pinned down), always answers 200 so Arena
|
||
never retries. `WEBHOOK_CAPTURE_ONLY` logs+acks; otherwise
|
||
`pollOutboundIntegration({dryRun})` with dryRun defaulting true. **The
|
||
consumer never syncs**: `arena-plm.js:1252-1256` is a literal TODO — non-dry
|
||
runs only mark event items reconciled in Arena. The webhook path writes
|
||
nothing to SAP.
|
||
|
||
**Outbound-integration pull queue (`arena-plm.js:1197-1263`)** — Arena holds
|
||
the cursor: `GET /outboundintegrations/{guid}/events?itemsReconciled=false`,
|
||
per-event `…/items`, then `PUT …/items/{itemGuid} {reconciled:true}`
|
||
(idempotent). No local state.
|
||
|
||
**Change poller (`routes/poll.js` → `arena-plm.js:1117-1191`)** — manual only
|
||
(no scheduler exists in the repo). Watermark precedence: explicit `since` →
|
||
newest `A_ChangeMaster.ChangeNumberValidFromDate` where
|
||
`startswith(ReasonForChangeText,'ARENA ')` → `defaults.arenaChangeCutoff`;
|
||
refuses non-dry runs with neither (`POLL/NO_CUTOFF`). Pages the whole Arena
|
||
`/changes` feed client-side (no working server-side date filter), filters
|
||
`effectiveDateTime > since`. Dedup via one bulk read of every `ARENA `-marked
|
||
change master out of SAP into a Map. Real runs create change masters only.
|
||
|
||
## 2. Staging & approval
|
||
|
||
SQLite via `node:sqlite` on the **ephemeral CF filesystem** (`db.js`):
|
||
`approvals(id, item_number, change_type PRODUCT|CHANGE_MASTER|BOM|DOCUMENTS,
|
||
status PENDING|APPROVED|REJECTED, arena_data, sap_preview, sap_result, error,
|
||
reject_reason, …)` — one row per (item × change type); queue lost on restart.
|
||
|
||
- Seeding (`POST /api/approvals/seed`): `previewFromArena()` per item — all
|
||
four syncs dry-run under `Promise.allSettled` (concurrency 6). **No dedup**
|
||
on (item, changeType): re-seeding duplicates rows.
|
||
- Approve: `bulk-approve` returns immediately; work runs in a detached async
|
||
IIFE guarded by a **module-level in-memory `postJob` singleton** (progress
|
||
source AND concurrency guard; forces `instances: 1`; on restart, in-flight
|
||
jobs leave rows PENDING forever). `chain-approve` is synchronous per item,
|
||
halts on first failure. Reject is a pure DB flip.
|
||
- **On approval only `item_number` is passed** — the stored `sap_preview` is
|
||
never replayed; every sync re-fetches Arena and recomputes. What was
|
||
approved is not necessarily what gets written (→ rebuild ARN-010).
|
||
- **`REJECTED` is overloaded**: human rejection AND SAP write failure land in
|
||
the same status, distinguishable only by `error` vs `reject_reason`; no
|
||
retry path from REJECTED except re-seeding (→ rebuild ARN-002 states).
|
||
|
||
## 3. Sync execution — the SAP write chain
|
||
|
||
Order from `config/sync-chain-order.json`: PRODUCT → CHANGE_MASTER → BOM →
|
||
DOCUMENTS (CM's material object record needs the material; BOM's
|
||
`EngineeringChangeDocForEdit` needs the CM). Services as in PRD interface
|
||
contracts (V4 product, V2 product fallback, BOM write `;v=0002` / read v1,
|
||
change master `;v=0002`, CV attachment).
|
||
|
||
**PRODUCT (`:1336-1663`)** — exists-check (V4 GET, 404 ⇒ create path).
|
||
Create: `ProductType` strictly from Arena attr "SAP Product Type" (missing ⇒
|
||
`422 SYNC/NO_PRODUCT_TYPE` — deliberately "not a material"); UOM via config +
|
||
hardcoded ISO map; plants from attr "SAP Plant" through `plantMapping`; weight
|
||
g→×1000, dims mm→×1000; description truncated 40. Deep insert with
|
||
`_ProductDescription`, `_ProductPlant` only when no `AvailabilityCheckType`;
|
||
else per-plant V2 `A_ProductPlant` POSTs; `ProcurementType` requires `MRPType`
|
||
in the same `ProductPlantSupplyPlanning` PATCH. Best-effort CM link keyed on
|
||
`startswith(ChangeNumberDescription,'ECO:')` — a different key than everything
|
||
else (landmine). Update path: `ProductType`/`BaseUnit` immutable;
|
||
`ProductDescription` PATCH by full key (nav PATCH → 501), POST fallback on
|
||
404/405.
|
||
|
||
**CHANGE_MASTER (`:1794-1873`)** — EFFECTIVE revision with `change.number`
|
||
via `/revisions` (`/items/{guid}/changes` 404s — does not exist).
|
||
Double-condition dedup: `ReasonForChangeText eq 'ARENA <eco>'` AND a material
|
||
object record for *this* item. Create: `A_ChangeMaster` with
|
||
`ChangeNumberDescription: "ECO: title"[0..40]`, `/Date(ms)/` valid-from,
|
||
`ReasonForChangeText: "ARENA <eco>"`; then object-type assignments Material +
|
||
MaterialBOM (`IsActiveForChangeNumber: true` mandatory — else 201 +
|
||
silently discarded), then the material object record. Errors returned, not
|
||
thrown.
|
||
|
||
**BOM (`:1665-1792`)** — first mapped plant wins; lines `quantity > 0`, item
|
||
numbers (i+1)*10 zero-padded 4; category via `bomlineByCategory` default 'L'.
|
||
Opportunistic ECM lookup (bare catch — silently drops ECM and creates the BOM
|
||
plain if the object-type assignment fails: traceability quietly lost).
|
||
Component pre-check: if all components missing in SAP → early return, no
|
||
orphan header (SAP's real error is an opaque consistency-check message).
|
||
Header POST with `EngineeringChangeDocForEdit` (never
|
||
`EngineeringChangeDocument` — creatable=false, silently dropped), validity end
|
||
`/Date(253402214400000)/`. Items POSTed serially; each item re-reads the
|
||
header on the v1 read service; ECM BOMs must carry `HeaderChangeDocument`.
|
||
`BOMItemIsSalesRelevant:'X'` hardcoded. Partial failure is a return value.
|
||
**Not idempotent — re-approval creates a second BOM header.**
|
||
|
||
**DOCUMENTS (`:1875-1913`)** — URL links only, no bytes: filters files to
|
||
`storageMethodName === 'WEB' && file.location` (blob-stored files silently
|
||
skipped), then V2 function import `CreateUrlAsAttachment` with hardcoded
|
||
`BusinessObjectTypeName BUS1001006`, `MIMEType 'text/url'`, description
|
||
"<item> <category> Rev <edition>"[0..40]. The editable
|
||
`arena-doc-category-to-sap-doctype.json` config is **ignored by code**.
|
||
**Not idempotent — re-approval duplicates attachments.**
|
||
|
||
**Cross-cutting:** CSRF token+cookie cached per service, invalidated only on
|
||
403 with exactly one retry; PRODUCT services reject bare-root token fetch
|
||
(seed from `Product?$top=1` / `A_Product?$top=1`). Arena 401 → re-login, one
|
||
retry. No backoff, no retry queue, no transactions. Idempotency exists only
|
||
for CHANGE_MASTER (double-check) and weakly for PRODUCT (exists→PATCH).
|
||
|
||
## 4. Arena reads
|
||
|
||
Session `POST /login {email, password, workspaceId}` → `arena_session_id`
|
||
header, cached 24 min vs 25-min expiry, single-flight promise lock, one 401
|
||
retry. Endpoints used by sync: `/items?number=` (resolves EFFECTIVE),
|
||
`/items/{guid}`, `/items/{guid}/revisions`, `/items/{guid}/bom`,
|
||
`/items/{guid}/files`, `/changes/{guid}`, `/changes?limit&offset`,
|
||
`/changes/{guid}/items`, `/settings/items/categories`, outbound-integration
|
||
trio. Write-back (`POST /changes/statuschanges`) is parked behind
|
||
`ARENA_WRITEBACK_ENABLED`.
|
||
|
||
**Load-bearing subtlety — WORKING-revision overlay (`:508-552`):** the seven
|
||
SAP classification attributes (SAP Object Type / Product Type / Product
|
||
Category / Sub-Category / Product Family / Business Unit / Plant) are only
|
||
writable on the WORKING revision (Arena error 3224), while `/items?number=`
|
||
returns the EFFECTIVE revision — so those seven are merged from WORKING over
|
||
EFFECTIVE. The overlay is best-effort: a transient Arena failure silently
|
||
loses "SAP Product Type" and turns a valid sync into `NO_PRODUCT_TYPE`.
|
||
|
||
## 5. Config actually read vs dead
|
||
|
||
Live (read by sync code, hot-reloaded, git-backed via external
|
||
`sap-apps-config` repo): `defaults.json`, `plant-mapping.json`,
|
||
`uom-arena-to-sap.json`, `material-group-by-product-type.json`,
|
||
`availability-check-by-product-type.json`, `lifecycle-to-xplant-status.json`,
|
||
`bomline-category-by-arena-category.json`, `sync-chain-order.json`.
|
||
|
||
Dead (loaded, zero read sites — confirms the KNOWLEDGE.md landmine):
|
||
`arena-category-to-sap.json`, `procurement-type.json` (code uses a hardcoded
|
||
map keyed on the first char of Arena "Item Type"),
|
||
`catalog-to-planning-strategy.json`. Ignored-but-editable:
|
||
`arena-doc-category-to-sap-doctype.json`. Hardcoded despite the "no
|
||
hardcoding" comment: ISO UOM map, status map, `ARENA ` marker, procurement
|
||
map, service paths, working-attrs list, BOM validity end, sales-relevant
|
||
flag, Language 'EN', 40-char truncations, BUS1001006.
|
||
|
||
## 6. Fragilities the rebuild must not inherit
|
||
|
||
1. In-memory `postJob` singleton = progress + concurrency guard; restart
|
||
orphans PENDING rows; forces `instances: 1`.
|
||
2. Fire-and-forget approval worker; no resume, no cancellation.
|
||
3. Seed dedup absent; BOM and DOCUMENTS syncs non-idempotent.
|
||
4. `REJECTED` overload (human vs SAP failure).
|
||
5. Silent ECM degradation (bare catches around ECO lookup, object-type
|
||
assignment, revision overlay).
|
||
6. Two disagreeing CM lookup keys (`ChangeNumberDescription` startswith vs
|
||
`ReasonForChangeText` eq; 40-char truncation can split them).
|
||
7. The `ARENA <eco>` free-text marker is the entire ledger — dedup key,
|
||
watermark, and status all hang on an editable 40-char field.
|
||
8. O(history) paging on every poll (Arena full feed; up to 100k SAP change
|
||
masters).
|
||
9. Arena credentials base64 in SQLite on the ephemeral filesystem; the whole
|
||
approvals queue dies on restart/redeploy.
|
||
10. Approved preview is decorative — execution re-fetches and recomputes.
|