Architecture, Entity Model, Relationships & Documentation — interactive reference for the All Point ecosystem.
An Astro 5 SSR application with React 19 components, Tailwind 4, and Radix UI primitives. Serves as the unified management UI for two Layer 2 platform services:
Security & Permissions: Users, Roles, Permissions, Scopes, Clients, Areas. General: Companies, Applications (OAuth/OIDC settings).
Configuration: Actors, Config Templates, Configs, Secrets, Document Types, Custom Units, Environments.
An Astro 5 SSR application with React 19 components and Tailwind 4. Provides a central landing page for every authenticated user in the platform, showing personalised application tiles based on their role-based permissions.
Next.js 16 full-stack application with React 19, MUI v7 (Data Grid Premium), and Emotion CSS-in-JS. Uses App Router with Server Actions for mutations. Covers 22+ business domains: customers, products, catalog (style/price/cost uploads), vendors, locations, purchase orders, transfer orders, stock adjustments, shipments, sales orders/receipts, employees, settings, reports (embedded Looker), and analytics. Authenticates via Cognito OAuth through All Point Access with per-user permission checks and franchise group filtering.
Data API (REST via @all-point/ap-fetch with Bearer token), Access (Cognito OAuth + JWT sessions), Jobs Sidecar (socket.io WebSocket for real-time upload progress), Config Service (runtime config + CDN branding), S3 (file uploads via next-s3-upload).
A monorepo full-stack application (Express backend + Next.js 14 frontend) for product catalog management. Handles bulk Excel imports with schema validation, product option/variant configuration (color, size, etc. with dependency rules), and catalog synchronization to the Connect API. Uses a webhook-driven workflow — registers TransmitConfirmation webhooks at startup, and tracks sync status through IMPORT_PROCESSING → READY_FOR_SALE → ERROR lifecycle.
All Point uses a connector model for external system integration. An inbound connector (source) ingests data from an external system into All Point entities. An outbound connector (destination) pushes All Point entity changes to the external system. The combination of an inbound + outbound connector for a given platform forms a productized integration. Each connector extends the state machine framework from connectors-common.
A two-part system: config-service (NestJS 11 backend on PostgreSQL 17 + Redis 7) provides versioned config management with rollout rules, JSON Schema validation, percentage-based targeting, and multi-tenancy. @all-point/config (client library) provides lightweight DI-friendly access to configs and AWS Secrets Manager from any service. Cache invalidation propagates via SNS/SQS. Every service in the platform consumes Config for runtime settings, feature flags, environment resolution, and document type definitions.
Version history with rollback, percentage-based rollouts with condition targeting (ANY_OF/NONE_OF), ETag-based conflict detection, Redis caching with 5-minute TTL, SNS/SQS distributed invalidation, JWT + API key authentication, Kubernetes health probes.
The public-facing gateway for external consumers. Authenticates every request by validating JWE tokens issued by All Point Access, then enforces scope-based authorization to gate access to specific resources and operations. Proxies authorized requests down to Layer 3 services (Data API, IMC, Jobs Sidecar) without exposing internal service topology. Responsible for rate limiting, API key management, versioned endpoint contracts, and request/response transformation between the public contract and internal service formats.
External consumer authenticates against All Point Access → receives scoped OAuth token → passes token on each Public API request → Public API validates JWE signature and checks granted scopes against the requested resource/operation → proxies to internal services.
Next.js 14 application serving as both SSO provider and User Management API. Implements Authorization Code Grant with AWS Cognito as identity provider. Features per-application session management (multiple apps with independent login states), hierarchical RBAC with base and client-specific roles, and per-environment franchise-based access control. URL-based application validation enforces multi-tenant isolation. Integrates with Shopify OAuth and Looker for analytics provisioning.
Controller → Service → Repository → DocumentDB, all wired through InversifyJS DI. Route handlers use a routeCallbackWrapper that provides DI container management, standardized error handling, and consistent HTTP status mapping.
Owns all real-time inventory state and movement operations. Tracks on-hand, committed, incoming, reserved, safety stock, damaged, and QC quantities per Item × Location. Emits domain events consumed by the Stock Ledger Service for immutable audit trail.
In monolith — currently housed in the Connectors Data API. Candidate for extraction as an independent microservice.
Master product catalog, pricing, and merchandising. Defines the product hierarchy (Product → Item), classification taxonomy (Department → Class → Subclass), configurable dimensions (Attributes, Options), all pricing/costing (price levels, cost levels, adjustments, promotions, discounts, coupons, audit ledgers), and merchandising (catalogs, product groups, menus for channel publication).
In monolith — currently housed in the Connectors Data API.
End-to-end order and fulfillment lifecycle. Captures sales intent (SalesOrder), orchestrates fulfillment (FulfillmentOrder → ShipOrder → Shipment), and records completed revenue transactions (Sale). Integrates with payment processing and triggers inventory decrements.
In monolith — currently housed in the Connectors Data API.
Complete procurement lifecycle from purchase order creation through goods receipt and three-way invoice matching. Manages vendor relationships, payment terms, and cost tracking. ASN (Advance Shipping Notice) support for pre-receipt visibility.
In monolith — currently housed in the Connectors Data API.
Append-only immutable ledger recording every stock movement across the platform. Consumes domain events (StockAdjusted, GoodsReceived, FulfillmentCompleted, TransferPosted) from Inventory, Purchasing, and Sales services. Provides point-in-time inventory reconstruction, valuation queries, and the audit trail required for fiscal compliance.
In monolith — currently housed in the Connectors Data API. Candidate for early extraction due to clear event-driven boundary and no synchronous dependencies.
Platform communication hub. Chat channels support direct, group, and broadcast messaging with franchise-group-scoped visibility. Email templates with merge-field interpolation trigger automated transactional emails via AWS SES. Calendar events support attendees, recurrence, and entity linking.
In monolith — currently housed in the Connectors Data API.
Platform organizational hierarchy and physical presence. Organizations contain Companies, which scope all commerce operations. Locations are the atomic unit for inventory and sales. Markets define fiscal rules (currency, tax, pricing). Franchise Groups drive data isolation and visibility filtering.
Split — Organization/Company entities in Data API; Application/User entities in All Point Access.
The largest service — a 4-tier Express application (Controller → Service → BusinessLogic → Repository). Uses CQRS-like separation with distinct CommandService and QueryService base classes. The inversify.config.ts registers 160+ DI bindings. Houses 35+ DocumentBusinessLogic classes for domain entities and 18+ lookup business logic classes. Features CLI tools for bulk imports (products, lookups) via ExcelJS/CSV.
Dual database: MySQL via Prisma (50+ relational models with compound indexes, soft-delete) for structured data, plus MongoDB via @all-point/document-db for document storage. S3 for file storage, SQS for async messaging.
A lightweight routing service that sits between AWS Lambda triggers and downstream connector services. Uses a CatchAllDocumentStateMachine (from connectors-common) that accepts any document type, validates it, then queries AWS Service Discovery to find all active connectors-* services. Documents are fanned out to each matching SQS queue in parallel. Supports partial failure — a message only fails if ALL target queues reject it. Service cache refreshes every 60 seconds. Runs on a 1-minute cron schedule.
A WebSocket-based microservice that offloads heavy processing from the Franchise Portal. Clients connect via socket.io, submit jobs, and receive real-time progress updates. A WorkerPoolService manages Node.js worker threads (CPU-aware auto-scaling). Uses a Factory pattern — 5 factory maps (config, mapper, parser, schemaValidator, validator) resolve the correct processing pipeline at runtime by upload type. The UploadProcessorServiceBase provides an 8-step pipeline: S3 download → parse → structure validate → schema validate → domain validate → duplicate check → batch DAPI send → result export.
CatalogStyle, Product, Employee, PurchaseOrder, TransferOrder, StockAdjustment uploads, plus media processing (sharp for images, ffmpeg for video) and PDF generation (Puppeteer).
Enterprise-grade bi-directional Shopify connector using a primary/worker cluster architecture. The primary process runs the HTTP server, state machines (OutboundProduct, OutboundCustomer, InboundProduct, InboundCustomer, etc.), supervisor jobs, and webhook processing. Worker processes poll SQS FIFO queues and execute sync tasks against Shopify's GraphQL Admin API. Supports multi-store per client — a single connector instance maps AllPoint locations to multiple Shopify store domains.
Adaptive rate limiting per store, entity circuit breaker (triggers bulk API fallback on variant quota exhaustion), three-way merge for webhook conflicts, echo prevention (2-minute window), GenericChangeDetector with nested collection tracking, allowlist-based webhook business rules, SNS/SQS FIFO task fanout, DynamoDB for state docs, PostgreSQL for sync coordination.
Outbound (AllPoint → Shopify): State machines consume DynamoDB docs, filter by status + sales channel, map to Shopify ProductSetInput/CustomerInput, publish tasks to SQS FIFO, workers execute via GraphQL. Inbound (Shopify → AllPoint): Webhooks received → change detection → business rules → REVERT/PROPAGATE/INGEST/IGNORE actions → queue to IMC for AllPoint ingestion.
A lightweight AWS Lambda that listens for SES email delivery status events via SNS/SQS. Filters by client/environment tags, transforms SES events to a standardized EmailStatusUpdateEvent, and POSTs them to the Data API via internal DNS resolution. Enables email delivery tracking within the platform.
The most complex connector in the platform — a bidirectional CHQ/Teamwork integration handling 20+ inbound and 14+ outbound document types. Uses an Initiation/Continuation async pattern for long-running CHQ API operations: sends an initiation request, then polls for completion with exponential backoff (max 20 checks). Augmentation services enrich CHQ data exports by querying the CHQ RDB directly via Prisma + SQL Server (MSSQL). MongoDB handles transient request/cache state. 100+ Inversify DI bindings. Includes full-sync scripts, custom table support, and franchise-based location isolation.
Inbound: Products, Customers, SalesOrders, PurchaseOrders, TransferOrders, Employees, Vendors, StockAdjustments, Shipments, Locations, and 10+ lookup/dictionary types. Outbound: CatalogStyles, Products, Prices, PurchaseOrders, TransferOrders, Employees, Vendors, Customers, StockAdjustments, and more.
An outbound-only financial document connector that pushes Bills, Vendor Credits, Journal Entries, Transfers, and Financial Summaries to QuickBooks Online. Uses a two-tier state machine pattern: the AllPoint state machine detects entity changes, then the QBO state machine handles the API submission lifecycle. OAuth2 token management with 55-minute cache, per-company semaphore for concurrent token refresh, and DynamoDB-backed token persistence. Rate limiting enforces 8 requests/sec and 4 concurrent operations. Client hooks extensibility via @ap-client-hooks/qbo allows per-client customization of mapping logic.
The backbone library for reliable document processing. Implements two state machine patterns: DocumentStateMachine (individual docs with 6-state flow, per-document metadata, orphan detection) and BatchDocumentStateMachine (batch processing with lease tracking, memory-safe generators). Includes a BackoffCalculator (exponential/linear/logarithmic), JobScheduler (cron-based), REST controllers for retry/health/job management, and instrumentation decorators. AWS ECS integration for instance tracking and Service Discovery registration.
Schema-first design: Zod schemas in validation/ define shape + constraints, then TypeScript types in all-point/ are derived via z.infer<>. Entity hierarchy: Identifiable → EntityDetail → BaseDocument → OperationalDocument → OperationalSubDocument. Covers the full retail domain: products/catalog, customers, sales orders/receipts, purchase orders/receipts, transfer orders, stock adjustments, shipments, vendors, employees, financial summaries, and 30+ dictionary/lookup types. PreCalcs types use Omit<> to strip calculated fields for pre-processing contexts.
Exports named configurations: eslintConfig (TypeScript-first with Google TS Style inspiration, JSON property sorting, no describe.only/it.only), prettierConfig (120 width, 4-space tabs, avoid arrow parens, Prisma plugin), vitestConfig (Istanbul coverage, JUnit XML for Bitbucket), and tsconfig.base.json (strict, ESNext, decorators, source maps). Also ships a package-version-validator CLI that blocks caret/tilde/star versions for @all-point/* and inversify* packages.
A multi-transport logging library that wraps both winston and pino with a unified API. Provides structured JSON output for production (CloudWatch-compatible) and human-readable console output for development. Environment-aware log level configuration (DEBUG in dev, INFO in prod). Used by every service in the platform for consistent, searchable log output across all layers.
An SQS abstraction layer providing typed producer/consumer patterns with message envelope handling. Supports both standard and FIFO queues, batch send/receive operations, dead-letter queue integration, and message visibility timeout management. The producer serializes typed message payloads into SQS-compatible envelopes; the consumer deserializes and dispatches to registered handlers. Used by IMC for fan-out, connectors for task distribution, and Data API for async event processing.
Shared foundation for all Express-based web applications. Provides base classes for InversifyJS DI container bootstrapping, DocumentDB (MongoDB) client wrapper with connection pooling, auth middleware for JWT token validation, standardized error handling middleware, health check endpoints (liveness + readiness), and request context propagation. Reduces boilerplate when standing up new services — a new microservice inherits auth, logging, DB access, and health probes out of the box.
The platform's DynamoDB abstraction layer, used by 9+ services. Provides four repository patterns: DocumentRepository (CRUD with auto-compression via compress-json for documents >400KB), BatchDocumentRepository (state document management with leasing, TTL, partitioned queries, and status filtering via StateDocumentStatus enum), CachedDocumentRepository (LRU in-memory cache decorator wrapping any DocumentRepository), and CapacityTracker (sliding-window rate limiting with EWMA smoothing for DynamoDB throughput management).
${AP_CLIENT}.${AP_ENVIRONMENT}.${AP_COMPONENT_NAME}.${collection} — ensures multi-tenant isolation at the DynamoDB table level. State document statuses: New, Processing, Success, Failed, Error, Deleted, Discarded.
A centralized OpenTelemetry wrapper that provides consistent distributed tracing across all platform services. Features @traceClass and @traceClassMethod decorators for automatic span creation, plus traceFunction() for standalone functions. Auto-instruments HTTP, Express, AWS SDK, Prisma, and Undici libraries. Uses AsyncLocalStorageContextManager for trace context propagation across async boundaries. Environment-aware: 10% sampling in production (BatchSpanProcessor) vs 100% in development (SimpleSpanProcessor). Also supports Next.js via a dedicated initNextjsOtelInstrumentation entry point.
The @all-point/types package is the platform's contract layer. Zod schemas define both runtime validation rules and compile-time TypeScript types from a single source. Every service depends on this package, ensuring the 74 domain entities are consistently validated and typed across the entire ecosystem. The entity hierarchy (Identifiable → EntityDetail → BaseDocument → OperationalDocument) cleanly separates master data from transactional documents.
All Point Access is the single authentication gateway. It implements OAuth 2.0 Authorization Code flow with AWS Cognito, issuing JWE-encrypted tokens. The RBAC model supports per-environment, per-client role hierarchies with franchise-based access control. Per-application session management allows users to maintain independent login states across multiple All Point applications simultaneously.
connectors-common provides the reliability backbone. Two state machine patterns — individual (6-state with per-document tracking) and batch (lease-based with memory-safe generators) — handle all document processing with configurable retry algorithms, orphaned document detection, and full transaction step recording. Every connector service inherits this framework, ensuring consistent error handling and observability.
The Connectors Data API separates read and write concerns through distinct CommandService and QueryService base classes. A factory-driven BusinessLogic layer handles domain validation, RDB↔domain mapping, and Prisma includes per entity. The dual-database strategy uses MySQL (Prisma, 50+ models) for relational product/inventory data and MongoDB for flexible document storage, with S3 for files and SQS for async messaging.
Jobs Sidecar solves the heavy-lift problem with a WebSocket (socket.io) server backed by Node.js worker threads. The Factory pattern (5 factory maps) resolves the entire processing pipeline at runtime by upload type. The UploadProcessorServiceBase provides an 8-step pipeline covering S3 download through batch DAPI submission, with real-time progress streamed back to the Franchise Portal UI.
Every service uses InversifyJS for dependency injection with Symbol-based bindings. This provides constructor injection, lifecycle management (singleton vs transient), and testability through interface-based contracts. The Data API alone registers 160+ bindings. Combined with the shared code-quality config (ESLint, Prettier, tsconfig, Vitest), this creates a remarkably consistent development experience across all repositories.
All Point Config is a two-part system: a NestJS 11 backend (PostgreSQL 17 + Redis 7) managing versioned configs with JSON Schema validation and percentage-based rollout targeting, plus a lightweight client library (@all-point/config) consumed via DI across all services. Cache invalidation propagates via SNS/SQS. Every service depends on Config for runtime settings, feature flags, environment resolution, and document type definitions.
All Point uses a connector model for external system integration. An inbound connector (source) ingests data from an external system into All Point entities. An outbound connector (destination) pushes entity changes to the external system. The combination of inbound + outbound for a given platform forms a productized integration. Each connector extends the state machine framework from connectors-common, ensuring consistent lifecycle management, retry logic, and observability across all integrations.
Eight foundational libraries underpin the platform: @all-point/types (74 Zod-validated entities), @all-point/connectors-common (state machine framework), @all-point/document-db (DynamoDB abstraction), @all-point/instrumentation (OpenTelemetry tracing), @all-point/queue (SQS producer/consumer), @all-point/logging (winston/pino multi-transport), @all-point/webapps-common (Express/Inversify/MongoDB base classes), and @all-point/code-quality (ESLint/Prettier/tsconfig/Vitest). This layered approach means a new microservice inherits auth, logging, DB access, queuing, tracing, validation, and health probes out of the box.
@all-point/document-db is a DynamoDB abstraction used by 9+ services. It provides four repository patterns: DocumentRepository (CRUD with auto-compression >400KB), BatchDocumentRepository (state document management with leasing, TTL, and partitioned queries), CachedDocumentRepository (LRU in-memory cache decorator), and CapacityTracker (sliding-window rate limiting with EWMA smoothing). Multi-tenant table naming (client.env.component.collection) ensures data isolation. Note: this is distinct from webapps-common's DocumentDB wrapper, which targets MongoDB.
@all-point/instrumentation provides consistent OpenTelemetry-based distributed tracing across all services. Decorator-driven (@traceClass, @traceClassMethod) for automatic span creation, with auto-instrumentations for HTTP, Express, AWS SDK, Prisma, and Undici. Environment-aware sampling (10% production, 100% development) balances observability cost with debugging capability. OTLP HTTP exporter ships traces to the observability backend. Sensitive field protection ensures PII is excluded from trace arguments.
The platform uses six distinct database technologies, each chosen for its strengths: MySQL (Data API — relational product/inventory via Prisma), PostgreSQL (Config Service, Shopify sync coordination), MSSQL/SQL Server (Teamwork CHQ RDB queries via Prisma), MongoDB (webapps-common — documents, auth state), DynamoDB (@all-point/document-db — connector state docs, product builder storage), and Redis (config caching via cache-manager). Prisma serves as the ORM for three of these (MySQL, PostgreSQL, MSSQL).
A change occurs in the source system (Shopify, Teamwork, NetSuite, etc.) — new order, product update, inventory adjustment. This triggers a webhook or scheduled poll via the inbound connector.
Lambda posts the document payload to the IMC's /documents endpoint. The CatchAllStateMachine validates structure, then queries AWS Service Discovery for all active connectors-* services.
IMC sends the validated document to each matching connector's SQS queue in parallel. Partial failure is tolerated — a document only fails if ALL queues reject it.
Each connector service (extending connectors-common) picks up the SQS message. The DocumentStateMachine transitions it through NEW → PROCESSING, runs IMapper transforms, validates via @all-point/types, then writes to the Data API.
The Data API's CommandService validates the document against its BusinessLogic layer, maps to the Prisma RDB model, and writes to MySQL. Any document-db records (state, audit) go to MongoDB. Files to S3.
A franchise operator uploads an Excel/CSV file (products, purchase orders, stock transfers, etc.) through the Franchise Portal UI. The file is stored in S3.
The Franchise Portal emits a jobCreated event via socket.io to the Jobs Sidecar. The JobService creates a PENDING job record in document-db.
WorkerPoolService dispatches to a worker thread. The appropriate UploadProcessor (resolved by factory) downloads from S3, parses Excel, validates structure → schema → domain, checks duplicates, then batch-sends to the Data API.
Worker threads emit WorkerProgressEvent at each stage. JobService relays these as JOB_UPDATED_EVENT via socket.io to the Portal, which updates its UI in real time. On completion, a JOB_FINISHED_EVENT is emitted with the final result summary.
A product, customer, or other entity is created or updated in the Data API — either by a user action in the Franchise Portal, an inbound connector sync, or a bulk upload via Jobs Sidecar.
The outbound connector's state machine (e.g., OutboundProduct, OutboundCustomer) detects the change via DynamoDB state documents. Filters by status, sales channel, and business rules to determine if sync is required.
The IMapper transforms the All Point entity into the destination system's format (e.g., Shopify ProductSetInput). Tasks are published to SQS FIFO queues with deduplication for reliable delivery.
Cluster worker processes consume SQS tasks and execute API calls to the destination system (e.g., Shopify GraphQL Admin API). Adaptive rate limiting per store, circuit breakers for quota exhaustion, and three-way merge for webhook conflict resolution ensure reliable delivery.