Internal — Ergomotion Engineering

IOR Automation — Build Plan

10 tickets to migrate from CSV-based processing to a fully automated SAP-integrated AWS pipeline.

Tech Lead — Nayan
Full-Stack Engineer
DevOps Engineer 16 Weeks • March 2026
Team Roles
⚖

Tech Lead

Architecture decisions, core business logic (processor, ETL, comparison engine), security, SAP team coordination, code review, cutover calls.

❮/❯

Full-Stack Engineer

Lambda functions (ingestion & ETL), backend API endpoints, React frontend (dashboards, auth UI), validation scripts, integration tests.

⚙

DevOps Engineer

CDK infrastructure (VPC, S3, DynamoDB, ECS, CloudFront), GitHub Actions CI/CD, Docker, CloudWatch monitoring, IAM, production runbook.

External Dependency: 6 of 8 SAP APIs must be built by the SAP/China team. Only ZFMIOR002 and ZFMIOR003 are live today. Tickets 1-2 and 6-8 proceed in parallel while waiting. Tickets 3-5 start with the 2 live APIs, expand when others ship.
10 Tickets
T-01 Infrastructure Foundation & CI/CD Wk 1-2 ▸
TaskOwnerRationale
Design VPC (public/private subnets, NAT, security groups)Tech LeadArchitecture decision
CDK project scaffolding (infra/, app.py, base stacks)DevOpsSets IaC patterns for the project
Secrets Manager entries (SAP API creds, JWT secret)Tech LeadSecurity-sensitive
GitHub Actions pipeline: Infrastructure (cdk diff → deploy)DevOpsCI/CD ownership
GitHub Actions pipeline: Backend (Docker → ECR → ECS)DevOpsCI/CD ownership
GitHub Actions pipeline: Frontend (build → S3 → CloudFront)DevOpsCI/CD ownership
GitHub Actions pipeline: Lambda/ETL (package → deploy)DevOpsCI/CD ownership
ECR repository CDK constructDevOpsContainer registry infra
Deliverable: cdk deploy creates VPC + networking. 4 CI/CD pipelines run on push. Secrets stored.
T-02 Storage Layer (S3 + DynamoDB) Wk 2-3 ▸
TaskOwnerRationale
S3 buckets CDK (6 buckets, lifecycle policies, SSE-KMS encryption)DevOpsInfrastructure
DynamoDB tables CDK (4 tables, on-demand billing, schemas)DevOpsInfrastructure
S3 Object Lock for audit bucket (COMPLIANCE mode, 7-year)Tech LeadCompliance-critical config
Cross-account S3 bucket policy (SD-WAN account → IOR account read)DevOpsPer meeting decision (Mar 26)
IAM roles: Lambda execution, ECS task role, S3 access policiesTech LeadSecurity-sensitive
Deliverable: 6 S3 buckets + 4 DynamoDB tables deployed. Cross-account policy for SD-WAN bucket. IAM roles ready.
T-03 SAP Data Ingestion Lambdas Wk 3-5 ▸
TaskOwnerRationale
Write ingest-product-weights Lambda template (ZFMIOR002 → S3)Tech LeadFirst Lambda = the pattern
SQS Dead Letter Queue + 3x retry → SNS alertDevOpsReliability infrastructure
EventBridge schedules (daily for ZEMM07/ZSDR002, weekly for rest)DevOpsScheduling infra
Replicate pattern for remaining 6 ingestion LambdasFull-StackFollows established template
Unit tests for all 7 Lambdas (mock SAP responses)Full-StackQuality & regression coverage
Deliverable: 7 Lambda functions deployed. #1-#2 tested against live SAP APIs. #3-#7 tested against mocks (blocked on SAP team).
T-04 ETL Processing (Raw → DynamoDB) Wk 5-7 ▸
TaskOwnerRationale
etl-product-master (merge ZFMIOR002+ZEMM005, derived weight fields)Tech LeadCore business formulas
etl-hts-reference (cross-reference, rate range validation)Tech LeadDomain knowledge required
etl-transactions (parse ZSDR002/ZEMM004 invoice data)Tech LeadComplex field mapping
Schema validation framework (shared across all ETL)Tech LeadQuality standard
etl-packaging (simple normalize — material + isPackaging)Full-StackSimplest ETL, follows pattern
SNS quality alert integration (wire to ETL errors)DevOpsAlerting infrastructure
Tests: validate ETL output matches current CSV dataFull-StackRegression testing
Deliverable: 4 ETL Lambdas deployed. Raw JSON → validated → DynamoDB. Quality alerts on schema violations.
T-05 Shadow Mode & Data Validation Wk 7-9 ▸
TaskOwnerRationale
Shadow mode flag (pipeline writes DynamoDB, app still reads CSVs)Tech LeadArchitecture toggle
Validation script: DynamoDB ProductMaster vs Product_List.csvFull-StackPandas comparison
Validation script: HTSReference vs HTS_Code.csvFull-StackSame pattern
Validation script: TariffSequencing vs HTS Tariff.csvFull-StackSame pattern
Validation script: PackagingMaterials vs Packaging_Material.csvFull-StackSame pattern
Fix data mapping mismatches found during validationTech LeadDomain knowledge
Document all field mapping adjustmentsFull-StackDocumentation
Deliverable: 2-week parallel run. Written report proving DynamoDB data matches CSV data to 100%. Sign-off before proceeding.
T-06 Backend Migration to ECS Fargate Wk 9-11 ▸
TaskOwnerRationale
Refactor data_loader.py: DynamoDB reads + TTL caching + fallbackTech LeadCore application change
Add GET /api/customs-lines endpointFull-StackNew endpoint, spec defined
Add POST /api/compare endpointFull-StackNew endpoint, spec defined
Dockerfile for FastAPI appDevOpsContainerization
CDK: ECS Fargate (1 vCPU/2GB), ALB, auto-scaling (1→2)DevOpsInfrastructure
CDK: WAF rules on ALBDevOpsSecurity infra
Test: processor.py identical output from DynamoDB vs CSVTech LeadCritical regression
Deliverable: FastAPI on ECS Fargate behind ALB. Same processor logic, DynamoDB source. Verified output parity.
T-07 Frontend Migration to CloudFront Wk 10-12 ▸
TaskOwnerRationale
CDK: S3 + CloudFront distribution + Origin Access ControlDevOpsHosting infrastructure
Build Customs Lines Dashboard page (view results by date)Full-StackNew React component
Build Comparison Report page (discrepancies, pass/fail)Full-StackNew React component
Wire dashboards to API endpointsFull-StackAPI integration
UX review and design directionTech LeadDesign decisions
Code review all new componentsTech LeadQuality gate
Deliverable: React frontend on CloudFront with dashboard pages. Full-Stack owns UI; Tech Lead reviews.
T-08 Auth System Fix & User Migration Wk 11-12 ▸
TaskOwnerRationale
Fix main.py: remove TESTING_MODE, fix DI chainTech LeadSecurity-critical
Migrate users.json → DynamoDB Users tableTech LeadAuth storage
Update user_storage.py to DynamoDBTech LeadAuth code path
Frontend: remove mock admin, uncomment auth checkFull-StackBounded frontend change
Test login/logout for admin & operator rolesFull-StackQA testing
Add credentials: 'include' to audit HTML fetchesFull-StackSmall targeted fix
Deliverable: Auth fully working. Both roles can log in, see appropriate content, access audit trail.
T-09 Auto-Generation & Comparison Engine Wk 13-15 ▸
TaskOwnerRationale
generate-customs-lines Lambda (reuses processor.py)Tech LeadCore business logic
EventBridge daily trigger (after ETL completion)DevOpsScheduling infra
comparison.py engine (field-by-field, tolerances)Tech LeadNew business logic
Comparison Lambda (chained after generation)Tech LeadOrchestration logic
SNS discrepancy alertsDevOpsAlerting infra
Integration tests: generate → compare → reportFull-StackEnd-to-end test scripts
Comparison report UI componentFull-StackFrontend display
Deliverable: Daily automated pipeline: ingest → generate customs lines → compare vs ZSDR002 → alert on mismatches.
T-10 Monitoring, Testing & Production Cutover Wk 15-16 ▸
TaskOwnerRationale
CloudWatch dashboards CDK (pipeline health, data freshness)DevOpsMonitoring infrastructure
CloudWatch alarms (ETL fail, SAP down, comparison fail rate)DevOpsAlerting infrastructure
SNS → email/Slack alert routingDevOpsNotification wiring
Parallel run: old system alongside new (1-2 weeks)Tech Lead Full-StackBoth monitor
Validate output parity: manual path = automated pathTech LeadBusiness sign-off
Production runbook (restart, rollback, health checks)DevOpsOps documentation
Cutover decision and executionTech LeadTech lead call
Verify manual upload fallback post-cutoverFull-StackRegression test
Deliverable: Production system live. Monitoring active. Runbook written. Manual fallback verified.
16-Week Timeline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
T1 Infra + CI/CD
TL
DO
T2 Storage Layer
DO
TL
T3 Ingestion Lambdas
TL
FS
FS
T4 ETL Processing
TL
TL
FS
T5 Shadow Validation
TL
FS
FS
T6 Backend → ECS
TL
DO
FS
T7 Frontend → CF
DO
FS
FS
T8 Auth Fix
TL
FS
T9 Auto-Gen + Compare
TL
TL
FS
T10 Monitor + Cutover
DO
TL
Workload Split
TicketTech LeadFull-StackDevOps
T1 Infra + CI/CD25%—75%
T2 Storage Layer20%—80%
T3 Ingestion Lambdas30%50%20%
T4 ETL Processing65%25%10%
T5 Shadow Validation20%80%—
T6 Backend → ECS45%20%35%
T7 Frontend → CF15%70%15%
T8 Auth Fix60%40%—
T9 Auto-Gen + Compare65%25%10%
T10 Monitor + Cutover30%15%55%
Tech Lead ~38% Full-Stack ~33% DevOps ~29%
Open Decisions (Resolve by Week 3)

ETL Technology

Recommended: Lambda

Data volume is tiny (~73 products, ~67 packaging, ~21 HTS rates). Lambda + pandas is 35x cheaper than Glue and the team already knows Python.

Authentication

Recommended: Custom JWT + DynamoDB

Lowest risk. Move users.json to DynamoDB now. Defer Cognito/Entra ID until M365 integration decision is final.

Risk Mitigations
RiskMitigation
SAP team delays on 6 APIsBuild against mocks (T3-T4). Swap endpoint URLs when APIs ship.
Cross-account S3 access complexityAgreed in Mar 26 meeting: bucket policy grants read to IOR account. Test in T2.
Auth migration breaks thingsT8 is isolated. Test in staging before merging.
Processor output differs between CSV and DynamoDBShadow mode (T5) catches this before any cutover.
GitHub Actions → GitLab migration mid-projectPer meeting: start with GH Actions now. GitLab migration is optional in ~1 month.
Scope creep from Document Distribution PipelineSeparate project. Do not bundle into these 10 tickets.
Ergomotion IOR Automation • Project Build Plan v1.0 • March 2026
Document Distribution Pipeline is a separate workstream — not included in this plan.