Architecture and workflows

The diagrams provide one system-wide view and two complementary views of the planning scope:

View	Purpose
Confi-GRIP target system architecture	High-level interaction of the components from AP 2 to AP 7
Grasp-process planning and validation	H&W-relevant planning scope, shown as a structural view and an execution flow

Confi-GRIP target system architecture

This high-level target architecture shows how the Confi-GRIP components from AP 2 to AP 7 interact. It focuses on planning orchestration, simulation, training, data management, presentation, and the real demonstrator. It is not a complete deployment or responsibility model. An arrow indicates communication or data flow, not necessarily a direct Python method call.

        %% Confi-GRIP component architecture.
%%{init: {'theme':'base','themeCSS':'svg { background-color: #ffffff; }','themeVariables':{
  'background':'#ffffff',
  'fontFamily':'ui-sans-serif, system-ui, sans-serif',
  'fontSize':'14px',
  'textColor':'#1f2933',
  'lineColor':'#616e7c',
  'clusterBkg':'#fbfbfa',
  'clusterBorder':'#c1c7cd',
  'edgeLabelBackground':'#ffffff'
}}}%%
flowchart TB

  subgraph AP6["AP 6 · Presentation Layer"]
    UI["UI / web frontend"]
  end

  subgraph AP2["AP 2 · Interfaces and methods for gripper and simulation control"]
    SEQ["Planning Sequence API"]
    JOBS[("Job Store<br/>Job type · status · current step<br/>Input and result references")]
  end

  subgraph COMPONENTS["Components invoked by AP 2"]
    subgraph PLANNING["Planning and checking · replaceable backends"]
      GRASP["Gripper and grasp planning<br/>GraspGenX / custom AI model"]
      COLLISION["Grasp-candidate<br/>collision checking · optional per workflow"]
      GRASP_VIS["Grasp-pose<br/>visualization"]
      PATH["Motion planning<br/>cuRoboV2 / custom AI model"]
    end

    subgraph EVALUATION["Simulation and evaluation"]
      SIM["AP 3 · High-fidelity simulation<br/>Training-data generation · candidate validation<br/>Unity / ManiSkill · Newton / MuJoCo · IWU"]
      VALIDATION["Validation<br/>produces validation report"]
      VIDEO["Video rendering"]
    end
  end

  subgraph AP54["AP 4 · Training and assets / AP 5 · Result delivery"]
    MDB[("MongoDB<br/>Raw data · feature store")]
    TRAINING["Offline training H&W<br/>Batch"]
    ASSETS[("Asset Store<br/>Models/checkpoints · grippers · scenes<br/>Grasp objects · robots<br/>Components load directly via asset_id / URI<br/>Retrieval edges omitted")]
    TELEMETRY["Telemetry ingest / fan-out<br/>Transport TBD"]
    TB[("ThingsBoard<br/>Metrics · dashboards")]
  end

  subgraph AP7["AP 7 · Real World"]
    LSA["LSA controller<br/>Physical demonstrator"]
  end

  %% synchronous / REST
  UI   <-->|S-01| SEQ
  SEQ  <-->|Job state| JOBS
  SEQ  <-->|"S-03-G / S-06-G"| GRASP
  SEQ  <-->|Collision check| COLLISION
  SEQ  <-->|Visualization| GRASP_VIS
  SEQ  <-->|"S-03-B / S-06-B"| PATH
  SEQ  <-->|"S-02 · Simulation job"| SIM
  SEQ  <-->|Validation| VALIDATION
  SEQ  <-->|Rendering| VIDEO
  SEQ  <-->|S-07| LSA
  SEQ  <-->|"Upload / catalog"| ASSETS

  %% Simulation results are evaluated downstream.
  %% Control flow and selection remain with SEQ; direct edges carry data.
  SIM  -.->|"Data reference · SimulationResult"| VALIDATION
  SIM  -.->|"Simulation artifacts · AssetRef"| VIDEO

  %% asynchronous / messaging
  SEQ  -.->|"S-04 · Results"| TB
  LSA  -.->|"Publish telemetry once"| TELEMETRY
  TELEMETRY -.->|Metrics| TB
  TELEMETRY ==>|Raw data| MDB
  TB   -.->|Dashboard| UI

  %% data and offline training
  SIM   ==>|"S-05a · Training data"| MDB
  MDB   ==>|Training data| TRAINING
  TRAINING ==>|"Models / checkpoints"| ASSETS

  classDef pres fill:#e7f6ec,stroke:#3f9c5d,stroke-width:1.5px,color:#14301f
  classDef orch fill:#fdf3d7,stroke:#c8941a,stroke-width:1.5px,color:#3d2c05
  classDef exec fill:#eeeff1,stroke:#78838f,stroke-width:1.5px,color:#1f2933
  classDef data fill:#e2eefb,stroke:#2b7bc4,stroke-width:1.5px,color:#0f2c47
  classDef real fill:#fdeaea,stroke:#cf4444,stroke-width:1.5px,color:#4a1414

  class UI pres
  class SEQ,JOBS orch
  class SIM,GRASP,COLLISION,GRASP_VIS,PATH,VALIDATION,VIDEO,TRAINING exec
  class MDB,ASSETS,TELEMETRY,TB data
  class LSA real

    

Confi-GRIP target system architecture

Grasp-process planning and validation

These diagrams show the H&W-relevant planning chain: generate gripper-pose candidates, check them, plan the robot motion, and validate selected strategies in simulation.

Physical validation flow

The flow checks each candidate for collisions, plans its motion, simulates it, records failures, and ranks the results.

        %% Compact target flow for the physical validation of grasp strategies.
%% Implemented interfaces: GraspPlanner, GraspCollisionChecker, MotionPlanner.
%% Simulation, validation, video rendering, and orchestration are planned.
%% Each result remains correlated through candidate_id.
%%{init: {'theme':'base','themeCSS':'svg { background-color: #ffffff; }','themeVariables':{
  'background':'#ffffff',
  'fontFamily':'ui-sans-serif, system-ui, sans-serif',
  'fontSize':'14px',
  'textColor':'#1f2933',
  'lineColor':'#616e7c',
  'edgeLabelBackground':'#ffffff'
}}}%%
flowchart TB

  API(["Planning Sequence API<br/>runs the workflow as a job"])
  WORKFLOW(["GraspPlanningWorkflow<br/>orchestrates the domain calls<br/>arrows show data flow"])
  GRASP["GraspPlanner<br/>plan_grasps(request)<br/>run GraspGenX for each gripper"]
  GRASP_SELECTION["CandidateEvaluator · planned<br/>evaluate backend scores<br/>and preselect candidates"]
  COLLISION["GraspCollisionChecker<br/>check_grasp_collisions(request)<br/>required for physical validation"]
  COLLISION_SELECTION["CandidateEvaluator · planned<br/>exclude collisions<br/>and select remaining candidates"]
  MOTION["MotionPlanner<br/>plan_motion(request)"]
  MOTION_FOUND{"GraspPlanningWorkflow decides:<br/>Feasible motion found?"}
  SIMULATION["PhysicsSimulator<br/>simulate(request)<br/>configured perturbed rollouts"]
  MOTION_REJECTION["Exclusion reason:<br/>no feasible motion"]
  FINAL_EVALUATION["CandidateEvaluator · planned<br/>evaluate simulation results and exclusion reasons<br/>produce final cross-gripper ranking"]
  VIDEO["VideoRenderer<br/>render(request)<br/>selected results"]
  OUTPUT(["Physically validated ranking<br/>of gripper–grasp-pose combinations"])

  API -->|"Job worker starts"| WORKFLOW
  WORKFLOW -->|"GraspPlanningRequest<br/>object + 1..* concrete gripper designs"| GRASP
  GRASP -->|"GraspPlanningResult<br/>gripper–grasp-pose combinations + reproducibility"| GRASP_SELECTION
  GRASP_SELECTION -->|"Preselection"| COLLISION
  COLLISION -->|"Tuple[GraspCollisionResult]"| COLLISION_SELECTION
  COLLISION_SELECTION -->|"selected collision-free candidates<br/>then per candidate"| MOTION

  MOTION -->|"MotionPlan or None"| MOTION_FOUND
  MOTION_FOUND -->|No| MOTION_REJECTION
  MOTION_FOUND -->|Yes · MotionPlan| SIMULATION

  COLLISION_SELECTION -.->|"Exclusion reasons"| FINAL_EVALUATION
  SIMULATION -->|"SimulationResult<br/>Lift · hold · transport · slip"| FINAL_EVALUATION
  MOTION_REJECTION --> FINAL_EVALUATION

  FINAL_EVALUATION -->|Ranking| OUTPUT
  FINAL_EVALUATION -.->|"selected candidate_ids"| VIDEO
  SIMULATION -.->|"Simulation artifacts · AssetRef<br/>via GraspPlanningWorkflow"| VIDEO
  VIDEO -.->|"Tuple[AssetRef]"| OUTPUT

  classDef result fill:#e7f6ec,stroke:#3f9c5d,stroke-width:1.5px,color:#14301f
  classDef orch fill:#fdf3d7,stroke:#c8941a,stroke-width:1.5px,color:#3d2c05
  classDef process fill:#eeeff1,stroke:#78838f,stroke-width:1.5px,color:#1f2933
  classDef decision fill:#f3e8ff,stroke:#8b5cf6,stroke-width:1.5px,color:#32145f
  classDef failure fill:#fdeaea,stroke:#cf4444,stroke-width:1.5px,color:#4a1414

  class API,WORKFLOW orch
  class OUTPUT result
  class GRASP,GRASP_SELECTION,COLLISION,COLLISION_SELECTION,MOTION,SIMULATION,FINAL_EVALUATION,VIDEO process
  class MOTION_FOUND decision
  class MOTION_REJECTION failure

    

Physical validation flow for grasp strategies

Interfaces and adapters

The class diagram shows workflow interfaces and replaceable backend adapters. Solid arrows are dependencies; dashed arrows show protocol implementations.

        %% Static interface view of the grasp-planning sequence.
%% Implemented in configrip-contracts: GraspPlanner, GraspCollisionChecker,
%% and MotionPlanner. The remaining interfaces and the orchestration are planned.
%% GraspPlanningWorkflow runs inside a worker of the Planning Sequence API.
%% The synchronous methods are logical worker interfaces; transport-specific job
%% handling remains in the API or adapter. The workflow coordinates the interfaces;
%% it implements no planning, simulation, validation, or rendering algorithm.
%%{init: {'theme':'base','themeCSS':'svg { background-color: #ffffff; }','themeVariables':{
  'background':'#ffffff',
  'fontFamily':'ui-sans-serif, system-ui, sans-serif',
  'fontSize':'14px',
  'primaryTextColor':'#1f2933',
  'lineColor':'#616e7c'
}}}%%
classDiagram
direction LR

class GraspPlanningWorkflow {
  <<planned orchestration>>
}

class GraspPlanner {
  <<Protocol · implemented>>
  +plan_grasps(request: GraspPlanningRequest) GraspPlanningResult
}

class GraspCollisionChecker {
  <<Protocol · implemented>>
  +check_grasp_collisions(request: GraspCollisionCheckRequest) Tuple~GraspCollisionResult~
}

class MotionPlanner {
  <<Protocol · implemented>>
  +plan_motion(request: MotionPlanningRequest) Optional~MotionPlan~
}

class PhysicsSimulator {
  <<Protocol · planned>>
  +simulate(request: SimulationRequest) SimulationResult
}

class CandidateEvaluator {
  <<planned component>>
}

class GraspPoseVisualizer {
  <<Protocol · planned>>
  +render(request: GraspVisualizationRequest) Tuple~AssetRef~
}

class VideoRenderer {
  <<Protocol · planned>>
  +render(request: VideoRenderRequest) Tuple~AssetRef~
}

class GraspGenXAdapter
class CollisionBackendAdapter
class CuRoboAdapter
class UnityManiSkillAdapter
class NewtonMuJoCoAdapter
class HeadlessGraspRenderer
class SimulationVideoRenderer

GraspPlanningWorkflow --> GraspPlanner : Generate gripper–grasp-pose candidates
GraspPlanningWorkflow --> GraspCollisionChecker : Check candidates
GraspPlanningWorkflow --> MotionPlanner : Plan motion
GraspPlanningWorkflow --> PhysicsSimulator : Simulate motion
GraspPlanningWorkflow --> CandidateEvaluator : Evaluate and select candidates in stages
GraspPlanningWorkflow --> GraspPoseVisualizer : Render grasp poses
GraspPlanningWorkflow --> VideoRenderer : Render simulation evidence

GraspGenXAdapter ..|> GraspPlanner
CollisionBackendAdapter ..|> GraspCollisionChecker
CuRoboAdapter ..|> MotionPlanner
UnityManiSkillAdapter ..|> PhysicsSimulator
NewtonMuJoCoAdapter ..|> PhysicsSimulator
HeadlessGraspRenderer ..|> GraspPoseVisualizer
SimulationVideoRenderer ..|> VideoRenderer

    

Interfaces and adapters for grasp-process planning

Workflow steps, interface status, and data types are mapped in Using planning contracts.