Workflows

Audience: Programmer

Overview

pyck combines Temporal workflow engine with our CRUD+ API to create complete, customized warehouse software solutions. This document explains the workflow architecture, integration points, and development patterns.

System Architecture

Key Components

1. pyck Core Services: Our CRUD+ microservices providing foundational warehouse functionality

2. Temporal Service: Durable execution service integrated within our core

3. Database: Persistent storage for both pyck services and Temporal

4. API Layer: GraphQL API from pyck services and gRPC API from Temporal

5. External Workflows: Custom workflow implementations that operate outside the core system

6. Frontend Applications: Mobile and web UIs that interact with the system

Workflow Architecture

                         ┌─────────────────────────────────────────────┐
                         │              pyck Core                      │
                         │                                             │
                         │  ┌─────────────┐    ┌─────────────────┐     │
                         │  │             │    │                 │     │
                         │  │ Database    │◄───┤ Core Services   │     │
                         │  │             │    │                 │     │
                         │  └─────────────┘    └────────┬────────┘     │
                         │                             ▲               │
                         │                             │               │
                         │  ┌─────────────┐            │               │
                         │  │             │            │               │
                         │  │ Temporal    │────────────┘               │
                         │  │             │                            │
                         │  └─────────────┘                            │
                         │                                             │
                         └─────────────┬──────────────┬────────────────┘
                                       │              │
                                       ▼              ▼
                         ┌─────────────────┐    ┌────────────────┐
                         │                 │    │                │
                         │  GraphQL API    │    │  gRPC API      │
                         │                 │    │  (Temporal)    │
                         └────────┬────────┘    └────────┬───────┘
                                  │                      │
                                  ▼                      ▼
           ┌─────────────────────────────────────────────────────────────┐
           │                     External Systems                         │
           │                                                             │
           │  ┌─────────────────┐                  ┌─────────────────┐   │
           │  │                 │                  │                 │   │
           │  │ Workflows       │────────────────►│ Frontend Apps   │   │
           │  │                 │                  │                 │   │
           │  └─────────────────┘                  └─────────────────┘   │
           │                                                             │
           └─────────────────────────────────────────────────────────────┘

Why Temporal?

Temporal provides critical capabilities for warehouse operations:

1. Crash-proof Execution: Workflows never lose state, even when systems fail

2. Fault Tolerance: Business logic continues despite network failures or API outages

3. Long-running Processes: Support for workflows that run for days, weeks, or months

4. Task Retries: Automatic retries with configurable policies for intermittent failures

5. Visibility: Complete history and real-time status of workflow executions

Per-Tenant Worker Deployment

pyck automatically provisions dedicated Temporal workers for each tenant during registration:

• Automatic Setup: When a tenant is registered via the registerTenant mutation, the system automatically creates a Kubernetes-based Temporal worker deployment for that tenant

• Isolation: Each tenant runs workflows in its own dedicated worker, ensuring workload isolation

• Kubernetes Native: Workers are deployed using the temporal-worker-controller as TemporalWorkerDeployment custom resources

• Namespace Mapping: Each tenant's worker connects to a Temporal namespace matching the tenant ID

• Configuration: Worker deployments include environment-specific settings for Temporal connection, pyck gateway URL, and tenant credentials

This architecture eliminates manual worker setup and ensures that each tenant has dedicated computational resources for their workflow executions from the moment of registration.

Workflow Development

Setup and Configuration

1. Project Structure: A typical warehouse workflow project includes directories for workflow definitions, activity implementations, data models, API clients, worker service entry point, and tests.

2. Dependencies: Workflow projects require dependencies for the Temporal SDK and pyck client libraries.

Workflow Registration

Workflows must be registered with both Temporal and the pyck API. This two-step registration process happens during application startup:

1. Temporal Registration: Workflows and activities are registered with the Temporal worker

2. pyck API Registration: Workflows are registered with the pyck GraphQL API to associate them with specific events

The registration process maps workflows to specific event types in the system. For example, a workflow named "TemporalWorkflowPicking_OrderCreate" would be triggered by the "OrderCreate" event.

Workflow ID Generation

Workflow IDs in pyck are generated deterministically to ensure consistency and prevent duplicate executions:

• With pyck_data_id: When pyck_data_id is present in the search attributes, it is used as the suffix for the workflow ID (e.g., WorkflowName_<pyck_data_id>)

• Without pyck_data_id: When no pyck_data_id is provided, a new UUID is generated as the suffix (e.g., WorkflowName_<uuid>)

This deterministic ID generation ensures that:

• Workflows tied to specific data entities can be uniquely identified

• Duplicate workflows for the same data entity are prevented

• Workflow state can be reliably tracked across the system

Workflow Search Attributes

pyck uses Temporal search attributes to enable filtering, querying, and grouping of workflow executions. These search attributes are automatically configured when pyck initializes:

Available Search Attributes

• pyck_data_id: Links the workflow to a specific data entity (e.g., order ID, shipment ID)

• pyck_workflow_assignee: Identifies the user or team assigned to the workflow

• pyck_workflow_name: The name of the workflow being executed

• pyck_tenant_id: The tenant context for multi-tenant deployments

• pyck_data_type: The type of data entity the workflow operates on

• pyck_service: The pyck service that owns the workflow

• pyck_group_by: Custom grouping criteria for categorizing and filtering workflows

• pyck_title: Human-readable per-workflow display label shown in the UI (set via workflowsdk.SetWorkflowTitle)

• pyck_group_title: Human-readable label for the workflow group header (set via workflowsdk.SetGroupTitle, pairs with pyck_group_by)

• pyck_sort_key: Numeric value used by the UI to impose a stable display order on workflow executions

Using Search Attributes

Search attributes can be used to query and filter workflows through the Temporal API:

// Example: Find all picking workflows for a specific order
const workflows = await temporal.workflow.list({
  query: 'pyck_workflow_name="OrderPicking" AND pyck_data_id="order-123"'
});

// Example: Find all workflows assigned to a user
const userWorkflows = await temporal.workflow.list({
  query: 'pyck_workflow_assignee="user-456"'
});

// Example: Group workflows by custom criteria
const groupedWorkflows = await temporal.workflow.list({
  query: 'pyck_group_by="priority-high" AND pyck_service="picking"'
});

// Example: Order workflows by sort key
const orderedWorkflows = await temporal.workflow.list({
  query: 'pyck_sort_key >= 10 AND pyck_sort_key <= 99'
});

These search attributes enable powerful filtering and reporting capabilities for monitoring and managing warehouse operations across your pyck deployment.

Setting Workflow Display Labels

The pyck SDK exposes helpers to set per-workflow display metadata as Temporal search attributes. The UI uses these labels for workflow lists and group headers, and clients can filter/sort directly on them through the GraphQL workflowExecutions query.

SetWorkflowTitle

Use workflowsdk.SetWorkflowTitle to write the pyck_title search attribute — the per-workflow display label shown in lists and dashboards.

import "github.com/pyck-ai/pyck/backend/workflowsdk"

func MyWorkflow(ctx temporalworkflow.Context, input MyInput) error {
    // Set the display label so the UI can show a human-readable title
    // and so callers can query workflowExecutions(where: {title: ...}).
    if err := workflowsdk.SetWorkflowTitle(ctx, "SKU-123"); err != nil {
        return err
    }

    // ... rest of the workflow
    return nil
}

• Passing an empty string clears the attribute so the frontend falls back to the workflow type name:

  workflowsdk.SetWorkflowTitle(ctx, "") // unset

• The attribute is a Temporal Keyword type; both equality and prefix/suffix predicates are available via the GraphQL title* filters.

• workflowsdk.GetWorkflowTitle(ctx) reads the current value.

SetGroupTitle

Use workflowsdk.SetGroupTitle to write the pyck_group_title search attribute — the human-readable label for the workflow group header. It pairs with pyck_group_by (the technical join key); set both together so the UI can group workflows under a shared label.

if err := workflowsdk.SetGroupBy(ctx, "order-42"); err != nil {
    return err
}
if err := workflowsdk.SetGroupTitle(ctx, "ORDER-42"); err != nil {
    return err
}

• Passing an empty string clears the attribute so the frontend falls back to "No Group".

• workflowsdk.GetGroupTitle(ctx) reads the current value.

SetSortKey

The pyck_sort_key search attribute can be set from within a workflow using the workflowsdk to control the display order of workflow executions in the UI:

import (
  "github.com/pyck-ai/pyck/backend/workflowsdk"
)

// Set a sort key to control display order
func MyWorkflow(ctx workflow.Context) error {
  sortKey := int64(42)
  if err := workflowsdk.SetSortKey(ctx, &sortKey); err != nil {
    return err
  }

  // ... rest of workflow logic

  // Clear the sort key (falls back to default start-time ordering)
  if err := workflowsdk.SetSortKey(ctx, nil); err != nil {
    return err
  }

  return nil
}

// Read the current sort key
func CheckSortKey(ctx workflow.Context) {
  sortKey := workflowsdk.GetSortKey(ctx)
  if sortKey != nil {
    // Sort key is set to *sortKey
  } else {
    // Sort key is not set, will use default ordering
  }
}

The sort key is particularly useful when you need workflows to appear in a specific order in the UI, rather than relying solely on start time. Pass nil to SetSortKey to unset the attribute and fall back to default start-time ordering.

Migration from SetWorkflowMemo

workflowsdk.SetWorkflowMemo and the associated WorkflowMemo struct (with Title, Subtitle, and Data fields) have been removed. Memo fields are opaque to Temporal Visibility and cannot be indexed; search attributes are fully queryable.

Before:

workflowsdk.SetWorkflowMemo(ctx, workflowsdk.WorkflowMemo{
    Title:    "SKU-123",
    Subtitle: "order-uuid",
    Data:     map[string]any{"priority": "high"},
})

After:

// The per-workflow title and the group label are now first-class search
// attributes. Custom auxiliary data should move into the workflow input,
// the workflow state, or pyck_group_by depending on its purpose.
workflowsdk.SetWorkflowTitle(ctx, "SKU-123")
workflowsdk.SetGroupBy(ctx, "order-uuid")
workflowsdk.SetGroupTitle(ctx, "ORDER UUID")

The signal router also accepts pyck_title and pyck_group_title keys when propagating attributes from incoming signal payloads, alongside the existing pyck_workflow_assignee and pyck_group_by keys.

Workflow Definition

Workflows in Temporal are defined as functions that orchestrate a series of activities. A typical picking workflow would include steps like:

• Fetching order details

• Allocating inventory

• Generating picking tasks

• Updating mobile app UI for picker

• Waiting for picking completion

• Processing results and updating inventory

Frontend Control Points

Workflows control frontend behavior through a combination of:

1. Activity Properties: Each workflow activity can define UI properties that should be displayed

2. Signal Channels: Workflows wait for user input through signal channels

3. NextActivityName: Defining the next activity tells the frontend what screen to show

Event Triggering and Signal Routing

Workflows are triggered by events from the system through an advanced signal routing mechanism that differentiates between START and INTERMEDIATE signals:

1. START Signals: Create new workflow instances when triggered

2. INTERMEDIATE Signals: Forward signals to already running workflow instances

When an event occurs in the pyck system:

• The system uses JetStream-based routing to determine which workflows should be triggered

• START signals will create new workflow instances (or fail if a workflow with the same ID is already running)

• INTERMEDIATE signals will be forwarded to matching running workflows

For example, when an order is created in the system, the "OrderCreate" event is fired, which then triggers the "TemporalWorkflowPicking_OrderCreate" workflow.

Important Breaking Change: As of the latest version, running workflows cannot be reassigned on re-trigger. Attempting to start a workflow that is already running will result in an error. This ensures workflow state consistency and prevents accidental duplicate executions.

User Input Communication Flow

The communication between Temporal workflows and the frontend when waiting for user input follows a specific stateful pattern:

1. Workflow State Management

Workflows maintain their state through a query mechanism that allows external systems to inspect their current state without affecting execution:

1. Query Handler: Each workflow exposes its current UI state through a query handler for "nextActivityProperties"

2. Activity Definition: The workflow defines what activity the user should perform next

3. Signal Channels: The workflow creates channels to receive signals from the frontend

4. Waiting for Input: The workflow pauses execution while waiting for user input

2. Frontend Interaction Cycle

The frontend interacts with workflows through a well-defined cycle:

1. Discover Workflows: Frontend finds workflows assigned to the current user via the checkAssignedWorkflows GraphQL mutation

2. Get Current Activity: The system retrieves the current activity information from the workflow

3. Render UI Components: Frontend displays the appropriate UI based on the activity properties

4. Collect User Input: User interacts with the UI and provides the requested information

5. Submit Input and Get Next Step: When the user submits input, the frontend calls a single GraphQL mutation (userDataInput) that:

   • Sends the user input to the workflow

   • Waits for the workflow to process the input

   • Returns the next activity in the same response

This means the frontend doesn't need to separately poll for the next activity after submitting user input - it gets both acknowledgment of the submitted data and instructions for the next step in a single GraphQL mutation.

3. GraphQL Mutation Synchronization

When the userDataInput GraphQL mutation is called:

1. Signal Sending: The GraphQL resolver sends the user input to the workflow via a Temporal signal

2. Synchronous Waiting: The resolver then waits for the workflow to process the input

3. State Transition Detection: The resolver monitors the workflow state, waiting for it to change from the current activity to the next activity

4. Response with Next Step: Once the workflow updates its state, the GraphQL mutation returns with information about the next activity the frontend should display

4. Workflow Processing

Inside the workflow, when it receives user input:

1. Signal Reception: The workflow's signal channel receives the user input

2. State Update: The workflow updates its activity state to indicate the current activity is passed

3. Input Processing: The workflow processes the input, often calling an activity

4. Next Activity Definition: The workflow defines the next activity for the user

5. State Exposure: The updated state becomes available to the GraphQL resolver

This continuous cycle allows the workflow to drive the frontend state while maintaining the reliability guarantees provided by Temporal's durable execution model.

BPMN to Code Translation

We use BPMN for visual workflow design and translate it to Temporal code:

1. BPMN Process: Designed with logistics team for business requirements

2. Activities: BPMN tasks become Temporal activities

3. Gateways: BPMN gateways become workflow conditions

4. Events: BPMN events map to Temporal signals/triggers

The workflow directory structure typically follows this pattern:

processes/
  {priority}_{domain}/
    {id}_{event}/
      activities/      # Activity implementation
      domain/          # Data models
      workflows/       # Workflow definition
      {event}.bpmn     # Visual workflow design

Reference Implementation

For a practical example, see our reference implementation: https://github.com/pyck-ai/pyck-customer-template

Best Practices

1. Idempotency: Ensure all activities are idempotent for reliable retries

2. Deterministic Workflows: Keep workflows deterministic by moving non-deterministic code to activities

3. Error Handling: Use appropriate retry policies for different failure types

4. Testing: Write comprehensive tests for workflows and activities

5. Monitoring: Set up proper monitoring and alerting using Temporal metrics

6. Versioning: Use versioning for workflow changes to handle in-flight workflows