Hot, Warm, and Cold: Designing Data Paths That Scale Without Breaking the Bank

The Three Paths

Hot Path – Act Within Seconds

The hot path exists for moments when delay costs money, damages equipment, or endangers safety. It is the most expensive path to operate and the most complex to build correctly, so you should be deliberate about what enters it.

A good rule of thumb: If no automated system or human operator will act on a data point within 60 seconds of receiving it, that data point does not belong in the hot path.

Concrete example: A temperature sensor in a food-processing cold chain reports 38°F. The critical threshold is 40°F. The hot path evaluates this reading in a Stream Analytics window, compares it to the threshold stored in a reference dataset, and generates an alert. If the temperature reaches 42°F, the hot path also triggers a command through IoT Hub to activate a backup refrigeration unit.

Here is the Stream Analytics query pattern:

-- Hot path: evaluate critical thresholds in real time
SELECT
  System.Timestamp AS ProcessingTime,
  deviceId,
  readings.temp AS temperature,
  r.threshold AS criticalThreshold,
  'CRITICAL' AS alertLevel
INTO
  AlertOutput
FROM
  TelemetryInput TIMESTAMP BY ts AS t
JOIN
  ThresholdReference AS r
ON
  t.deviceId = r.deviceId
WHERE
  t.readings.temp > r.threshold

The key insight: the hot path processes a filtered subset of the full telemetry stream. IoT Hub routing or a Stream Analytics WHERE clause filters out readings that are well within safe ranges, dramatically reducing the compute required. In the manufacturing example above, only 3% of vibration readings exceeded the alert threshold. By filtering at the ingestion layer, we reduced Stream Analytics SU requirements from 8 to 2 – an 75% cost reduction for the hot path alone.

Warm Path – Analyze Within Minutes

The warm path handles telemetry that needs near-real-time processing for trend analysis, anomaly detection, and digital twin state updates. This is the data that answers “is this equipment degrading?” rather than “is this equipment failing right now?”

Concrete example: A building management system tracks hourly energy consumption across 200 HVAC units. The warm path aggregates readings into 5-minute windows, enriches them with building metadata (floor, zone, occupancy schedule), and writes the results to Azure Data Explorer for trend analysis. A facility manager opens a dashboard at 2 PM and sees energy usage for the last 8 hours, broken down by zone.

Here is an Azure Function pattern for warm-path enrichment:

[FunctionName("EnrichTelemetry")]
public static async Task Run(
    [EventHubTrigger("telemetry-warm", Connection = "EventHubConnectionString")]
    IEnumerable<EventData> messages,
    [CosmosDB("devices", "telemetry", Connection = "CosmosDbConnectionString")]
    IAsyncCollector<Document> enrichedTelemetry,
    ILogger log)
{
    foreach (var message in messages)
    {
        var telemetry = JsonSerializer.Deserialize<TelemetryReading>(message.Body.ToString());
        var deviceMetadata = await _deviceCache.GetDeviceAsync(telemetry.DeviceId);

        var enriched = new EnrichedTelemetry
        {
            DeviceId = telemetry.DeviceId,
            BuildingId = deviceMetadata.BuildingId,
            Zone = deviceMetadata.Zone,
            Temperature = telemetry.Readings.Temp,
            OccupancyAdjusted = telemetry.Readings.Temp / deviceMetadata.OccupancyFactor,
            Timestamp = telemetry.Timestamp
        };

        await enrichedTelemetry.AddAsync(enriched);
    }
}

The warm path is where most IoT value is created. Raw sensor readings are transformed into contextualized, actionable insights. The enrichment step – joining telemetry with device metadata, operational state, and business context – is what turns a number into a decision.

Cold Path – Archive and Analyze in Batches

The cold path handles telemetry destined for long-term storage, regulatory compliance, batch analytics, and ML training. This data is never accessed in real time.

Concrete example: A pharmaceutical cold-chain solution must maintain temperature records for every shipment for five years to satisfy FDA 21 CFR Part 11. The cold path writes raw telemetry to Data Lake Storage in Parquet format, organized by date partition (/year=2025/month=03/day=15/). A nightly Databricks job validates data completeness, applies calibration corrections, and writes cleaned data to a gold-layer table. A compliance auditor queries the gold table through Synapse serverless SQL.

Here is the IoT Hub routing configuration that sends a copy of all telemetry to Blob Storage for cold-path archival:

{
  "endpoint": {
    "serviceBusQueues": [
      {
        "name": "hot-path-queue",
        "connectionString": "${ServiceBusHotConnectionString}"
      }
    ],
    "eventHubs": [
      {
        "name": "warm-path-events",
        "connectionString": "${EventHubWarmConnectionString}"
      }
    ],
    "storageContainers": [
      {
        "name": "telemetry-archive",
        "connectionString": "${StorageAccountConnectionString}",
        "batchFrequency": 120,
        "maxChunkSize": 5000000,
        "encoding": "avro"
      }
    ]
  },
  "routes": [
    {
      "name": "critical-to-hot",
      "source": "DeviceMessages",
      "condition": "eventType = 'telemetry' AND readings.temp > criticalThreshold",
      "endpointNames": ["hot-path-queue"],
      "isEnabled": true
    },
    {
      "name": "standard-to-warm",
      "source": "DeviceMessages",
      "condition": "eventType = 'telemetry'",
      "endpointNames": ["warm-path-events"],
      "isEnabled": true
    },
    {
      "name": "all-to-cold",
      "source": "DeviceMessages",
      "condition": "true",
      "endpointNames": ["telemetry-archive"],
      "isEnabled": true
    }
  ],
  "fallbackRoute": {
    "name": "$fallback",
    "endpointNames": ["telemetry-archive"]
  }
}

This configuration achieves three things simultaneously:

• Critical readings flow to the hot path (Service Bus queue → immediate processing).
• All telemetry flows to the warm path (Event Hubs → enrichment → ADX).
• Every message is archived to cold storage (Blob capture in Avro format) regardless of routing.

The batchFrequency of 120 seconds and maxChunkSize of 5MB balance write frequency against storage cost. The Avro encoding preserves schema information for downstream batch processing.