H2: Monitoring 300+ Remote Pipeline Stations

A major midstream operator needed to modernize monitoring across hundreds of remote pipeline stations while maintaining 24/7 operations. Legacy RTUs required expensive middleware, limited data collection, and provided no local intelligence.

[Architecture Overview]

H2: EdgeConnect-Driven Pipeline Architecture

Deployment Pattern

• EdgeConnect at each station (300+ sites) on CISCO routers

• Enterprise Unlimited at control center

• DataHub Stations at regional hubs (6 locations)

Key Design Decisions

• EdgeConnect deployed in Cisco IR1101 routers (reduced hardware footprint)

• Local SQL database for 30-day historian at each site

• Store-and-forward for satellite communication resilience

• UNS architecture with Sparkplug B for auto-discovery

[Technical Specifications]

H2: Architecture Components

[Implementation Details]

H2: Edge Intelligence Configuration

At Each Station (EdgeConnect)

• Direct PLC/RTU communication (Modbus, DNP3)

• Local calculations (flow totals, leak detection algorithms)

• Python scripts for anomaly detection

• MQTT Sparkplug B publication to regional/central

Regional Hubs (DataHub Station)

• Aggregation of area stations

• Regional historian (1 year retention)

• Alarm management for area

• Report generation

Control Center (Enterprise Unlimited)

• System-wide visualization

• Predictive analytics

• Regulatory reporting

• Integration with PHMSA compliance systems

[Data Flow]

H2: UNS Architecture

1. Field Level: Sensors → PLCs/RTUs → EdgeConnect

2. Edge UNS: Local namespace definition, context added

3. Regional UNS: Area aggregation via DataHub Station

4. Enterprise UNS: Complete operational view via Enterprise Unlimited

5. Cloud Integration: Select KPIs to Azure for corporate dashboards

[Results Achieved]

H2: Measured Outcomes

Operational Improvements

• 75-80% reduction in operational costs

• Zero middleware licensing fees

• 100% uptime during satellite outages (store-and-forward)

• 60% reduction in alarm flooding

Technical Achievements

• Sub-second local response times

• 30-day edge autonomy during communication loss

• Automatic configuration deployment to new sites

• Native integration with existing SCADA without replacement

[Scalability Path]

H2: Growth Architecture

Phase 1 (Completed): 300 pipeline stations
Phase 2 (In Progress): Add 150 compressor stations
Phase 3 (Planned): Integrate 50 storage facilities
Future: ML models for predictive maintenance

All phases use same architecture pattern with no redesign required.

[Bottom Section]

H2: Build This Architecture

Deployment Time for Phase 1: 8-12 weeks for 50 sites
Current Stage: 300+ sites
Required Products: EdgeConnect ($750/site), Enterprise Unlimited ($11,900), DataHub Station ($2,000 x 6)
Total Architecture Cost: ~$250,000 for complete 300-site system

1. The Problem

Challenge: Operate and observe hundreds of distributed midstream sites with PLCs and intermittent, bandwidth-limited links—while keeping on-site autonomy and ensuring a unified, secure, and scalable publish/subscribe model for corporate operations, analytics, and alarms.

Specific pain points:

• Multiprotocol collection (ControlLogix + DF1) into a single publish model without custom middleware.

• Reliable delivery over satellite with high latency/jitter and strict bandwidth budgets.

• Broker high-availability and end-to-end observability at scale (350 sites, 500k+ tags).

• Simple, repeatable deployment on router-class Linux hardware with unattended recovery.

Impact: Without a standardized, resilient gateway + MQTT pattern, sites face delayed event visibility, manual correlation, higher truck-rolls, and longer MTTR during incidents.

Example: “Prior to MQTT + EdgeConnect, engineers had to manually pull logs from PLCs after outages; post-event diagnostics stretched MTTR and risked SLA breaches.”

2. The Solution

2.1 Overview

• Edge tier (site): EdgeConnect on Linux (router) polls ControlLogix (CIP/EtherNet/IP) and DF1 devices, and publishes via MQTT/Sparkplug B.

• Backhaul: Satellite or constrained WAN with store-and-forward, rate limiting, compression, and payload batching.

• Broker tier: Redundant MQTT brokers (N=4).

• Clients/consumers: Third-Party brokers consumers.

2.2 Logical Diagram (high level)

[PLC Layer: ControlLogix / DF1]           [Edge Layer: Per-Site Gateway]                [Network Layer]
  • CIP/EtherNet/IP (CLX)                   • FrameworX (EdgeConnect on Linux)         • MQTT brokers (HA, N=4)
  • DF1 (serial)                            • Poll ? Buffer ? Publish                  • Subscribers: Third-Party brokers consumers
                                            • Watchdog, AutoStart                      

   ????TCP/IP ???????????????????????????????  Router  ???????????????????????????????????  Broker

2.3 Topology

2.4 Network Architecture

• Segmented per site VLANs; secure egress to broker endpoints only.

• Backpressure control: publish rate limiting, payload compression, delta/exception reporting.

2.5 Redundancy & Failover

• Brokers: 4-node HA cluster with client failover and session persistence.

• Edge: Store-and-forward with disk queues; automatic reconnect & replay; service watchdog and OS autostart.

• Links: Multi-endpoint broker list with exponential backoff and jitter to avoid thundering herd.

2.6 Protocols & Equipment

• Drivers/Interfaces: ControlLogix (CIP/EtherNet/IP), DF1 (serial), TCP/IP.

• Messaging: MQTT, MQTT Sparkplug B (birth/death, metrics, model).

2.7 Data Model & Topics

• Sparkplug namespaces per site/asset; Group ID, Node ID and Device ID.

2.8 Scale & Capacity

• Sites: ~350

• Points/Tags: 500,000+

• Ingestion/Publish rate: 10 seconds

• Concurrent sessions: 1

2.9 Observability & Health

• Edge and broker watchdogs.

• Heartbeats (LWT/BIRTH), latency and backlog metrics per site; topic-level delivery KPIs.

3. Key Enablers

• Multiprotocol edge collection (ControlLogix + DF1) with a unified MQTT/Sparkplug output.

• Router-grade Linux runtime enabling low footprint deployment close to the PLCs.

• Store-and-Forward + AutoStart + Watchdog for unattended resilience over satellite.

• Broker HA (N=4) and access governance to prevent network overflow and ensure continuity.

• Real-time broker/edge monitoring with alarms on heartbeat loss, queue growth, or reconnect storms.

Why it’s non-trivial elsewhere: The combination of CIP + DF1 ingestion, Sparkplug governance at scale, true edge resilience over high-latency links, and 4-node broker HA across 350 sites typically requires significant custom engineering; EdgeConnect standardizes it.

4. The Results

• Reduced MTTR by 70%: Eliminated manual log collection from PLCs post-outage through real-time MQTT telemetry, enabling immediate incident visibility and faster root cause analysis.

• Achieved 99.5% uptime across 350 sites: 4-node broker HA cluster with store-and-forward edge resilience maintained continuous operations despite satellite link interruptions and bandwidth constraints.

• Standardized deployment at scale: Single EdgeConnect solution replaced custom middleware across all sites, reducing deployment complexity and enabling consistent 500k+ tag collection from mixed ControlLogix/DF1 environments.

• Enhanced operational visibility: Real-time publish/subscribe model with Sparkplug B provided enterprise-wide monitoring and analytics capabilities, replacing reactive maintenance with proactive incident management.