Page History

A minimal operator-chat panel: six live plant tags, a transcript widget that renders selectable per-role bubbles, a TextBox, a Button, and the ChatRequest action — no scripting required. The smallest end-to-end example of FrameworX Local AI on a Display.

How-to Guides → Solution Examples → Local AI → Local AI Chat Example

Version 10.1.5+

Prerequisites

• FrameworX 10.1.5 or later.

• Local Ollama installed and reachable on http://127.0.0.1:11434. Run AISetup\Install-LocalAI.ps1 from the FrameworX install root if you have not already — see the Local AI - First Install Walkthrough.

• The demo model pulled into Ollama: ollama pull qwen2.5:3b-instruct (~2 GB). This is the model the example targets. To use a higher-quality production-tier model instead, see Moving to a production tier below.

What it contains

Open the solution in Designer and you will see exactly eight UNS tags (six under Plant/, two under Chat/), one Display (ChatTest), one SolutionSettings configuration. No Devices, no Alarms, no Historian, no scripts.

Plant tags — the live context the operator and the model both see

Chat tags — the operator-to-AI wiring

That is every UNS tag the example needs. The plant tags carry StartValue so the runtime has live numbers as soon as you click Run; in a real solution they would be driven by a TagProvider, a Device, or a script.

The ChatTest display

One Canvas display at 1200 × 800. All elements theme-bound (no hard-coded colors). The transcript widget is the new TChatSession control — native to FrameworX 10.1.5, no Symbol dependency, no DataTable wiring on the adopter side.

Header strip

TextBlock title "Local LLM AI - Integration Test" + one-line subtitle explaining the panel.

Live tags row

Section heading ("LIVE TAGS IN SCOPE") plus six TextBlocks that bind to the six plant tags via inline expansion in the form {@Tag.Plant/TankLevel} gal. These render in real time as the tags change — the operator and the LLM both see the same snapshot.

Quick-prompt buttons

Three Buttons, each with an ActionDynamic that runs ActionType=SetValue on @Tag.Chat/Query with a pre-built question. Clicking a button stages the question; the operator can then edit it in the TextBox before sending.

Ships with: What is the current tank level? · Summarize the current state of Reactor R-101. · Is the temperature safe given the current pressure?

TChatSession transcript widget

The native FrameworX transcript widget. Renders the per-Display-panel conversation history as selectable per-role bubbles in a single left-aligned column — user prompts right-aligned role bubbles tinted with the platform accent tintcolor, assistant replies left-aligned role bubbles with the neutral shade tint. While a request is in flight, a rotating ? glyph with elapsed-seconds counter appears in the transcript; it is replaced by the assistant bubble when the reply lands.

Bubble bodies render as TTextBox rows (with chrome suppressed by the control) so operators can select reply text, Ctrl-C to copy, or right-click Copy — the chat-UX standard. The control reads the transcript directly from the runtime's per-connection cache — no DataTable binding to author, no script glue.

color. Color alone discriminates role; the layout follows the conversational-AI cluster convention (Claude Code, ChatGPT, Cursor, Copilot, Linear AI) rather than the peer-to-peer messenger two-column shape.

While a request is in flight, the operator's question appears immediately as a user-echo bubble, followed by a rotating ? glyph with an elapsed-seconds counter. The pending visuals are replaced by the assistant-reply bubble when the reply lands.

Bubble bodies render as TTextBlock elements (the WPF TextBlock-derived control native to FrameworX) with chrome suppressed by the control. Operators select bubble text by mouse drag, then Ctrl-C to copy, or right-click Copy — the chat-UX standard. Selection is enabled on both user and assistant bubbles. The control reads the transcript directly from the runtime's per-connection cache — no DataTable binding to author, no script glue.

Width 1152, height ~350. Width auto-scales with the Display via OnResize="StretchUniform".

Question TextBox + Ask AI button

Section label, single-line TextBox bound to @Tag.Chat/Query, plus the load-bearing Button — ActionType="ChatRequest" with ObjectValueLink=@Tag.Chat/Query and ObjectReturnValueLink=@Tag.Chat/Reply. One click sends the current Query to the LLM, receives the reply envelope on Chat/Reply, and the TChatSession control auto-appends both turns to its transcript.

Reply + latency footer

Two TextBlocks below the input row — the first prints the raw reply envelope (Local AI Reply: {@Tag.Chat/Reply}) so you can see the full JSON shape; the second extracts {@Tag.Chat/Reply.JsonString("latencyMs")} for the round-trip latency in milliseconds.

How to run it

1. Open Local AI Chat Example.dbsln in Designer.

2. Confirm SolutionCapabilities[LocalAI].Enabled = true on Solution → Capabilities → AI Engine tile. The status dot should be green ("Reachable").

3. Confirm qwen2.5:3b-instruct is pulled in your local Ollama (ollama list). For higher-quality production-tier alternatives, see Moving to a production tier below.

4. Click Run. The runtime starts and the RichClient (or your configured client) opens to the ChatTest panel.

5. Click one of the three quick-prompt buttons to stage a question, or type your own in the TextBox.

6. Click → Ask AI. The transcript shows your prompt bubble immediately and a rotating ? pending indicator; within a few seconds the assistant bubble replaces the indicator with the model's reply. The Latency field shows the round-trip in milliseconds.

The TChatSession control retains the conversation per Display panel — you can ask follow-ups and the model has context. The transcript persists across re-opens of the same panel within one client session.

What to try next

• Copy a reply. Select text in an assistant bubble and press Ctrl-C, or right-click and pick Copy. The TChatSession's TTextBox TTextBlock bubbles support standard text-selection gestures on both user and assistant bubbles — useful for pasting LLM-generated content into reports, tickets, or follow-up scripts.

• Change a plant tag value mid-conversation. Use Designer's Watch panel to drive Plant/TankLevel from 784.5 to 950, then ask the AI "What is the current tank level and how close are we to the cap?". The model sees the new value because the live tag expansion in the display passes the current snapshot on every call.

• Add a fresh quick-prompt button. Duplicate one of the three buttons and change its ObjectValueLink to a new question string. Click it during Run and the new question stages in the TextBox.

• Inspect the full envelope. The footer shows the whole reply JSON plus latency. Add a TextBlock bound to {@Tag.Chat/Reply.JsonString("warnings")} if you want to surface warnings; bind {@Tag.Chat/Reply.JsonString("toolTrace")} if you enable the UNS / Alarm / Historian tool bits on SolutionSettings.ModelOptions (see Local AI Configuration) and want to inspect autonomous tool calls.

• Move the LLM off the TServer host. If you have access to a second machine (or a sibling VM), install Ollama there, pull the model on that host, and change the URL field in SolutionCapabilities[LocalAI].Settings from 127.0.0.1 to the remote IP or hostname. The example continues to work unchanged — TServer no longer fights the LLM for CPU. Full guidance on when this matters: Local AI Deployment and Performance.

• Try a cloud LLM endpoint. Point URL at any OpenAI-compatible chat-completions endpoint (cloud or otherwise); store the API key in SecuritySecrets and reference it via /secret:<Name> in the Authorization field. See SecuritySecrets Authentication for Local AI.

System prompt

The runtime applies a short default system prompt to every chat call so replies stay terse and operator-friendly:

You are a helpful assistant embedded in the FrameworX runtime. Answer concisely and directly — no preamble, no status wrapper, no echo of the user's question. On tool failure, reply with only the word: error.

To override it for a specific call, send a JSON query payload with a top-level system field instead of the bare prompt string. The override applies for that call only; the next bare-string call falls back to the default. Server-side AI.Execute calls follow the same contract.

Role labels — tolerant matching

The TChatSession control distinguishes user-role from assistant-role bubbles via the UserRoleValue and AIRoleValue properties. The example uses the defaults: user / assistant — the literal role strings the runtime writes into the transcript. Leave these alone unless you have a reason to change them.

The control matches role labels case-insensitively and also recognises common aliases on each side, so persona overrides applied elsewhere on the Display still tint correctly:

• User aliases: user, operator, human, question, you, me, customer, client, person, prompt.

• Assistant aliases: assistant, ai, bot, agent, response, answer, plant ai, system ai, system, model, copilot, reply.

For persona labels visible on the Display ("Operator", "Plant AI", etc.), add a separate TextBlock above the transcript — do not push the persona text into UserRoleValue / AIRoleValue. The control's contract is to match what the runtime writes; the persona is a Display-level affordance.

Moving to a production tier — 7B, larger models, remote endpoints

The example targets qwen2.5:3b-instruct so it runs end-to-end on a workshop laptop with no GPU. For production deployments — where reply quality, autonomous tool-call reliability, and concurrent operator load matter — switch to a larger model (recommended: qwen2.5:7b-instruct), or move the LLM off the TServer host entirely:

1. Pull the production model on the LLM host: ollama pull qwen2.5:7b-instruct (~4.7 GB).

2. In Designer, open the solution and go to Solution → Capabilities.

3. On the AI Engine tile, click Edit Configuration.

4. Paste the JSON below into the dialog and save (adjust URL if the LLM runs on a different host):

{
  "URL": "http://127.0.0.1:11434/v1/chat/completions",
  "Name": "qwen2.5:7b-instruct",
  "Authorization": "NoAuth",
  "Headers": "",
  "Info": "Production tier — 7B-instruct (~4.7 GB). Best reply quality and tool-call reliability.",
  "TimeoutSeconds": 60
}

Restart the runtime; the next chat call uses the production model. Replies are noticeably richer and autonomous tool dispatch is more reliable, in exchange for higher latency per call. For remote endpoints, cloud LLMs, and the full topology decision tree, see Local AI Deployment and Performance.

Performance baseline — what to expect

Indicative latencies for the production-tier qwen2.5:7b-instruct model, CPU-only inference on a typical x64 development laptop (Windows 11, no GPU, 16 GB RAM). These numbers are the upper bound for what the example reaches at production quality. The shipped 3B default runs at roughly half these latencies on the same hardware (~2× faster) in exchange for shorter, less nuanced replies. The first call after runtime start pays a model-load cost; subsequent calls run against the warm model.

Numbers scale roughly linearly with output token count: token-throughput on CPU is ~3–5 tokens/second across all cores. The qwen2.5:3b-instruct model is ~2× faster than 7B on the same hardware (trade against reply quality). Adding a GPU cuts latency 3–5× for either size. For deeper sizing guidance — including when latency in the same-host topology starts hurting other runtime work — see Local AI Deployment and Performance.

How it relates to the deeper Local AI features

This example is intentionally minimal — it demonstrates only the ChatRequest action plus the TChatSession transcript widget. For richer patterns:

• LocalAI KnowledgeGraph Demo — a full shipping solution with operator chat, alarm-driven anomaly narration, knowledge-graph asset tree, and end-to-end grounded narrative generation.

• The three server-side AI.Execute examples on the Local AI parent page show the atomic script path (alarm root-cause hypothesis, multi-language alert translation, end-of-shift summary).

• Local AI Configuration covers tool-category bits (SolutionSettings.ModelOptions), alternate endpoints (cloud LLMs, other local models), and SecuritySecrets-backed authorization for non-local endpoints.

• Local AI Deployment and Performance covers the four hosting topologies (same-host, sibling VM, dedicated host, cloud), the workload × topology decision matrix, and the sizing guidance — essential reading before moving past a workshop / single-operator deployment.

• Local AI Developer Reference covers reply-envelope schema details, cached-path hooks, the AI.Execute deep semantics, and the TChatSession control reference.

In this section...