Page History

A minimal operator-chat panel: six live plant tags, a transcript widget that renders selectable per-role bubbles, a TextBox, a Button, and the ChatRequest action — no scripting required. The smallest end-to-end example of FrameworX Local AI on a Display.

How-to Guides → Solution Examples → Local AI → Local AI Chat Example

Version 10.1.5+

Prerequisites

• FrameworX 10.1.5 or later.

• Local Ollama installed and reachable on http://127.0.0.1:11434. Run AISetup\Install-LocalAI.ps1 from the FrameworX install root if you have not already — see the Local AI - First Install Walkthrough.

• The demo model pulled into Ollama: ollama pull qwen2.5:3b-instruct (~2 GB). This is the model the example ships configured fortargets. To use a higher-quality production-tier model instead, see Moving to a production tier below.

What it contains

Open the solution in Designer and you will see exactly eight UNS tags (six under Plant/, two under Chat/), one Display (ChatTest), one SolutionSettings configuration. Nothing elseNo Devices, no Alarms, no Historian, no scripts.

Plant tags — the live context the operator and the model both see

Chat tags — the operator-to-AI wiring

That is every UNS tag the example needs. No Devices, no Alarms, no Historian, no Scripts. The plant tags carry StartValue so the runtime has live numbers as soon as you click Run; in a real solution they would be driven by a TagProvider, a Device, or a script.

The ChatTest display

One Canvas display at 1200 × 800. Seventeen All elements total, all theme-bound (no hard-coded colors). The transcript widget is the new TChatSession control — native to FrameworX 10.1.5, no Symbol dependency, no DataTable wiring on the adopter side.

Header strip

TextBlock title "Local LLM AI

—

- Integration Test" +

a

one-line subtitle explaining the panel.

Live tags row

Section

Seven TextBlocks. The first is a section

heading ("LIVE TAGS IN SCOPE")

; the next

plus six TextBlocks that bind to the six plant tags via inline expansion in the form {@Tag.Plant/TankLevel} gal. These render in real time as the tags change — the operator and the LLM both see the same snapshot.

Quick-prompt buttons

Three Buttons, each with an ActionDynamic that

sets

runs ActionType=SetValue on @Tag.Chat/Query

to

with a pre-built question. Clicking a button stages the question; the operator can then edit it in the TextBox before sending.

ChatSession control (Uid 13).

Ships with: What is the current tank level? · Summarize the current state of Reactor R-101. · Is the temperature safe given the current pressure?

TChatSession transcript widget

The native FrameworX transcript widget. Renders the per-Display-panel conversation history as selectable per-role bubbles — user prompts right-aligned with

accent tint, assistant replies left-aligned with shade tint. While a request is in flight, a rotating ? glyph with elapsed-seconds counter appears in the transcript; it is replaced by the assistant bubble when the reply lands.

Bubble bodies render as TTextBox rows (with chrome suppressed by the control) so operators can select reply text, Ctrl-C to copy, or right-click Copy — the chat-UX standard. The control reads the transcript directly from the runtime's per-connection cache — no DataTable binding to author, no script glue.

Width 1152, height ~350. Width auto-scales with the Display via OnResize="StretchUniform".

Question TextBox + Ask AI button

Section label, single-line TextBox bound to @Tag.Chat/Query

, plus the load-bearing

Button — ActionType="ChatRequest"

with ObjectValueLink=

@Tag.Chat/Query

and ObjectReturnValueLink=

@Tag.Chat/Reply

. One click sends the current Query to the LLM, receives the reply envelope on Chat/Reply, and the

TChatSession control auto-appends

both turns to its transcript.

Reply + latency footer

Two TextBlocks below the input row — the first prints the raw reply envelope (Local AI Reply: {@Tag.Chat/Reply

}) so you can see the full JSON shape; the second extracts {@Tag.Chat/Reply.JsonString("latencyMs")

} for the round-trip latency in milliseconds.

How to run it

1. Open Local AI Chat Example.dbsln in Designer.

2. Confirm SolutionCapabilities[LocalAI].Enabled = true on Solution → Capabilities → AI Engine tile. The status dot should be green ("Reachable").

3. Confirm qwen2.5:3b-instruct is pulled in your local Ollama (ollama list). This is the model the example ships configured for. For higher-quality production-tier alternatives, see Moving to a production tier below.

4. Click Run. The runtime starts and the RichClient (or your configured client) opens to the ChatTest panel.

5. Click one of the three quick-prompt buttons to stage a question, or type your own in the TextBox.

6. Click → Ask AI. Within The transcript shows your prompt bubble immediately and a rotating ? pending indicator; within a few seconds the assistant bubble replaces the indicator with the model's reply text appears in the ChatSession transcript above and the footer shows status=ok with the latency . The Latency field shows the round-trip in milliseconds.

The ChatSession TChatSession control retains the conversation per Display panel — you can ask follow-ups and the model has context. If you want a fresh conversation, close the panel and reopen itThe transcript persists across re-opens of the same panel within one client session.

What to try next

• Copy a reply. Select text in an assistant bubble and press Ctrl-C, or right-click and pick Copy. The TChatSession's TTextBox bubbles support standard text-selection gestures — useful for pasting LLM-generated content into reports, tickets, or follow-up scripts.

• Change a plant tag value mid-conversation. Use Designer's Watch panel to drive Plant/TankLevel from 784.5 to 950, then ask the AI "What is the current tank level and how close are we to the cap?". The model sees the new value because the live tags tag expansion in the display passes the current snapshot on every call.

• Add a fresh question on the canvasquick-prompt button. Duplicate one of the three quick-prompt buttons and change its ObjectValueLink to a new question string. Click it during Run and the new question stages in the TextBox.

• Inspect the full envelope. The footer shows only status and latencyMsthe whole reply JSON plus latency. Add a fourth TextBlock bound to {@Tag.Chat/Reply.JsonString("warnings")} if you want to see surface warnings; bind the toolTrace field {@Tag.Chat/Reply.JsonString("toolTrace")} if you enable the UNS / Alarm / Historian tool bits on SolutionSettings.ModelOptions (see Local AI Configuration) and want to inspect autonomous tool calls.

• Move the LLM off the TServer host. If you have access to a second machine (or a sibling VM), install Ollama there, pull the model on that host, and change the URL field in SolutionCapabilities[LocalAI].Settings from localhost 127.0.0.1 to the remote IP or hostname. The example continues to work unchanged — TServer no longer fights the LLM for CPU. Full guidance on when this matters: Local AI Deployment and Performance.

• Try a cloud LLM endpoint. Point URL at any OpenAI-compatible chat-completions endpoint (cloud or otherwise); store the API key in SecuritySecrets and reference it via /secret:<Name> in the Authorization field. See SecuritySecrets Authentication for Local AI.

System prompt

The runtime applies a short default system prompt to every chat call so replies stay terse and operator-friendly:

You are a helpful assistant embedded in the FrameworX runtime. Answer concisely and directly — no preamble, no status wrapper, no echo of the user's question. On tool failure, reply with only the word: error.

To override it for a specific call, send a JSON query payload with a top-level system field instead of the bare prompt string. The override applies for that call only; the next bare-string call falls back to the default. Server-side AI.Execute calls follow the same contract.

Role labels — tolerant matching

The TChatSession control distinguishes user-role from assistant-role bubbles via the UserRoleValue and AIRoleValue properties. The example uses the defaults: user / assistant — the literal role strings the runtime writes into the transcript. Leave these alone unless you have a reason to change them.

The control matches role labels case-insensitively and also recognises common aliases on each side, so persona overrides applied elsewhere on the Display still tint correctly:

• User aliases: user, operator, human, question, you, me, customer, client, person, prompt.

• Assistant aliases: assistant, ai, bot, agent, response, answer, plant ai, system ai, system, model, copilot, reply.

For persona labels visible on the Display ("Operator", "Plant AI", etc.), add a separate TextBlock above the transcript — do not push the persona text into UserRoleValue / AIRoleValue. The control's contract is to match what the runtime writes; the persona is a Display-level affordance.

Moving to a production tier — 7B, larger models, remote endpoints

The example ships configured for targets qwen2.5:3b-instruct so it runs end-to-end on a workshop laptop with no GPU. For production deployments — where reply quality, autonomous tool-call reliability, and concurrent operator load matter — switch to a larger model (recommended: qwen2.5:7b-instruct), or move the LLM off the TServer host entirely:

1. Pull the production model on the LLM host: ollama pull qwen2.5:7b-instruct (~4.7 GB).

2. In Designer, open the solution and go to Solution → Capabilities.

3. On the AI Engine tile, click Edit Configuration.

4. Paste the JSON below into the dialog and save (adjust URL if the LLM runs on a different host):

{
  "URL": "http://127.0.0.1:11434/v1/chat/completions",
  "Name": "qwen2.5:7b-instruct",
  "Authorization": "NoAuth",
  "Headers": "",
  "Info": "Production tier — 7B-instruct (~4.7 GB). Best reply quality and tool-call reliability.",
  "TimeoutSeconds": 60
}

Restart the runtime; the next chat call uses the production model. Replies are noticeably richer and autonomous tool dispatch is more reliable, in exchange for higher latency per call. For remote endpoints, cloud LLMs, and the full topology decision tree, see Local AI Deployment and Performance.

Performance baseline — what to expect

Indicative latencies for the production-tier qwen2.5:7b-instruct model, CPU-only inference on a typical x64 development laptop (Windows 11, no GPU, 16 GB RAM). These numbers are the upper bound for what the example reaches at production quality. The shipped 3B default runs at roughly half these latencies on the same hardware (~2× faster) in exchange for shorter, less nuanced replies. The first call after runtime start pays a model-load cost; subsequent calls run against the warm model.

Numbers scale roughly linearly with output token count: token-throughput on CPU is ~3–5 tokens/second across all cores. The qwen2.5:3b-instruct model is ~2× faster than 7B on the same hardware (trade against reply quality). Adding a GPU cuts latency 3–5× for either size. For deeper sizing guidance — including when latency in the same-host topology starts hurting other runtime work — see Local AI Deployment and Performance.

How it relates to the deeper Local AI features

This example is intentionally minimal — it demonstrates only the ChatRequest action plus the TChatSession transcript widget. For richer patterns:

• LocalAI KnowledgeGraph Demo — a full shipping solution with operator chat, alarm-driven anomaly narration, knowledge-graph asset tree, and end-to-end grounded narrative generation.

• The three server-side AI.Execute examples on the Local AI parent page show the atomic script path (alarm root-cause hypothesis, multi-language alert translation, end-of-shift summary).

• Local AI Configuration covers tool-category bits (SolutionSettings.ModelOptions), alternate endpoints (cloud LLMs, other local models), and SecuritySecrets-backed authorization for non-local endpoints.

• Local AI Deployment and Performance covers the four hosting topologies (same-host, sibling VM, dedicated host, cloud), the workload × topology decision matrix, and the sizing guidance — essential reading before moving past a workshop / single-operator deployment.

• Local AI Developer Reference covers reply-envelope schema details, cached-path hooks, and the AI.Execute deep semantics, and the TChatSession control reference.

In this section...