Local AI Chat Example

A minimal operator-chat panel: six live plant tags, a transcript widget that renders selectable per-role bubbles, a TextBox, a Button, and the ChatRequest action — no scripting required. The smallest end-to-end example of FrameworX Local AI on a Display.

How-to Guides → Solution Examples → Local AI → Local AI Chat Example

Version 10.1.5+

Prerequisites

• FrameworX 10.1.5 or later.

• An LLM endpoint of your own, reachable from the FrameworX runtime over HTTP. The endpoint must speak the OpenAI-compatible chat-completions API (Ollama is the default reference; LM Studio, vLLM, llama.cpp's server, or any cloud endpoint with the same shape also works). The endpoint should NOT run on the same machine as TServer — see the warning above and Local AI Deployment and Performance for where to place it.

• A model pulled on that endpoint. Choose a size that matches your endpoint's hardware. As a rule: 3B-class models (e.g. qwen2.5:3b-instruct, ~2 GB) work on a CPU-only machine for evaluation; 7B-class models (e.g. qwen2.5:7b-instruct, ~4.7 GB) are the realistic floor for production-quality chat; 13B+ benefits from a GPU. See Local AI Deployment and Performance for the matrix.

• The endpoint URL and the model name written into the AI Engine tile on Solution → Capabilities. You must edit these — the shipped .dbsln carries placeholder values that almost certainly do not match your network.

What it contains

Open the solution in Designer and you will see exactly eight UNS tags (six under Plant/, two under Chat/), one Display (ChatTest), one SolutionSettings configuration. No Devices, no Alarms, no Historian, no scripts.

Plant tags — the live context the operator and the model both see

Chat tags — the operator-to-AI wiring

That is every UNS tag the example needs. The plant tags carry StartValue so the runtime has live numbers as soon as you click Run; in a real solution they would be driven by a TagProvider, a Device, or a script.

The ChatTest display

One Canvas display at 1200 × 800. All elements theme-bound (no hard-coded colors). The transcript widget is the new TChatSession control — native to FrameworX 10.1.5, no Symbol dependency, no DataTable wiring on the adopter side.

Header strip

TextBlock title "Local LLM AI - Integration Test" + one-line subtitle explaining the panel.

Live tags row

Section heading ("LIVE TAGS IN SCOPE") plus six TextBlocks that bind to the six plant tags via inline expansion in the form {@Tag.Plant/TankLevel} gal. These render in real time as the tags change — the operator and the LLM both see the same snapshot.

Quick-prompt buttons

Three Buttons, each with an ActionDynamic that runs ActionType=SetValue on @Tag.Chat/Query with a pre-built question. Clicking a button stages the question; the operator can then edit it in the TextBox before sending.

Ships with: What is the current tank level? · Summarize the current state of Reactor R-101. · Is the temperature safe given the current pressure?

TChatSession transcript widget

The native FrameworX transcript widget. Renders the per-Display-panel conversation history as selectable per-role bubbles in a single left-aligned column — user-role bubbles tinted with the platform accent color, assistant-role bubbles with the neutral shade color. Color alone discriminates role; the layout follows the conversational-AI cluster convention (Claude Code, ChatGPT, Cursor, Copilot, Linear AI) rather than the peer-to-peer messenger two-column shape.

While a request is in flight, the operator's question appears immediately as a user-echo bubble, followed by a rotating ? glyph with an elapsed-seconds counter. The pending visuals are replaced by the assistant-reply bubble when the reply lands.

Bubble bodies render as TTextBlock elements (the WPF TextBlock-derived control native to FrameworX) with chrome suppressed by the control. Operators select bubble text by mouse drag, then Ctrl-C to copy, or right-click Copy — the chat-UX standard. Selection is enabled on both user and assistant bubbles. The control reads the transcript directly from the runtime's per-connection cache — no DataTable binding to author, no script glue.

Width 1152, height ~350. Width auto-scales with the Display via OnResize="StretchUniform".

Question TextBox + Ask AI button

Section label, single-line TextBox bound to @Tag.Chat/Query, plus the load-bearing Button — ActionType="ChatRequest" with ObjectValueLink=@Tag.Chat/Query and ObjectReturnValueLink=@Tag.Chat/Reply. One click sends the current Query to the LLM, receives the reply envelope on Chat/Reply, and the TChatSession control auto-appends both turns to its transcript.

Reply + latency footer

Two TextBlocks below the input row — the first prints the raw reply envelope (Local AI Reply: {@Tag.Chat/Reply}) so you can see the full JSON shape; the second extracts {@Tag.Chat/Reply.JsonString("latencyMs")} for the round-trip latency in milliseconds.

How to run it

1. Open Local AI Chat Example.dbsln in Designer.

2. Go to Solution → Capabilities → AI Engine tile. Edit the URL and Name fields to point at your own LLM endpoint and model (per the Prerequisites above) — the shipped values are placeholders for the author's bench setup. Confirm SolutionCapabilities[LocalAI].Enabled = true and that the status dot turns green ("Reachable") once your endpoint is set.

3. Confirm the model you wrote into the Name field is pulled on your LLM host (ollama list if you're using Ollama). If the model is not present, the chat surface will show an error bubble explaining what's missing — see When the LLM is unavailable below.

4. Click Run. The runtime starts and the RichClient (or your configured client) opens to the ChatTest panel.

5. Click one of the three quick-prompt buttons to stage a question, or type your own in the TextBox.

6. Click → Ask AI. The transcript shows your prompt bubble immediately and a rotating ? pending indicator; within a few seconds the assistant bubble replaces the indicator with the model's reply. The Latency field shows the round-trip in milliseconds.

The TChatSession control retains the conversation per Display panel — you can ask follow-ups and the model has context. The transcript persists across re-opens of the same panel within one client session.

When the LLM is unavailable

The TChatSession control surfaces error states as bubbles in the chat track, not as a separate band or toast. When you click Ask AI:

• If the master kill-switch is off (SolutionCapabilities[LocalAI].Enabled = false), your question lands as an accent bubble, the pending indicator clears, and a danger-tinted bubble appears below it with text: "Local AI master kill-switch is off."

• If Ollama is not installed or not running, you see a danger bubble with the auto-warm failure guidance (install / start steps).

• If the LLM endpoint is unreachable (network blocked, wrong URL), you see a danger bubble carrying the HTTP error detail.

• In every case, your typed question is preserved in the transcript — the chat thread shows what you asked and why no answer arrived. Re-enable the LLM and submit again; subsequent turns work normally.

This mirrors the Claude / ChatGPT / Cursor convention — errors are part of the conversation history, not a separate visual surface. Useful for offline demos: show a customer the panel works end-to-end, disable LocalAI mid-demo, click Send, and the chat surface explains why the AI isn't replying without breaking the operator's mental model.

What to try next

• Copy a reply. Select text in an assistant bubble and press Ctrl-C, or right-click and pick Copy. The TChatSession's TTextBlock bubbles support standard text-selection gestures on both user and assistant bubbles — useful for pasting LLM-generated content into reports, tickets, or follow-up scripts.

• Change a plant tag value mid-conversation. Use Designer's Watch panel to drive Plant/TankLevel from 784.5 to 950, then ask the AI "What is the current tank level and how close are we to the cap?". The model sees the new value because the live tag expansion in the display passes the current snapshot on every call.

• Add a fresh quick-prompt button. Duplicate one of the three buttons and change its ObjectValueLink to a new question string. Click it during Run and the new question stages in the TextBox.

• Inspect the full envelope. The footer shows the whole reply JSON plus latency. Add a TextBlock bound to {@Tag.Chat/Reply.JsonString("warnings")} if you want to surface warnings; bind {@Tag.Chat/Reply.JsonString("toolTrace")} if you enable the UNS / Alarm / Historian tool bits on SolutionSettings.ModelOptions (see Local AI Configuration) and want to inspect autonomous tool calls.

• Move the LLM off the TServer host. If you have access to a second machine (or a sibling VM), install Ollama there, pull the model on that host, and change the URL field in SolutionCapabilities[LocalAI].Settings from 127.0.0.1 to the remote IP or hostname. The example continues to work unchanged — TServer no longer fights the LLM for CPU. Full guidance on when this matters: Local AI Deployment and Performance.

• Try a cloud LLM endpoint. Point URL at any OpenAI-compatible chat-completions endpoint (cloud or otherwise); store the API key in SecuritySecrets and reference it via /secret:<Name> in the Authorization field. See SecuritySecrets Authentication for Local AI.

System prompt

The runtime applies a short default system prompt to every chat call so replies stay terse and operator-friendly:

You are a helpful assistant embedded in the FrameworX runtime. Answer concisely and directly — no preamble, no status wrapper, no echo of the user's question. On tool failure, reply with only the word: error.

To override it for a specific call, send a JSON query payload with a top-level system field instead of the bare prompt string. The override applies for that call only; the next bare-string call falls back to the default. Server-side AI.Execute calls follow the same contract.

Role labels — tolerant matching

The TChatSession control distinguishes user-role from assistant-role bubbles via the UserRoleValue and AIRoleValue properties. The example uses the defaults: user / assistant — the literal role strings the runtime writes into the transcript. Leave these alone unless you have a reason to change them.

The control matches role labels case-insensitively and also recognises common aliases on each side, so persona overrides applied elsewhere on the Display still tint correctly:

• User aliases: user, operator, human, question, you, me, customer, client, person, prompt.

• Assistant aliases: assistant, ai, bot, agent, response, answer, plant ai, system ai, system, model, copilot, reply.

For persona labels visible on the Display ("Operator", "Plant AI", etc.), add a separate TextBlock above the transcript — do not push the persona text into UserRoleValue / AIRoleValue. The control's contract is to match what the runtime writes; the persona is a Display-level affordance.

Moving to a production tier — 7B, larger models, remote endpoints

The example targets qwen2.5:3b-instruct so it runs end-to-end on a workshop laptop with no GPU. For production deployments — where reply quality, autonomous tool-call reliability, and concurrent operator load matter — switch to a larger model (recommended: qwen2.5:7b-instruct), or move the LLM off the TServer host entirely:

1. Pull the production model on the LLM host: ollama pull qwen2.5:7b-instruct (~4.7 GB).

2. In Designer, open the solution and go to Solution → Capabilities.

3. On the AI Engine tile, click Edit Configuration.

4. Paste the JSON below into the dialog and save (adjust URL if the LLM runs on a different host):

{
  "URL": "http://127.0.0.1:11434/v1/chat/completions",
  "Name": "qwen2.5:7b-instruct",
  "Authorization": "NoAuth",
  "Headers": "",
  "Info": "Production tier — 7B-instruct (~4.7 GB). Best reply quality and tool-call reliability.",
  "TimeoutSeconds": 60
}

Restart the runtime; the next chat call uses the production model. Replies are noticeably richer and autonomous tool dispatch is more reliable, in exchange for higher latency per call. For remote endpoints, cloud LLMs, and the full topology decision tree, see Local AI Deployment and Performance.

Performance baseline — what to expect

Indicative latencies for the production-tier qwen2.5:7b-instruct model, CPU-only inference on a typical x64 development laptop (Windows 11, no GPU, 16 GB RAM). These numbers are the upper bound for what the example reaches at production quality. The shipped 3B default runs at roughly half these latencies on the same hardware (~2× faster) in exchange for shorter, less nuanced replies. The first call after runtime start pays a model-load cost; subsequent calls run against the warm model.

Numbers scale roughly linearly with output token count: token-throughput on CPU is ~3–5 tokens/second across all cores. The qwen2.5:3b-instruct model is ~2× faster than 7B on the same hardware (trade against reply quality). Adding a GPU cuts latency 3–5× for either size. For deeper sizing guidance — including when latency in the same-host topology starts hurting other runtime work — see Local AI Deployment and Performance.

How it relates to the deeper Local AI features

This example is intentionally minimal — it demonstrates only the ChatRequest action plus the TChatSession transcript widget. For richer patterns:

• LocalAI KnowledgeGraph Demo — a full shipping solution with operator chat, alarm-driven anomaly narration, knowledge-graph asset tree, and end-to-end grounded narrative generation.

• The three server-side AI.Execute examples on the Local AI parent page show the atomic script path (alarm root-cause hypothesis, multi-language alert translation, end-of-shift summary).

• Local AI Configuration covers tool-category bits (SolutionSettings.ModelOptions), alternate endpoints (cloud LLMs, other local models), and SecuritySecrets-backed authorization for non-local endpoints.

• Local AI Deployment and Performance covers the four hosting topologies (same-host, sibling VM, dedicated host, cloud), the workload × topology decision matrix, and the sizing guidance — essential reading before moving past a workshop / single-operator deployment.

• Local AI Developer Reference covers reply-envelope schema details, cached-path hooks, the AI.Execute deep semantics, and the TChatSession control reference.

In this section...