KevinBytes
Back to writing
4 min readIndustrial Security

AI Safety in Industrial Control Systems

A hypothetical laboratory boundary case showing where AI recommendations, human authorization, and independent safety controls belong.

By Kevin O'Connor

Last reviewed

An AI recommendation becomes a different engineering problem when it can change a physical process. A misleading maintenance summary is one kind of failure. A setpoint change applied to equipment is another, particularly when an apparently reasonable adjustment defeats a safety margin the model doesn't understand.

I would keep the initial role of an industrial assistant narrow: read approved information, explain its recommendation, and submit a proposal for review. Moving from advice to actuation should require a separate design decision with the people responsible for the equipment and process safety.

That is a boundary choice, not a claim that every model is unsuitable for industrial use. The question is which decisions can be justified, bounded, and verified in this particular system.

Standards address different parts of the decision

A standards reference should explain which work it supports. It should not become a shorthand claim that an AI integration is safe.

ReferenceUseful scopeWhat still needs engineering
IEC 61508-1:2010, edition 2General requirements for electrical, electronic, and programmable electronic safety-related systemsSafety functions, lifecycle activities, and assurance appropriate to the actual application
ISA/IEC 62443 seriesSecurity requirements and processes for industrial automation and control systemsThe applicable parts and editions, system boundaries, access paths, and operational responsibilities
ISO/IEC 23894:2023Guidance for AI-related risk managementTranslating identified risks into controls for the particular process
NIST SP 800-82 Revision 3, finalOT security guidance that considers performance, reliability, and safetySite-specific architecture, procedures, and validation

These references were checked in September 2026. They do not establish that fixed software is automatically safe or that a particular agent meets a functional-safety requirement. This article applies their broad areas of concern to a hypothetical design; it does not claim a certified installation.

A hypothetical lab-to-control boundary

Imagine a pilot laboratory with a temperature-controlled vessel and a separate historian. An assistant reads replicated operating data and maintenance notes. It proposes a temperature adjustment to improve consistency between batches. This is an invented scenario, not a client engagement, tested installation, or new industrial service offering.

The assistant has no controller credential. It writes a proposal to an application queue containing the equipment identifier, proposed change, source-data timestamps, relevant operating mode, model configuration, and a reason the operator can assess. A qualified operator reviews the proposal through the existing authorized workflow.

That approval must bind to the specific equipment and change. If the equipment enters maintenance mode, a sensor becomes stale, or the proposed value changes, the old approval should no longer authorize execution. The application should not interpret an earlier conversation message saying "looks fine" as continuing permission to alter a physical process.

A deterministic control layer checks the current operating envelope and interlocks before accepting the command. The actual limits and safe-state behavior come from process engineering and hazard analysis, not from this example. There are no transferable numeric setpoints in a generic blog post.

Historian replica --> Assistant --> Proposal queue --> Authorized operator
                                                         |
                                                         v
                                               Control validation
                                                         |
                                                         v
                                                    Equipment

Independent protective functions remain outside the assistant's authority.

Read-only historian access also needs verification. Trace its network route, database role, available procedures, credentials, and any management interface reachable from the assistant host. Calling an integration read-only doesn't make the surrounding machine incapable of reaching the control network.

Walk through the stale-sensor case

Suppose the historian replica stops updating while the vessel remains in operation. The model can still generate a fluent recommendation from yesterday's plausible readings. A freshness check needs to reject that proposal before an operator mistakes it for current analysis.

Next suppose the measurements are current but inconsistent with another sensor. The system needs a defined response to uncertain input. The assistant may explain the discrepancy, but it should not be able to promote its explanation into an override of the independent validation rule.

Finally, suppose the operator approves a valid proposal and the command response times out. Blindly retrying could repeat a consequential operation. The workflow needs a stable operation identifier and a way to reconcile acceptance and resulting equipment state. A timeout means the result is unknown until reconciled; it does not necessarily mean nothing happened.

These are different tests. They exercise input freshness, uncertainty handling, authority, and execution semantics. None requires a model to defeat a real interlock. Start with offline replay and a simulator, with scope agreed by the equipment and safety owners before any physical test.

Keep protective functions independent

Emergency stops, relief devices, safety controllers, and other protective functions should remain outside the assistant's authority according to the site's safety design. Avoid a shared credential or configuration channel that allows the AI integration to change both the operating recommendation and the limit intended to contain it.

Human review is valuable when the operator has the necessary context, time, and authority. It is a weak safeguard if the interface hides stale data, makes approval routine, or forces an overloaded operator to inspect hundreds of low-value suggestions. Evaluate the approval workflow as part of the design rather than listing "human in the loop" as a completed control.

Record the source snapshot, proposal, approval identity, validation decision, command identifier, acknowledgement, and observed result. That trail supports later analysis, but logging does not replace the safety function. Decide how a logging outage affects operation before an outage occurs.

For a first integration, I would stop at recommendations against a historian replica and measure whether operators can identify stale inputs and reject an unsuitable proposal. Actuation should wait for a separate, reviewed safety and authorization design with a defined fallback that remains safe when the assistant is unavailable or wrong.

For the review itself, I set out the diagrams, access boundaries, operating assumptions, and decision points in Reviewing AI Assistants Connected to Lab and Industrial Systems on TKOResearch. Its hypothetical lab example keeps recommendations and physical actions under separate authority; it makes no claim about a deployed plant or a safety-certified product.

Email updates

Get new research by email

In-depth notes on AI security, threat research, and practical defensive work.

To unsubscribe, email kevin@kevinbytes.com.