AI Safety in Industrial Control Systems

An assistant can read maintenance notes without being able to change the equipment they describe. Giving it a controller credential changes the problem: an incorrect recommendation can now become a physical action.

For an initial integration, I'd keep the assistant on the information side of that boundary. Let it read approved operating data and submit a proposal through an existing review process. There is plenty to evaluate there before adding command execution, including whether the recommendations are useful and whether operators can see when the inputs are unreliable.

A proposal for a temperature-controlled vessel

Consider a pilot laboratory with a vessel, a historian, and an assistant that reads a replica of the historian's data. The assistant also retrieves maintenance notes and proposes an adjustment to improve consistency between batches.

Its proposal needs an equipment identifier, a specific change, the source-data timestamps, the operating mode, and an explanation the operator can inspect. The assistant has no controller credential. It writes to a proposal queue that a qualified operator reviews through the established workflow.

Historian replica --> Assistant --> Proposal queue --> Authorized operator
                                                         |
                                                         v
                                               Control validation
                                                         |
                                                         v
                                                    Equipment

Independent protective functions remain outside the assistant's authority.

Before relying on that separation, trace the assistant host's network access and credentials. A read-only database role helps, but a management interface on the same reachable network may create another path to the controller. Include available procedures, shared secrets, and administrative tools in the review.

Data can look plausible after it stops being useful

Suppose the historian replica stops updating. The assistant can continue producing sensible-looking recommendations from yesterday's readings while the vessel has moved into a different operating state.

The proposal needs a freshness check before review, and the execution path needs to check the relevant state again before acting. A timestamp displayed in small text is easy to overlook. Expired source data should make the proposal visibly ineligible for execution under the site's defined policy.

Conflicting sensors are a different case. A recent reading can still be unreliable. The operator needs to see the disagreement and follow the process's established response. The assistant can help explain the data, but its explanation must not override the validation rule or change the limit used to judge its own proposal.

That distinction matters when the model is persuasive. A detailed rationale can make uncertain input feel resolved even though the underlying measurements haven't changed.

Approval has to survive the trip to the equipment

Bind approval to the equipment, requested change, operating conditions, and permitted time window. If the value changes or the equipment enters maintenance mode, the old approval no longer describes the operation waiting to execute.

The control layer then checks the current operating envelope and interlocks. The limits and safe-state behavior come from the process's hazard analysis and engineering design. A deterministic check is useful only if it implements the right rule with the right inputs.

Now consider a timeout after execution. The control system may have accepted the command even though the application never received the acknowledgement. Retrying without checking can repeat a consequential operation.

Carry a stable operation identifier through the workflow and reconcile the command with its acknowledgement and observed equipment state. Keep “result unknown” distinct from “rejected.” Operators need that distinction to decide what can happen next.

I would first exercise stale data, sensor disagreement, changed approvals, and unknown results through offline replay and a simulator. The equipment and safety owners should define the scope and conditions for any later physical test.

Keep the protective functions separately controlled

Emergency stops, relief devices, safety controllers, and other protective functions have roles defined by the site's safety design. The assistant's integration should not gain the authority to change those functions through a shared credential or configuration channel.

Human review also has operating conditions. An operator needs time, context, and the ability to reject the proposal. If the interface hides stale input or produces hundreds of low-value suggestions, adding an approval button contributes much less than the architecture diagram suggests.

Retain the source snapshot, proposal, approver, validation decision, command identifier, acknowledgement, and observed result. Those records help explain a bad recommendation or uncertain command. They also make a logging outage visible enough for the operating procedure to address it.

Fit the review into the existing engineering work

The relevant standards cover different responsibilities:

The process and safety owners need to identify the applicable requirements and validation work for the installation. An AI integration doesn't remove the existing lifecycle responsibilities or make an application approval equivalent to a safety function.

My first release would stay at recommendations against the replica. I'd look at whether the assistant helps operators make a better decision, how often they reject its proposals, and whether bad inputs are caught. Command execution can be considered separately once its authorization, validation, and fallback behavior have been reviewed with the people responsible for the process.