Part 5 – Rules of Engagement: Establishing Governance for Multi-Agent Generative Systems

In parts 1 to 4 of the series, we explored the potential of Multi-Agent Generative Systems MAGS to transform industrial processes. We examined how these systems can enhance decision-making, improve efficiency, and adapt to complex operational environments. As we move forward with implementing MAGS for industrial settings, it’s essential to address a fundamental aspect of their deployment: the rules that govern their behavior.

In this fifth article in the series, Dr. Zoran Milosevic and I will focus on the concept of “Rules of Engagement” for MAGS. These rules serve as the foundation for the responsible and effective use of AI agents in industrial environments. They ensure that our technological advancements align with organizational goals, legal requirements, and ethical standards.

Figure 1 – Position of Rules of Engagement

The need for clear rules becomes apparent when we consider the autonomous nature of MAGS. These systems can make decisions and take actions with minimal human intervention. While this autonomy offers significant benefits, it also presents risks if not properly managed. Establishing robust rules of engagement helps mitigate these risks and builds trust in the technology.

Our discussion will cover various types of rules, including regulatory, organizational, professional, legal, and ethical guidelines. We’ll explore how these rules apply to different entities within a MAGS ecosystem, including human operators, organizations, and AI agentsthemselves.

We’ll also examine practical approaches to implementing these rules. This includes standards-based enterprise modeling and other structuring methods that can define intentions, trust relationships, and interactions between agents and humans. We will explain how these Rules of Engagementare areimplemented in XMPro MAGS.

Let’s begin by examining the fundamental concepts of rules and their importance in complex systems like MAGS.

Engagement Rules: Why

Complex systems consist of diverse actors, each with their own goals, as reflected in their intentions and behavioral preferences. These actors can engage in interactions and collaborations within existing structures, such as organizations, where they accept obligations or commitments associated with their defined roles. They can also form new partnerships with other actors based on mutual agreements.

While internal rules guide the behavior of actors as independent entities and their involvement in collaborations, they must also adhere to external regulations. Both internal and external rules contribute to maintaining system stability, predictability, and trust in an uncertain environment.

Our earlier articlesdemonstrate how AI agents are becoming an important element of complex systems, addressing the resource scarcity, data complexity and unlocking of new economic values.

AI agents, engaged by humans or organizations, are a type of actor whose behavior can simulate or mimic that of humans. However, ultimate accountability lies with the parties that have legal responsibilities, such as human creators or organizational owners. While AI agents may delegate tasks to other agents, the ultimate responsibility remains with the human or organization. The importance of rules in this context is to clearly define responsibility and accountability across both human and AI agent entities.

We use the term Rules of Engagement to encompass the various types of rules needed to govern the behavior of actors in complex systems. These rules can be used to define obligations, permissions, and prohibitions for actors involved in collaborative activities. They can originate from internal governance arrangements or from external rules.

The latter include regulatory compliance, legal requirements, ethical considerations, professional conduct, engineering and security, safety and risk management, resource management, environmental responsibility, and more.

Engagement Rules: What

When considering computable approaches for supporting engagement rule expressions, the above considerations suggest a need to express:

• Organizational context that defines rules, whether the context captures a regulatory domain that specifies controlling rules over members of the domain (both individual actors and organisational structures), or a collaboration structure whose objective drives the rules of collaboration.
• Legal entities that have their own independent life and identity, regardless of their participation in any collaboration
• Collaboration structures that can define templates for interactions; these allow for multiple instantiations, such as different parties filling roles in such collaboration structures in different times.
• Organizational or legal policies that cover constraints over behavior of both the actors (parties and AI agents) and collaboration structures, providing guardrails over their autonomous behavior.
• Range of organisational or legal policy types, including simpler rules associated with obligations, permissions and prohibitions, but also their derivation, needed for more complex  expressions of accountability and responsibility (see Figure).
• Dynamics of the obligations, permissions and prohibitions as a result of authorisation or delegation of services or responsibilities across actors, while ensuring clear traceability of responsibility across the actors
• Agentic behavior to define possible actions of actors over time, reflecting their objectives while also responding to the actions of others.
• System policies that can support the variability of system design over time and thus facilitate system evolutions, including the change of organisational or regulative policies.

Engagement rules: How

When building complex systems, the engagement rules should be articulated in a computer interpretable style, to ensureconsistency, interoperability and evolvabilityof the systems, as the operating environment rules change, including the availability of new technologies.

Such computable expressions in turn requires formal modelsto ensure that the computable language can be interpreted using suitable tools, including those that check consistency and traceability from business requirements to implementation, in support of conformance and compliance checking.

One practical approach to implementing such rules is based on formal models embedded in the concepts specifically developed to design and implement large or complex systems, namely the ISO/ITU-T/IEC open distributed processing systems (ODP) standard [1]. One component of the standard, the ODP Enterprise Language provides [2] much of the semantics needed semantic foundationfor describing rules of engagement.

(* adapted from an upcoming conference publicatiob by Zoran Milosevic )

For example, the ODP concept of party, can be used to model actors thar have broad set of responsibilities derived from some social or legal framework, and these can be natural persons or organisations.

Parties may engage automated agents], such as AI agents discussed previously, to perform actions on their behalf, and this involves delegation of their responsibility to address resource issues, either to do with capability specialisation or scalability reasons.   The aim here is to be able to trace the way that the rightsand responsibilities of the parties are linked to the individual system actions and their consequences, which is key to ensuring building responsible digital twin systems.

Further, the ODP concept of community is useful to define the collaboration structures introduced earlier, which serve as templates for interactions of actors participating in collaborations. A community is essentially a contract specifying how actors can participate in different community roles to fulfil their objectives, and that of the community’s objective. The contract is expressed in terms of obligations, permissions or prohibitionspolicies that apply to the actors fulfilling these roles.

These policies, often referred as deontic policies, are intended to capture the rules of the community contract, including those that propagate to the community, from the outer regulatory environment. Note that the term “Deontic” has roots in philosophy and logic,specifically in the study of obligation, permission, and prohibition and in recent AI developments they are employed to design systems that adhere to rules and regulations, such as access control mechanisms, legal frameworks, and ethical guidelines for AI [3].

The ODP-EL standard provides a practical implementation path for handling these deontic and accountability concepts, through encapsulating them within objects that can be handed over across the actors in the system, referred to as deontic tokens [2]. Here, the holders of these tokens, are constrained  by the nature of policies encapsulated within them. This is similar mechanism as used in many recent security policy approaches such as OAuth 2.0 access tokens, although the constraints here are broader and apply to any holder’s actions rather than to narrower, data access actions, in access control policies.

Further, deontic tokens can be associated with some important actions performed by actors, when they result in changes of obligations, permissions or prohibitions associated with the actors.

These actions are referred to as speech acts, introduced following the linguistic concepts of speech act, and they can be used to express how certain actions such as requests, orders or promises, change the state of world, from the perspective of obligations or permissions of actors involved. They are powerful way of expressing chain of responsibility across parties and their AI agents.

This foundational model for expressing engagement rules as constraints on expected behaviour of agents, also serves as a basis for the inclusion of related modelling concepts/formalism and techniques needed for analysing and reasoning about system properties related to AI agent technologies.

One such technique, referred to as Promise Theory [4], is suitable for describing intention of AI agents and how these can be related to their objectives, future actions, and decisions, and thus the deontic formalism mentioned above. Such decisions can be influenced by agents own objectives and commitments, while allowing for potential choices associated with expected behaviour and trust in other agents with which they can interact and collaborate.

How do we implement the Rules of Engagement in XMPro MAGS

XMPro’s Multi-Agent Generative System MAGS uses a structured approach to implement Rules of Engagement for AI agents.. This framework ensures AI agents operate within defined boundaries and follow organizational policies. It also helps agents make decisions that align with ethical standards and business objectives, which is crucial in complex industrial settings.

Here is an example from the oil and gas industry. A major refinery uses XMPro MAGS to manage its operations, from production planning to maintenance scheduling and safety monitoring. The system includes various AI agents responsible for different aspects of the refinery’s operations, each needing to work in harmony with others while adhering to strict industry regulations.

Figure 3 – Agent Profile and Agent Instance

"Prioritize safety in all operations" 
"Comply with environmental regulations at all times" 
"Optimize resource utilization to minimize waste" 
"Maintain product quality standards across all production lines" 

"Agents must report any safety anomalies immediately" 
"Production agents may adjust output levels within specified ranges" 
"Maintenance agents must not schedule repairs during peak production hours unless critical"