**Application area:** Energy Systems

**Researcher in charge:** Precious Ugo Abara

**Supervisor:** Prof. Sandra Hirche

**Host:** Technische Universität München, Germany

**Secondments:** Siemens, Germany and TNO, The Netherlands.

**State-of-the-Art**

One of the core challenges in networked cyber-physical systems is the control under resource constraints in the cyber part, e.g. communication constraints due to energy limits of battery-driven wireless sensors. Event-triggered control schemes, see Bernhardsson and Åström (1999), Molin and Hirche (2013) and Heemels et al. (2012), that is, control schemes where information is only transmitted when important, have been proven to perform particularly well if resource constraints are present. However, along with the benefits of event-triggered schemes, several issues emerge in the analysis and design of such systems that are not present in the time-triggered control. For example, the optimal design of control and event-triggered policies of a single loop is a challenging issue as two distributed decision makers are involved (Molin and Hirche (2014)). Even for simple systems with linear dynamics and quadratic cost, the optimal solution is hard to find when the information pattern is distributed (Witsenhausen (1968)). Moreover, what makes these problems challenging is that standard techniques of stochastic optimal control theory are not directly applicable. In the time-triggered sampling approach, there is no additional information contained in the timing variables since these are available beforehand. Meanwhile, in the event-triggered strategy, a particular attention has to be paid to the potential complication in the control design due to the dual effect of control (Ramesh et al. (2011), Bar-Shalom and Tse (1974)). The control is said to have a dual effect when it affects the system’s state evolution and it can probe the system to reduce its state uncertainty, i.e., improve the estimation, which ultimately helps to achieve the control objectives. In the project described in this proposal, we will investigate event-triggered optimal control framework and the influence on:

- the dual effect of control,
- the separation between estimation, control and scheduler,
- the information pattern in the distributed optimal control problem,
- the distributed estimation problem (i.e, also in case of
*negative information*), - the stability of interconnected systems.

**Research Question and Approach**

The research focus is on the design of efficient and scalable distributed control and scheduling algorithms for large-scale CPS with resource constraints, which represents a widely open research problem. Control of such systems over communication networks is an important topic with many application domains such as infrastructure systems and smart grids. Modelling and design of the various aspects in these domains involve many challenges, some of which will be addressed by this ESR topic:

- distributed control strategies such that the overall system maintains stability or meets/optimizes performance,
- control strategies that can cope with
- asynchrony between local loops,
- event-triggered control and scheduling of multiple event-triggered loops, aiming to reduce expensive and time-consuming information exchange between control loops.

The first question to be answered concerns the role of the information pattern in the optimal control, i.e. how the information pattern should look like such that the control problem has a feasible and possibly linear solution. Our starting point are recent results on dual effect and separation properties in single-loop optimal structure for linear-quadratic settings. The main idea is to extend those work to a multi-loop system setting where the single subsystems are physically coupled and derive feasible information pattern such that the optimal control problem can be solved in a distributed fashion employing standard dynamic programming techniques

. The result of this investigation will be a guideline for finding easy and fast solutions to the event-triggered optimal control of interconnected systems.

In a second step, we investigate the problem of optimal control subject to communication constraints. The main question then is to find an appropriate distributed control, which minimizes the control cost subject to the communication constraints (i.e., or a trade-off between the expected control cost and the expected communication cost). The obtained results on optimal information pattern and separation for tractability of the solution will guide the search as a starting point. This creates the theoretical foundation for the co-design of distributed control and communication architectures.

Successively, we will work on the distributed structure of the controllers and scheduling laws to guarantee efficiency, scalability and flexibility. Exploit the hybrid nature of event-triggered and time-periodic measurement information.

More specifically, we will investigate:

- effect of event-triggered sampling on estimation and optimal costs,
- state estimation between two consecutive events with “negative information” (i.e., the information carried by not receiving a measurement),
- combined stochastic and set-membership information filtering in multi-sensor systems.

Moreover, we will develop some rudimentary simulation framework. Embed the analysis and control synthesis tools into a toolbox control and validation of the methodologies.

**Approach:**

The continuing evolution of physically complex and distributed systems requires the allocation of energy and computational resources in an efficient manner. These requirements depend on the application as there might be some constraints on the information sharing between subsystems, for example, due to limits imposed by communication. Key challenge and innovation is the joint consideration of physical and cyber-constraints. The physical constraints are due to the physical coupling in large-scale interconnected systems, where the subsystem’s dynamics influence each other on the physical level. The cyber-constraints result from limited resources such as communication and computation.

In our current set-up we consider (see Figure):

- a physical graph with interconnections representing coupling (possibly large-scale),
- a control graph, where each control unit bases its decision on distributed information,
- a scheduler attach to each plant that decides whether or not to send most recent measurement the its local controller,
- a shared communication channel between all agents of the network (capacity constraints or minimization of expected communication cost).

We aim to develop a framework for the (joint) design of distributed control and scheduling laws, which guarantee efficiency, scalability, and flexibility. We will derive stochastic stability and convergence conditions for the overall system using drift criteria and Lyapunov-type analysis tools. Important aspects include the distributed and decentralized implementation of the protocols/controls and the potential for easy reconfigurability of the overall system. We will investigate the trade-offs between implementation complexity, platform specifications, and achievable control performance, which eventually leads to architectural guidelines for the design of large-scale cyber-physical systems.

**Tools: **

The developed analysis and control synthesis tools will be embedded into a toolbox for real-time control and protocol design, which will be further developed, tested and validated with the academic and industrial partners within the ITN.

- MATLAB/Simulink: software and graphical modelling environment used for simulations and analysis of concepts and ideas before trying to obtain analytical proofs.
- ROS: Physical experimental set-up where we plan to apply optimal control and/or trajectory tracking while keeping the communication overhead to minimum through an appropriate scheduling policy.

**Bibliography**

Astrom, K. J., & Bernhardsson, B. M. (2002, December). Comparison of Riemann and Lebesgue sampling for first order stochastic systems. In *Decision and Control, 2002, Proceedings of the 41st IEEE Conference on* (Vol. 2, pp. 2011-2016). IEEE.

Bar-Shalom, Yaakov, and Edison Tse. “Dual effect, certainty equivalence, and separation in stochastic control.” *IEEE Transactions on Automatic Control* 19.5 (1974): 494-500.

Heemels, W. P. M. H., Johansson, K. H., & Tabuada, P. (2012, December). An introduction to event-triggered and self-triggered control. In *Decision and Control (CDC), 2012 IEEE 51st Annual Conference on* (pp. 3270-3285). IEEE.

Molin, A., & Hirche, S. (2013). On the optimality of certainty equivalence for event-triggered control systems. *IEEE Transactions on Automatic Control*, *58*(2), 470-474.

Ramesh, C., Sandberg, H., Bao, L., & Johansson, K. H. (2011, June). On the dual effect in state-based scheduling of networked control systems. In *American Control Conference (ACC), 2011* (pp. 2216-2221). IEEE.

**Ugo Abara, P., Hirche, S. (2017). “Separation in Coupled Event-Triggered Networked Control Systems”. International Federation of Automatic Control (IFAC) World Congress 2017.**

Witsenhausen, Hans S. “A counterexample in stochastic optimum control.” *SIAM Journal on Control* 6.1 (1968): 131-147.