Quantifying the robustness of fuzzy rule sets in object-based image analysis

Robustness is one of the fundamental characteristics of biological systems. Numerous reports have been published on how robustness is involved in various biological processes and on mechanisms that give rise to robustness in living systems (Savageau, 1985a, 1985b, 1998; Barkai and Leibler, 1997; Alon et al, 1999; von Dassow et al, 2000; Bhalla and Iyengar, 2001; Csete and Doyle, 2002, 2004; Kitano et al, 2004, 2004a, 2004b; Stelling et al, 2004; Kitano and Oda, 2006; Kitano, 2007a). With increasing interest in systems biology, properties at the system level such as robustness have attracted serious scientific research. Nevertheless, a mathematical foundation that provides a unified perspective on robustness is yet to be established. For systems biology to mature into a solid scientific discipline, there must be a solid theoretical and methodological foundation. Often, systems biology is equated with computer simulation of cells and organs. Although computer simulation is a powerful technique for clarifying the complex dynamics of biological systems, it is also a useful tool for exploring the foundation of biological systems. While investigation on the dynamic properties of specific aspects of organisms is scientifically significant and can be widely applied, it is a study on specific instances of design within a design space that is shaped by fundamental principles, structural, environmental, and evolutionary constraints. The scientific goal of systems biology is not merely to create precision models of cells and organs, but also to discover fundamental and structural principles behind biological systems that define the possible design space of life (Figure 1). The value of understanding fundamental and structural theories is that they provide deeper insights into the governing principles that complex evolvable systems including biological systems follow. Building a solid theoretical foundation of biological robustness, and in particular defining a mathematical formulation of robustness, represents a key challenge in Systems Biology. Such a framework would be enormously useful, as it would provide general constraints on possible architectural features of living organisms. The concept of robustness First of all, there must be a common understanding on what ‘robustness' means. Defining any scientific term is a nontrivial issue, but in this paper, the following definition will be used: ‘robustness is a property that allows a system to maintain its functions against internal and external perturbations.' (Kitano, 2004a). A similar definition with a slightly different phrasing was used by others, such as ‘robustness, the ability to maintain performance in the face of perturbations and uncertainty, is a long-recognized key property of living systems' (Stelling et al, 2004) and is thus considered to be the most appropriate definition. It is important to choose the most reasonable and appropriate definition, rather than creating yet another definition of robustness. To discuss robustness, one must identify system, function, and perturbations. It important to realize that robustness is concerned with maintaining functions of a system rather than system states, which distinguishes robustness from stability or homeostasis. Homeostasis is described as follows: ‘The coordinated physiological processes which maintain most of the steady states in the organism are so complex and so peculiar to living beings—involving, as they may, the brain and nerves, the heart, lungs, kidneys, and spleen, all working cooperatively—that I have suggested a special designation for these states, homeostasis. The word does not imply something set and immobile, a stagnation. It means a condition—a condition which may vary, but which is relatively constant (Cannon, 1932)'. According to this definition, homeostasis is clearly a property that maintains the state of the system rather than its functions. Homeostasis, stability, and robustness will be identical if the function to be preserved is the one that maintains the state of the system. In addition, the robustness of a subsystem often contributes to homeostasis of the system at the higher level. Such examples can be seen in yeast diauxic shift (DeRisi et al, 1997) and glycolytic shift in tumor metabolism (Mazurek and Eigenbrodt, 2003) in which the state of the system changes at the level of metabolic functions that maintain ATP production despite environmental perturbations. This illustrates that robustness—not stability or homeostasis—of subsystems may contribute to homeostasis of the whole system when the function maintained, ATP production in our example, is related to the stability of the system state at the higher level. Whereas homeostasis and stability are somewhat related concepts, robustness is a more general concept according to which a system is robust as long as it maintains functionality, even if it transits through a new steady state or if instability actually helps the system to cope with perturbations (Figure 2). Such transition between states is often observed in biological systems when facing stress conditions. An extreme example can be seen in the anhydrobiosis of tardigrade that suspends metabolism almost completely, if not entirely, under extreme dehydration and enters the dormant state, surviving for years (Crowe and Crowe, 2000). This dormant state is attained by extensive production of trehalose and tardigrade become active again upon rehydration. Such dramatic shifts can be observed in other organisms as well (Singer and Lindquist, 1998), and some have argued that this is a third form of life called ‘cryptobiosis' (Clegg, 2001). These examples of extreme robustness under harsh stress conditions show that organisms can attain an impressive degree of robustness by switching from one steady state to the other, rather than trying to maintain a given state. Such a phenotypic switch is also observed in bacteria and can be considered to be important for drug-resistance (Balaban et al, 2004). Robustness is also not identical to stability. Some species gain robustness by increasing instability in a part of its system. The HIV-1 virus is robust against numerous therapeutic interventions due to a high mutation domain (Larder and Kemp, 1989; Tisdale et al, 1993), which is one of the general mechanisms for viral survivability (Eigen, 1993), and tumors are robust against various chemotherapies, because chromosome instability enhances heterogeneity within a tumor cell population (Baisse et al, 2001; Rasnick, 2002). In summary, whereas robustness is a general concept, homeostasis or stability can be considered as particular instances of robustness. Under modern control theory, a set of sophisticated methods generally called ‘robust control' has been developed. Robust control assumes uncertainties in a model and defines a method of applying stable control over the system such that proper control is guaranteed even if the model deviates from the real system due to modeling errors (Zhou and Doyle, 1997). Note that robust control assumes a control system that stabilizes the target system so as to be robust against model errors; this mechanism for robustness is consistent with the definition of robustness given above. Nevertheless, control theory assumes a system that is designed to meet given criteria, and so it cannot be directly applied to biological systems that have evolved and for which the desirable state of the system is not explicit. In addition, most of the mathematics used to describe robustness are mostly based on control theory, which tend to focus on stability and performance of monostable systems. A theory that take into account multistability and evolution of instable systems needs to be developed and new theoretical avenues need to be explored to provide a broad and unified account of robustness of biological systems. A particularly interesting topic in the context of robustnesss is its trade-offs. What kind of trade-off exists in biological systems? Is robustness conserved? Does a trade-off between robustness and fragility indicate some kind of conservation principle as claimed by Csete and Doyle (2002)? Highly optimized tolerance (HOT) theory demonstrates, taking the example of a forest fire, that a system that is optimized for a specific perturbation inevitably entails extreme fragility for unexpected perturbations (Carlson and Doyle, 1999, 2002) (see Box 1). Commercial jet airliners with fly-by-wire control are highly robust against most component failures and atmospheric perturbations, but become extremely fragile against highly improbable events such as a total power failure as they depend entirely on electric control. The Wright Flyer, on the other hand, is a non-robust system but free from power failure problems, because it does not use any electric system. Biological examples of such trade-offs are abundant. Some diseases can be considered as manifestations of such trade-offs (Kitano et al, 2004; Kitano, 2004b; Kitano and Oda, 2006), and the efficacy and side effects of drugs may be related to robustness trade-offs (Kitano, 2007b). In addition, biological trade-offs may actually not only involve robustness and fragility, but also resource demands and performance of the system. For example, having an entire backup copy of the system enhances robustness against component failure due to redundancy, but it doubles the resources required and may therefore degrade the performance of the system. Thus, when robustness of the system against certain perturbations is increased, it may result in increased fragility against unexpected perturbation, increased resource demands, and degradation of performance. A simultaneous increase of robustness and reduction of fragility may be achieved when additional resources are integrated properly into the system or if system performance is reduced. Alternatively, a system's performance can be maximized by giving up robustness of the system against various perturbations. We should also note that these features are not independent. Performance, in terms of maneuverability of some animals in a hostile environment, may translate into robustness against predator attacks. Increased resource demands may translate into fragility against severe resource competition as well as perturbation on available resources. The key issue is whether it is possible to find a formalism in which robustness and its trade-offs could be defined so that robustness is a conserved quantity or whether the trade-offs discussed above are bound to remain at the level of useful but empirical observations. Understanding such trade-offs would be critically important to understand the basic design principles of life at the level of individual organisms and cells. It may also explain the origin of the diversity of life through evolutionary selection of design space under competitive environments. Mammals have evolved to be highly robust against a broad range of perturbations, but require important resources for their development and maintenance of their daily life. Bacteria, on the other hand, have adopted a set of rather simple mechanisms at the individual level, but can reproduce very quickly and sustain huge populations due to smaller resource demands than other species. How can we map different evolutionary niches within a map based on robustness and its trade-offs? Mathematical formulation of biological robustness The effort towards formalizing a theory of robustness and its trade-offs is still in its infancy and much remains to be completed to build a mature theory. For a theory to be useful, it must be able to predict characteristics and behaviors of the system. This means that the theory has to be framed to explicitly describe constraints that bind the system. First of all, mathematical definitions of terms are given. Robustness can be defined as a system's characteristics that maintain one or more of its functions under external and internal perturbations. Under this definition, robustness (R) of the system (s) with regard to function (a) against a set of perturbations (P) can be mathematically described as: The function ψ(p)is the probability for perturbation ‘p' to take place, and this should be 1 when all perturbation to take place at equal probability. D (p) is an evaluation function under perturbation (p), and P is the entire perturbation space. The evaluation function determines if the system still maintains function under a perturbation and to what degree, and is defined as: A is a set of perturbations where the system failed to maintain its function. This means that D(p) is zero when a function does not meet a defined criteria under perturbation (p) and D(p) returns a relative viability of a function under perturbation compared against non-perturbed condition otherwise. For example, ATP production drop 20% under a certain perturbation compared with ATP production under unperturbed state, then 0.8 shall be returned. Note that p in this equation represents a specific instance of a perturbation. Figure 3 illustrates definition of robustness. A system ‘S1' can be said to be more robust than a system ‘S2' with regard to a function ‘a' against a certain set of perturbations ‘Y', when However, considering an entire perturbation space (P), or sufficiently broad perturbation space, robustness-fragility trade-off should hold, thus difference of robustness (ΔR) between two systems shall be: which is reminiscent of the Bode integral formula. If robustness is conserved, then above equation should be zero with equiprobability over the perturbation space (see also Figure 4A). Assuming that S1 and S2 are the same system but with parameters optimized for different subset of perturbations, this equation implies that any increase in robustness against a subset of perturbation will be off-set by decrease of robustness against other perturbations. In fact, the notion that trade-offs between robustness and fragility represents a conservation of robustness (fragility) in biological systems was initially inspired by the so-called Bode Integral formula (Csete and Doyle, 2002): Where S(ω) is the sensitivity of a system at a frequency ω. The Bode integral formula represents conservation of sensitivity of a negative feedback (NFB) system along the frequency axis (Bode, 1945) (the relevance of this theorem to biological systems is best described in (Csete and Doyle, 2002); see also Box 2). The Bode theorem indicates that an improvement of sensitivity gained by NFB in the low-frequency range is traded-off by increased instability in the high-frequency range. In addition, within the theoretical framework of Metabolic Control Analysis, the summation and connectivity theorems represent constraints that are imposed on parametric changes in metabolic pathways (Fell, 1992), implying that the sensitivity of the network is conserved for changes in rate constants. As noted above, trade-off between robustness and performance also need to be considered. It is often the case that systems that are particularly well tuned for a specific task under a given environment are fragile against change in the environment. In contrast, systems with moderate performance tend to be more robust and thus can remain functional under a broader range of conditions (Figure 4B). Such trade-offs need to be formulated as well. In electric engineering, amplifier design is known to involve constraints on Gain-Band Width, which represent similar trade-offs. How such trade-off can be generalized to biological systems remains to be explored. Similar argument apply to trade-offs between robustness and resource use, where robustness against component failure can be improved by having a greater level of redundancy, hence increased resource demand. An example of this is provided by reliability engineering, which offers a mathematical basis for reduced fault rate (Figure 4C). Nevertheless, it is unclear whether formulations for each trade-off can be integrated into a single unified system of equations. However, efforts to further elaborate such relationships shall provide us deeper mathematical insights into biological systems. Future challenges This article briefly discussed a primitive concept of how biological robustness may be formulated mathematically and raised some of the key issues that remain to be resolved. Although there are numbers of challenges ahead, it is clearly understood that much of the basic mathematics are already in place provided we deal with a well-chosen set. Further theoretical studies should be able to utilize such formalization as a starting point. Bode integral theorem and a set of theorems from metabolic analysis have already illustrated the conservation of robustness to some extent, and reliability engineering is a solid basis for component failure analysis. Mathematical and experimental studies are still required to characterize the trade-off relationship between robustness and performance. It will be a major challenge to find out under which conditions trade-offs exist and how to calculate system-level properties such as robustness or performance, when additional resource use is accompanied with changes in system configuration. In the long run, the theory should be extended to deal with major structural changes. This will require the elaboration of definitions based on biological network properties, and the development of a comprehensive set of innovative computational methods to derive such characteristic quantities for large systems. This is an important undertaking as it may bring abstract theory to practical utility by providing specific constraints underlying the organization of biological organisms and subsystems. As progresses in theoretical research will derive more concrete constraints, we should be able to better predict and reverse-engineer the structures of biological networks. Combined with various high-throughput experimental data, we should be able to derive the structures and dynamics of networks with higher accuracy. The current mathematical formulations are mostly concerned with the stability and maintenance of the system's functions against perturbations. As discussed at the outset, robustness is a broader concept than stability. A theory that would account for phase transition and instability as means to achieve robustness would need to be formulated and integrated with theories on stability. Although instability-based robustness involves survival of the fittest under selective pressure, it needs to be integrated with mathematical framework on evolution, genetics, and game theory (Maynard-Smith, 1982). Ultimately, the theory will have to be interfaced with thermodynamics. Studies on nonequilibrium dissipative systems are mostly focused on chemical reactions and some are trying to extend theories on nonequilibrium dissipative systems to the principles of life (Prigogine et al, 1974). However, the theories still do not take into account the heterogeneity and structured nature of biological systems as well as selection through evolution and it is a major challenge to attempt bridging this gap. The situation is similar for the fields of nonlinear dynamics and chaos, for which theories that embrace the characteristics of biological systems are yet to emerge. Formulation of a fundamental theory of biological systems is one of the grand challenges in biology. In very general terms, this will involve resolving the gap between the level of description used in thermodynamics and other basic physical sciences—for example, the properties of ensemble of molecules in a medium—and the abstraction level used to define the concepts elaborated in this article, which involve networks of biological interactions. Hopefully, the ideas and concepts discussed in this article will stimulate discussions and provide some stepping stones for research directed towards these ambitious objectives. Robustness-fragility trade-offs in forest fire countermeasures The HOT model argues that systems that have evolved to have a higher level of complexity are optimized for specific perturbations but, at the same time, are also inevitably extremely fragile against unexpected perturbations (Carlson and Doyle, 1999, 2002). Carson and Doyle use the example of a forest fire to illustrate the intrinsic nature of the system. Forest buffer zone patterns and tree planting patterns that are optimized for specific types of fire can be very fragile against unexpected types of fire, and may ignite from unexpected directions (A). They also argued that a design that can utilize a large degree of freedom can be more optimal for anticipated perturbations, and therefore can be extremely fragile against unexpected perturbations (Reynolds et al, 2002). Of course, such trade-offs are not limited to the simple design of buffer zone locations. If one decides to plant all trees around a city in a circular manner to be able to cope with fire from any direction, such design may actually allow a fire to spread and encircle the city, causing major damage to the city as well (B). However, if one cuts all the trees in the field, the city will be very fragile against the rainy season as flooding will be more likely due to the resulting loss of water-absorbing capacity of the trees (C). The point here is that such trade-off is inherent and cannot be avoided. This figure is inspired by the HOT theory (Carlson and Doyle, 1999, 2002). Robustness trade-offs in engineering and physics. There have been various studies on the trade-offs between robustness, fragility, and performance in engineering systems as well as in physics. In amplifier design, the trade-off between stability in specific frequency range provided by a NFB loop is compensated by increased instability in higher frequency region and less overall gain of the amplifier. This is a central issue in electric circuit design, and has been intensively investigated in control theory. Assuming a simple feedback circuit as seen in amplifiers, the steady-state sensitivity (S) against a perturbation (d) of the system having feedback gain (G) is defined by S=1/(1+G). Therefore, a larger gain reduces the sensitivity and hence increases the robustness against perturbations. However, frequency domain analysis shows that such increase in robustness increases fragility in a specific frequency domain (A and B). Sensitivity of the system against perturbations (fragility) is conserved. An increase in feedback level reduces the sensitivity in a specific frequency range (A), but creates a region of instability elsewhere as shown by the peak of normalized fragility in the middle. With a larger feedback strength, sensitivity in a specific frequency range may be significantly reduced, but fragility would be larger as a result (B; adapted from Yi et al, 2002). The mathematics behind this trade-off is well known, but is particularly well documented by Yi et al (2002) related to biological examples which they describe as follows: given the output of the system (Y(ω)) and disturbance (D(ω)), sensitivity function (S(ω)) can be defined as S(ω)=Y(ω)/D(ω). Let S 0(ω) be a sensitivity function of the open loop system, then we can define a base line sensitivity as Normalized sensitivity, hence fragility of the system, can be obtained by subtracting the sensitivity of the feedback system and the base line sensitivity of the system without feedback. This normalized sensitivity can be described by the equation: This inequality is essential as it implies that feedback control cannot improve overall sensitivity; it only improves sensitivity in one place in a trade-off for fragility elsewhere. In addition, theories that integrate trade-offs between robustness and fragility in a feedback system with a feedback channel of limited capacity have been developed recently, thus expanding the horizon of intrinsic trade-offs involved (Martins et al, 2004, 2007). It has been argued that the same trade-off may apply to biological systems, and so increased robustness against certain perturbations inevitably results in extreme fragility elsewhere (Csete and Doyle, 2002). At the same time, using NFB reduces the overall gain of the amplifier. Trade-offs between robustness and performance have also been thoroughly investigated. In amplifier design, it is well known that the gain-bandwidth product (GBWP) is conserved (C). In this case, the gain of the amplifier is considered as performance of the system, and the bandwidth corresponds to how broadly the system can ensure a certain level of insensitivity to disturbances on this circuit within the frequency region where the sensitivity is reduced by the feedback loop. For an amplifier with a gain of 1000 at 1 kHz (GBWP=1000 × 1=1000), the bandwidth can be extended using NFB to 100 kHz by reducing the gain to 10 (GBWP=100 × 10=1000). This high-frequency cut-off is extended due to the feedback loop.