A visual analytics approach for models of heterogeneous cell populations
 Jan Hasenauer^{1}Email author,
 Julian Heinrich^{2},
 Malgorzata Doszczak^{3},
 Peter Scheurich^{3},
 Daniel Weiskopf^{2} and
 Frank Allgöwer^{1}
DOI: 10.1186/1687415320124
© Hasenauer et al; licensee Springer. 2012
Received: 8 January 2012
Accepted: 31 May 2012
Published: 31 May 2012
Abstract
In recent years, cell population models have become increasingly common. In contrast to classic single cell models, population models allow for the study of celltocell variability, a crucial phenomenon in most populations of primary cells, cancer cells, and stem cells. Unfortunately, tools for indepth analysis of population models are still missing. This problem originates from the complexity of population models. Particularly important are methods to determine the source of heterogeneity (e.g., genetics or epigenetic differences) and to select potential (bio)markers. We propose an analysis based on visual analytics to tackle this problem. Our approach combines parallelcoordinates plots, used for a visual assessment of the highdimensional dependencies, and nonlinear support vector machines, for the quantification of effects. The method can be employed to study qualitative and quantitative differences among cells. To illustrate the different components, we perform a case study using the proapoptotic signal transduction pathway involved in cellular apoptosis.
1 Introduction
Cell populations are heterogeneous in terms of, e.g, cell age, cell cycle state, and protein abundance [1, 2]. This heterogeneity is ubiquitous, even in clonal population, and influences cell fate decisions [2, 3], such as cell death/proliferation [4–7]. Thus, to ultimately understand and control the behavior of populations, the key sources of celltocell variability have to be unraveled. Unfortunately, this is challenging due to experimental constraints. Most experimental systems and measurement devices only allow for the simultaneous assessment of a few cellular properties on a single cell basis. This prohibits the purely experimental analysis of processes which depend on many different cellular properties. Spencer et al. [5] have shown that the experimental limitations can be overcome partially using mathematical models.
To mathematically describe heterogeneous populations, agentbased models are used most frequently. Each agent provides a mechanistic description of the signal transduction within individual cells and thus of its behavior. In such a framework, variability can be modeled by either stochastic [8–10] or deterministic [4, 5, 11] differences among individual cells. The source of the former is the stochasticity of biochemical reactions, while the latter may arise from genetic and epigenetic differences, environmental heterogeneity, or slow dynamic processes (such as the cell cycle).
We focus on the deterministic differences among cells — also called extrinsic factors [12] — in populations of noninteracting cells. Those differences are commonly modeled by differential parameter values and initial conditions [5, 13]. Several methods exist to infer the distribution of parameters and initial conditions from experimental data [13–15] and to obtain quantitative, mechanistic models for cell populations. Unfortunately, the resulting agentbased models are in general highly complex. This complexity prevents the analysis of these models using common tools for dynamical systems [16], such as sensitivity and bifurcation analysis. To the best of our knowledge, for models of heterogeneous cell populations, no structured analysis approach is available. To study population models and to facilitate a modeldriven analysis of the heterogeneity, highly flexible methods are required which do not rely on an analytical analysis.
In this work, we propose two methods to fill this gap and to facilitate the analysis of population models. These methods — parallelcoordinates plots[17] and support vector (SV) machines[18–20] — are tools widely used for the analysis of highdimensional datasets. We outline how these tools can also be used to analyze complex models of heterogeneous cell populations, particularly addressing the question: "Which parameters cause the heterogeneity of the population's response?". Thereby, we extend our previous work [21] and consider qualitative heterogeneity among cells, in the context of cell fate decisions, as well as quantitative heterogeneity, such as the delay of a decision process.
We show that parallelcoordinates plots provide an easy tool to obtain a qualitative understanding of the system, whereas SV machines allow for assessing the performance of marker combinations quantitatively. Good markers are thereby defined as single cell parameters that facilitate a good prediction of the cell fate decision or the quantitative property under consideration of the individual cell. Furthermore, we show how the combination of parallelcoordinates plots and SV machines enables an indepth analysis of complex models using exploration techniques.
The article is structured as follows: In the section "Methods", the considered system class and problem are described in mathematical terms, the general idea is discussed, and the application of parallelcoordinates plots and SV machines is outlined. In the section "Results", we provide an exemplary application of our method to a model of the caspase cascade. The article is summarized in the section "Discussion".
2 Methodology
2.1 Models for heterogeneous cell populations and decision processes
2.1.1 Mechanistic population model
with state vector ${x}^{\left(i\right)}\left(t\right)\in {\mathbb{R}}_{+}^{n}$ and parameter vector ${\theta}^{\left(i\right)}\in {\mathbb{R}}_{+}^{q}$. The vector field $f:{\mathbb{R}}_{+}^{n}\times {\mathbb{R}}_{+}^{q}\to {\mathbb{R}}^{n}$ describing the cell dynamics is locally Lipschitz and the mapping ${x}_{0}:{\mathbb{R}}_{+}^{q}\to {\mathbb{R}}_{+}^{n}$ is continuously differentiable. The parameters θ^{(i)}may be kinetic constants, such as synthesis, degradation, or reaction rates.
This modeling framework is highly flexible and has been proven to be very useful, especially if fast signal transduction processes, such as cellular apoptosis, are investigated. For a more detailed introduction, we refer to the work of Spencer et al. [5] and Hasenauer et al. [14]. The properties of such populations of single cells have been studied by Spencer et al. [5], while Hasenauer et al. [14] have derived a partial differential equation model for the resulting population dynamics.
2.1.2 Qualitative and quantitative properties of the single cell response
Given the mathematical models introduced above, we study qualitative and quantitative properties of the single cell responses. Qualitative properties are defined as the outcome of a discrete decision processes, e.g., whether the state of a bistable system converges to one or the other stable steady state, or whether a certain concentration threshold is reached. In contrast, quantitative properties allow the assessment of small differences among cells, such as the time point when a particular threshold is exceeded.
To define single cell properties given the single cell trajectory x^{(i)}(·), the functionals F_{ φ } : ℓ^{1} → ℝ and F_{ δ } : ℓ^{1} → {1, +1} are introduced. The functional F_{ φ } is used to evaluate the quantitative property φ^{(i)}= F_{ φ } (x^{(i)}(·)) ∈ ℝ, while F_{ δ } determines the qualitative property δ^{(i)}= F_{ δ } (x^{(i)}(·)) ∈ {1, +1}.
and may be employed to achieve a quantitative understanding.
Note that the response x^{(i)}(·) of a cell merely depends on the cell's parameters θ^{(i)}, as the single cell model is deterministic. Therefore, the quantitative and qualitative properties of a single cell can be viewed as a function of the parameters, φ^{(i)}= φ(θ^{(i)}) and δ^{(i)}= δ(θ^{(i)}). Differences in the parameters — as they arise between different cells — may hence influence δ^{(i)}and φ^{(i)}, which determine cell fate decision and qualitative properties of the cells.
2.1.3 Response markers
is responsible for which aspect of the population heterogeneity. Mathematically, m is an index set and, e.g., for m = [2, 4] ^{T} only θ_{ m } = [θ_{2}, θ_{4}]^{T} is considered. The question of the relative importance of different parameters directly relates to the common problem of biomarker selection for stem cells and tumor cells, which is experimentally challenging.
If there exists a subset θ_{ m } of the parameters θ which allows for the reliable prediction of the response, not all sources for heterogeneity have to be assessed but only those associated to θ_{ m }. This enables a focusing of the model development, as well as the reduction of the experimental effort.
2.2 Analysis of population models using data analysis tools
cell properties of interest. These samples contain information about the dependency of φ and δ on the parameters θ, being analyzed in the following. To study the highdimensional mappings δ = δ(θ) and φ = φ(θ), parallelcoordinates plots will be employed. For the quantitative assessment of particular marker combinations SV machines will be applied. By combining both approaches it is possible to quickly gain an overview of important interrelations and quantify those.
2.2.1 Combining parallelcoordinates plots and SV machines to a visual analytics system
The proposed simulation databased analysis approach circumvents an analytical analysis of the system equations, which would be time consuming and could only be carried out by experts. However, the simulation databased approach creates the need for analyzing the large, highdimensional datasets, ${\mathcal{S}}_{\delta}$ and ${\mathcal{S}}_{\phi}$.
Besides an improved understanding of the model, results obtained during the analysis can be used to adapt the population model or to select additional experiments. This proposed framework, integrating interactive visualization with automated methods while allowing for a feedback to the actual system/model, thus incorporates important aspects of visual analytics [24].
2.3 Parallelcoordinates for the analysis of highdimensional data
As an Ndimensional data point is represented by a polyline intersecting axes at the respective values, parallelcoordinates greatly suffer from overplotting if many lines have to be drawn. In the resulting clutter of lines, interesting patterns might be hidden from the user. Exploiting the pointline duality, similar clutterreducing approaches as for Cartesian coordinates can be used, where a popular technique is to estimate the density of points (lines) and to render points (lines) transparently with blending enabled. Other approaches compute a continuous density [25] or estimate the overall density using density estimation techniques [26, 27]. In this work, both alpha and additive blending is used to visualize the parameter distribution in the different classes (φ^{(i)}= 1 and φ^{(i)}= 1), enabling a qualitative analysis of their multidimensional shape. An example of this alpha blending is shown in the section "Results".
For the analysis of a continuous variable, colormaps can be applied to the axis representing the dependent variable φ^{(i)}. Then, every polyline is rendered using a color according to φ^{(i)}, such that its value can be visually determined over the whole plot. The overall distribution of colors can then be used in conjunction with the shape of lines to analyze the dependency of independent variables from the dependent. Again, overplotting can become an issue for large datasets, such that a separation in few classes and a separate visualization of those might be more informative (see example in section "Results").
2.4 SV machines for the quantification of marker performance
Given a basic understanding of the importance of the parameters and a potential marker combination θ_{ m }, a quantitative assessment of the predictive power of θ_{ m } is desirable. To achieve this, the samples ${\mathcal{S}}_{\delta}$ and ${\mathcal{S}}_{\phi}$ are analyzed employing nonlinear SV classification and nonlinear SV regression, respectively. SV classification allows for the study of decision processes, while SV regression enables the analysis of quantitative system properties.
The performance of SV machines — which might be interpreted as databased predictors — provides a measure for the quality of the marker combination θ_{ m }. If a SV machine using only θ_{ m } provides good predictions for a decision process which depends on θ, then this means that θ_{ m } carries the most important information. This will be discussed in more detail in the following.
2.4.1 SV classification
is solved, in which w and b denote the normal vector of the separating hyperplane and its offset, respectively. The objective function combines a misclassification penalty, ${\sum}_{i=1}^{S}{\xi}_{i}$, and a margin maximization, $\frac{1}{2}{w}^{\text{T}}w$. The weighting of the different terms can be influenced via C. The constraints are that all data points $\mathrm{\Phi}\left({\theta}_{\mathbf{m}}^{\left(i\right)},{\delta}^{\left(i\right)}\right)$ are correctly classified within a certain error margin ξ_{ i } .
Assuming that the training set ${\mathcal{S}}_{\delta}$ is large, the predictive power of this predictor will be high — meaning that $\widehat{\delta}\left({\theta}_{\mathbf{m}}^{\left(i\right)}\right)=\delta \left({\theta}^{\left(i\right)}\right)$ for most θ^{(i)}~ Θ(θ) — if and only if the selected markers θ_{ m } are informative. This allows the quantitative assessment of the informativeness of the markers θ_{ m } using the SV classifier.
Therefore, a second sample ${\mathcal{S}}_{\delta}^{\prime}$ is computed which was not used to train the SV classifier, avoiding overfitting. For this sample, the predictor ${\widehat{\delta}}^{\left(i\right)}=\widehat{\delta}\left({\theta}_{\mathbf{m}}^{\left(i\right)}\right)$ is evaluated. These results are used to calculate the percentage of true positive classifications TP $\left({\delta}^{\left(i\right)}=1\wedge {\widehat{\delta}}^{\left(i\right)}=1\right)$ and false positive classifications FP $\left({\delta}^{\left(i\right)}=0\wedge {\widehat{\delta}}^{\left(i\right)}=1\right)$ achieved by the SV classifier. TP and FP provide information about the predictability of the outcome for θ^{(i)}using solely ${\theta}_{\mathbf{m}}^{\left(i\right)}$. Thus, the marker quality can be assessed via TP and FP. If a lowdimensional m exists that provides TP ≈ 1 and FP ≈ 0, the parameters θ_{ m } dominate the decision process and are good markers. For a quantification of this effect, the classification performance can be analyzed in receiveroperating characteristic (ROC) space [28].
2.4.2 SV regression
Aside from the penalization of prediction error, ${\sum}_{i=1}^{N}\left({\xi}_{i}+{\xi}_{i}^{*}\right)$, flatness and a unique solution is ensured using $\frac{1}{2}{w}^{\text{T}}w$. The tradeoff between those two is determined by the constant C > 0.
The optimal solution of (6) for w and b provides the optimal predictor (5) with respect to the loss function and kernel. This predictor ${\widehat{\phi}}^{\left(i\right)}=\widehat{\phi}\left({\theta}_{\mathbf{m}}^{\left(i\right)}\right)$ is applied to a second sample ${\mathcal{S}}_{\phi}^{\prime}$ to compute ${\widehat{\phi}}^{\left(i\right)}$, a prediction for φ^{(i)}. Employing φ^{(i)}and ${\widehat{\phi}}^{\left(i\right)}$ the marker combination m might be evaluated based on the relative prediction errors, ${e}_{\mathbf{m}}^{\left(i\right)}=\left\frac{{\phi}^{\left(i\right)}\widehat{\phi}\left({\theta}_{m}^{\left(i\right)}\right)}{{\phi}^{\left(i\right)}}\right$. Using ${e}_{\mathbf{m}}^{\left(i\right)}$, the prediction powers of different marker combinations can be assessed and compared using, e.g., the mean error $\frac{1}{N}{\sum}_{i=1}^{N}{e}_{\mathbf{m}}^{\left(i\right)}$. If the mean prediction error achieved by a marker combination is small, the parameters ${\theta}_{\mathbf{m}}^{\left(i\right)}$ carry most of the information about φ^{(i)}, and hence are suitable markers. In some situations, the information about the mean prediction error may be complemented by detailed information about the error statistics, ${\left\{{e}_{\mathbf{m}}^{\left(i\right)}\right\}}_{i=1}^{N}$. These statistics may be visualized using, for instance, box plots or histograms, and provide additional insight, e.g., in the structure of the error (short vs. longtailed distributions) and the potential causes.
Note that the performance and predictive power of SV machines strongly depend on the available training set. For the analysis performed, we ensured that the training sets are large enough and that a further increase in its size does not result in a significant improvement of the predictors. This is, in most situations where SV machines and SV regressions are used, impossible for data analysis, as the measurement devices are limited. However, in this work we study the problem of model analysis. The size of the dataset can be increased arbitrarily by repeated simulation of the model. Besides the size of the dataset, the parameters of the SV classification and SV regression are tuned to allow for a fair comparison between the marker combinations. With this and the existence of sophisticated SV machine toolboxes (e.g., LIBSVM [31]), the observed difference between marker combinations can be assumed to be due to the predictive power of the markers.
Summing up, SV machines allow for the derivation of predictors for qualitative and quantitative properties. These predictors can be used to assess the information content of a subset m of the parameters about the respective properties, thereby facilitating the assessment of a quantitative evaluation of the predictive power of θ_{ m }. For further details about SV machines we refer to [18–20, 30, 31] and references therein.
3 Results
3.1 Model for heterogeneous cancer cell population
To illustrate the proposed visual analytics framework, a model of the proapoptotic signaling is analyzed. Proapoptotic signaling is involved in the process of apoptosis [32–34], also called programmed cell death. Apoptosis is an important physiological process to remove infected, malfunctioning, or no longer needed cells from a multicellular organism. The apoptotic signaling pathways converge at the caspase cascade [32], where initiator caspases (e.g., caspase 8) and effector caspases (e.g., caspase 3) are activated. If the activity of effector caspases exceeds a certain threshold, apoptosis is induced.

From flow cytometric experiments, it is known that the amount of caspase 8 (C8), caspase 3 (C3), caspases 8 and 10associated RING protein (CARP), and inhibitor of apoptosis protein (IAP) is different among individual cells. The differences are modeled by differences in synthesis rates (k_{8}, k_{9}, k_{10}, and k_{12}) among individual cells. The distribution of k_{8}, k_{9}, k_{10}, and k_{12} within the population is modeled as lognormal distribution, with mean as published by Eissing et al. [35] and a coefficient of variation of 0.4 (own unpublished data). The initial conditions of C8, C3, CARP, and IAP are set to their steady state values.

Similar to the original publication [35], the activation of the caspase cascade is modeled by a nonzero initial condition of active caspase 8, C8a(0). In the population, C8a(0) is lognormally distributed with a median of 4,000 molecules per cells and a coefficient of variation of 0.4. The variation of C8a(0) accounts for variability upstream of the caspase cascade.
The binding affinities and kinetic rates are the same for all cells. For the numerical values, we refer to the article of Eissing et al. [35].
Given this model of the heterogeneous cell population, we analyzed (i) how the decision whether or not a cell undergoes apoptosis during the first 12 hours and (ii) how the time of cell death T_{ d } is influenced by the cell's parameters θ = [C8a(0), k_{8}, k_{9}, k_{10}, k_{12}]^{T}. This yields two variables of interest: δ (= +1 ⇒ cell survived; = 1 ⇒ cell died) providing the outcome of the decision process; and φ (= T_{ d } ) providing the time of apoptosis commitment. As indicator for apoptosis, the amount of active caspase 3 (C3a) is used. If more than 5,000 copies of C3a are present in a cell, this cell is assumed to undergo apoptosis within 10 minutes, defining the time of cell death T_{ d } . The functionals associated to the considered δ and φ are similar to (1) and (2), respectively. In the remainder, we search for a lowerdimensional subset of the parameters θ which provide good markers for cell death and survival as well as the time of cell death.
3.2 Parallelcoordinates plot establishes importance of C3 and IAP concentration for cell fate decision
From Figure 5, it is apparent that the second and fourth parameters (θ_{ m } = [k_{8}, k_{10}]^{T}) provide a reasonable separation between the classes (red = dead, blue = alive). Most of the surviving cells have high values of k_{8} and low values of k_{10}, which corresponds to a high IAP expression and a low C3 expression, respectively. Although the other parameters also influence the process, their influence seems to be minor.
3.3 SV classification proves that C3 and IAP expression are the best markers for the cell fate decision
Area under the ROC curve for different marker combinations
C8a(0)  k _{8}  k _{9}  k _{10}  k _{12}  

C8a(0)  0.569  0.747  0.626  0.808  0.690 
k _{ 8 }  0.747  0.736  0.760  0.898  0.800 
k _{ 9 }  0.626  0.760  0.603  0.822  0.709 
k _{ 10 }  0.808  0.898  0.822  0.795  0.858 
k _{ 12 }  0.690  0.800  0.709  0.858  0.676 
The markers θ_{ m } = k_{8} and θ_{ m } = k_{10} outperform all other single markers and marker pairs. In addition, the marker vector θ_{ m } = [k_{8}, k_{10}]^{T} outperforms all other combinations in terms of the area under the ROC curve. Some other combinations result in more than 50% false positive classifications (see Figure 6B). Of course, extending the marker vector, e.g., by adding k_{12}, results in further improvement.
3.4 Parallelcoordinates plots show a complex dependency of the time of death on the parameters
After the analysis of the decision process, we study the dependency of time of cell death T_{ d } on the parameters. The time of cell death T_{ d } is a quantitative property and can take any positive value, therefore an alternative visualization has to be used. One approach would be to use a different color for each line in parallelcoordinates, depending on ${T}_{d}^{\left(i\right)}$. Unfortunately, this approach suffers from heavy overplotting, which is why the data was split into three classes and separate plots were created for each class.
3.5 SV regression reveals ubiquitous importance of IAP an C3 expression levels
To quantify the predictive power of different marker combinations with respect to T_{ d } , we employ the SV regression based approach introduced in "Methods". As a performance measure, the relative prediction error $\left\frac{{T}_{d}^{\left(i\right)}{\widehat{T}}_{d}^{\left(i\right)}}{{T}_{d}^{\left(i\right)}}\right$, their ${\widehat{T}}_{d}^{\left(i\right)}$ is the prediction of the SV machine. Details on the implementation may be found in "Methods".
4 Conclusion
4.1 Visual analytics enable an indepth analysis of complex population models
In this article, a novel explorative approach has been presented to determine markers for decision processes in heterogeneous populations. It has been shown that methods used for data analysis can also be employed to gain insight into complex models, where common analytical methods seem to reach their limits. Especially, the potential of parallelcoordinates plots and support vector machines has been illustrated. While the first allows for the study of large, highdimensional datasets and the selection of potential markers, the latter can provide a quantitative assessment of their predictive power. Using both methods, the source of qualitative and quantitative celltocell variability may be unraveled.
This article provides a case study evaluating the potential of combining visualization and automated methods for the assessment of complex system models. The considered system class is only one example and the proposed framework can be generalized easily to other systems and questions.
4.2 Analysis of heterogeneous cell population allows for novel insight
We have illustrated the proposed visual analytics approach by analyzing a cell population model for proapoptotic signaling, which plays an essential role in programmed cell death. We have studied the cell fate decision as well as the time of cell death. These properties were analyzed before (see, e.g., [5]) in a purely qualitative way and without the tools proposed in this work.
Our study shows that parallelcoordinates plots are a proper tools to determine potential markers. The predictive power of these markers can then be quantified using SV machines. In this study, the markers we found agree well with those found in the literature. In particular, the important role of IAP—also called XIAP—for cell death commitment is outlined in several publications [39, 41]. While C3 abundance is known to be important [39], our analysis suggests that the amount of available C3 could be even more important than expected.
In addition, our analysis indicates that, under normal conditions, the time of cell death strongly depends on all parameters, which has been hypothesized earlier [5]. Only under altered conditions, e.g., a strongly increased initial amount of C8a(0), some parameters become more important than others (results not shown). This is again in agreement with the results of Spencer et al. [5]. Furthermore, this finding of a varying importance of parameters depending on the experimental setup, provides hints for possible future experiments. Thus, our visual analytics approach we propose also provides helpful feedback for model validation and development.
4.3 Outlook
In this work, we have proposed a method to determine decision markers for given models. However, all model possess uncertainties, rendering an uncertaintyaware analysis crucial. Therefore, a workflow including model development, parameter estimation, uncertainty analysis, and marker prediction has to be established. This requires improved modeling and parameter estimation tools, as well as methods to evaluate the uncertainty of the marker prediction, arising from model uncertainties.
Given such a workflow, beyond the analysis of models, our analysis tools might also be used to guide the search for biomarkers. This is possible as our methods allows for the assessment of the importance of any parameters which are different among cells of the population. Among others, the importance of common biomarkes, e.g., expression levels and transcription factor/protein abundance, may be determined based on a model of the population. This is much in the same way as the target selection using sensitivity analysis of single cell models based on ordinary differential equations (see, e.g., [42]). However, the marker selection requires population models, as differences between cells have to be considered, and is therefore more challenging.
Methods
Software
The model of the heterogeneous cell population was implemented in MATLAB using the SBtoolbox2[43]. For the SV classification and the SV regression, the LIBSVM toolbox for MATLAB is employed [31]. The visualization software for parallelcoordinates was implemented in C++ using the Qt library Version 4.8.0 and OpenGL.
Numerics
For the SV classification and SV regression, we employed as kernels radial basis function with γ = 0.25. The SV regression parameter which defines the interval of insensitivity was set to 0.01. All remaining parameters are set to the default constants, see LIBSVM manual. To improve the performance of the SV machines, we applied a logtransformation to the parameters θ.
Declarations
Acknowledgements
The authors acknowledge financial support from the German Research Foundation within the Cluster of Excellence in Simulation Technology (EXC 310/1) at the University of Stuttgart, from the German Federal Ministry of Education and Research (BMBF) within the FORSYSPartner program (grant nr. 0315280A and D), and from Center Systems Biology at the University of Stuttgart.
Authors’ Affiliations
References
 Avery S: Microbial cell individuality and the underlying sources of heterogeneity. Nat Rev Microbiol 2006, 4: 577587. 10.1038/nrmicro1460View Article
 Snijder B, Pelkmans L: Origins of regulated celltocell variability. Nat Rev Mol Cell Biol 2011, 12(2):11925. 10.1038/nrm3044View Article
 Eldar A, Elowitz M: Functional roles for noise in genetic circuits. Nature 2010, 467(9):17. 10.1038/nj73191
 Albeck J, Burke J, Spencer S, Lauffenburger D, Sorger P: Modeling a snapaction, variabledelay switch controlling extrinsic cell death. PLoS Biol 2008, 6(12):28312852.View Article
 Spencer S, Gaudet S, Albeck J, Burke J, Sorger P: Nongenetic origins of celltocell variability in TRAILinduced apoptosis. Nature 2009, 459(7245):428433. 10.1038/nature08012View Article
 Niepel M, Spencer S, Sorger P: Nongenetic celltocell variability and the consequences for pharmacology. Cur Opin Biotechnol 2009, 13(56):556561.
 Singh D, Ku CJ, Wichaidit C, Steininger R, Wu L, Altschuler S: Patterns of basal signaling heterogeneity can distinguish cellular populations with different drug sensitivities. Mol Syst Biol 2010, 6(369):110.
 Paulsson J: Models of stochastic gene expression. Phys Life Rev 2005, 2(2):157175. 10.1016/j.plrev.2005.03.003View Article
 Glauche I, Moore K, Thielecke L, Horn K, Loeffler M, Roeder I: Stem cell proliferation and quiescence — two sides of the same coin. PLoS Comput Biol 2009, 5(7):e1000447. [http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1000447] 10.1371/journal.pcbi.1000447MathSciNetView Article
 Huh D, Paulsson J: Nongenetic heterogeneity from stochastic partitioning at cell division. Nat Gen 2011, 43(2):95102. 10.1038/ng.729View Article
 Glauche I, Thielecke L, Roeder I: Cellular aging leads to functional heterogeneity of hematopoietic stem cells: a modeling perspective. Aging Cell 2011, 10: 457465. 10.1111/j.14749726.2011.00692.xView Article
 Swain P, Elowitz M, Siggia E: Intrinsic and extrinsic contributions to stochasticity in gene expression. Proc Natl Acad Sci USA 2002, 99(20):1279512800. 10.1073/pnas.162041399View Article
 Hasenauer J, Waldherr S, Doszczak M, Radde N, Scheurich P, Allgöwer F: Identification of models of heterogeneous cell populations from population snapshot data. BMC Bioinf 2011, 12: 125. 10.1186/1471210512125View Article
 Hasenauer J, Waldherr S, Doszczak M, Radde N, Scheurich P, Allgöwer F: Analysis of heterogeneous cell populations: a densitybased modeling and identification framework. J Process Control 2011, 21(10):14171425. 10.1016/j.jprocont.2011.06.020View Article
 Koeppl H, Zechner C, Ganguly A, Pelet S, Peter M: Accounting for extrinsic variability in the estimation of stochastic rate constants. Int J Robust Nonlinear Control 2012, 22(10):121.MathSciNetView Article
 Guckenheimer J, Holmes P: Nonlinear Oscillations Dynamical Systems and Bifurcations of Vector Fields. In Appl Math Sci. Volume 42. SpringerVerlag, New York; 1983.
 Inselberg A, Dimsdale B: Parallel coordinates: A tool for visualizing multidimensional geometry. In Proc of IEEE Visualization. Edited by: Kaufman A. Los Alamitos, California, IEEE Computer Society Press; 1990:361378.
 Vapnik V: The Nature of Statistical Learning Theory. Springer, New York; 1995.MATHView Article
 Cristianini N, ShaweTaylor J: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge; 2000.
 Ivanciuc O: Applications of Support Vector Machines in Chemistry. In Reviews in Computational Chemistry. Volume 23. Edited by: Lipkowitz KB, Cindari TR. WileyVCH, Weinheim; 2007.View Article
 Hasenauer J, Heinrich J, Doszczak M, Scheurich P, Weiskopf D, Allgöwer F: Visualization methods and support vector machines as tools for determining markers in models of heterogeneous populations: Proapoptotic signaling as a case study. In Proc of Workshop Comp Syst Biol. Edited by: Koeppl H, Aćimović J, Kesselin J, MäkiMarttunen T. Zürich, Switzerland; 2011:6164. (TICSP series # 57)
 Novak B, Pataki Z, Ciliberto A, Tyson J: Mathematical model of the cell division cycle of fission yeast. Chaos 2001, 11: 277286. 10.1063/1.1345725MATHView Article
 Pan J, Chen RH: Spindle checkpoint regulates Cdc20p stability in Saccharomyces cerevisiae. Genes Dev 2004, 18: 14391451. 10.1101/gad.1184204View Article
 Thomas J, Cook K: A visual analytics agenda. IEEE Comput Graph Appl 2006, 26: 1013.View Article
 Heinrich J, Weiskopf D: Continuous parallel coordinates. IEEE Trans Vis Comput Graph 2009, 15(6):15311538.View Article
 Feng D, Kwock L, Lee Y, Taylor R: Matching visual saliency to confidence in plots of uncertain data. IEEE Trans Vis Comput Graph 2010, 16(6):980989.View Article
 Heinrich J, Bachthaler S, Weiskopf D: Progressive splatting of continuous scatterplots and parallel coordinates. Comput Graph Forum 2011, 30(3):653662. 10.1111/j.14678659.2011.01914.xView Article
 Zweig M, Campbell G: Receiveroperating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem 1993, 39(8):561577.
 Schölkopf B, Sung K, Burges C, Girosi F, Niyogi P, Poggio T, Vapnik V: Comparing support vector machines with Gaussian kernels to radial basis function classifiers. IEEE Trans Signal Process 1997, 45: 27582765. 10.1109/78.650102View Article
 Smola A, Schölkopf B: A tutorial on support vector regression. Stat Comp 2004, 14(3):199222.View Article
 Chang CC, Lin CJ: LIBSVM: A library for support vector machines. IEEE/ACM Trans Intell Syst Tech 2011, 2(3):127.View Article
 Wajant H, Pfizenmaier K, Scheurich P: Tumor necrosis factor signaling. Cell Death Diff 2003, 10: 4565. 10.1038/sj.cdd.4401189View Article
 Gewirtz D, Holt S, Grant S (Eds): Cancer Drug Discovery and Development In Apoptosis, Senescence, and Cancer. 2nd edition. Humana Press, Totowa; 2007.View Article
 Spencer S, Sorger P: Measuring and modeling apoptosis in single cells. Cell 2011, 144(6):926939. 10.1016/j.cell.2011.03.002View Article
 Eissing T, Conzelmann H, Gilles E, Allgöwer F, Bullinger E, Scheurich P: Bistability analyses of a caspase activation model for receptorinduced apoptosis. J Biol Chem 2004, 279(35):3689236897. 10.1074/jbc.M404893200View Article
 Albeck J, Burke J, Aldridge B, Zhang M, Lau enburger D, Sorger P: Quantitative analysis of pathways controlling extrinsic apoptosis in single cells. Mol Cell 2008, 30: 1125. 10.1016/j.molcel.2008.02.012View Article
 Eissing T, Chaves M, Allgöwer F: Live and let die — a systems biology view on cell death. Comput Chem Eng 2009, 33(3):583589. 10.1016/j.compchemeng.2008.10.014View Article
 Schlatter R, Schmich K, Vizcarra I, Scheurich P, Sauter T, Borner C, Ederer M, Merfort I, Sawodny O: ON/OFF and beyond — a boolean model of apoptosis. PLoS Comput Biol 2009, 5(12):113.View Article
 Rehm M, Huber H, Dussmann H, Prehn J: Systems analysis of effector caspase activation and its control by Xlinked inhibitor of apoptosis protein. EMBO J 2006, 25(18):43384349. 10.1038/sj.emboj.7601295View Article
 Würstle M, Laussmann M, Rehm M: The caspase8 dimerisation/dissociation balance is a highly potent regulator of caspase8, 3, 6 signalling. J Biol Chem 2010, 285(43):3320933218. 10.1074/jbc.M110.113860View Article
 Jost P, Grabow S, Gray D, McKenzie M, Nachbur U, Huang D, Bouillet P, Thomas H, Borner C, Silke J, Strasser A, Kaufmann T: XIAP discriminates between type I and type II FASinduced apoptosis. Nature 2009, 460(7258):10351039. 10.1038/nature08229View Article
 Schöberl B, Pace E, Fitzgerald J, Harms B, Xu L, Nie L, Linggi B, Kalra A, Paragas V, Bukhalid R, Grantcharova V, Kohli N, West K, Leszczyniecka M, Feldhaus M, Kudla A, Nielsen U: Therapeutically targeting ErbB3: a key node in ligandinduced activation of the ErbB receptorPI3K axis. Sci Signal 2009, 2(77):ra31. 10.1126/scisignal.2000352
 Schmidt H, Jirstrand M: Systems biology toolbox for MATLAB: a computational platform for research in systems biology. Bioinf 2006, 22(4):514515. 10.1093/bioinformatics/bti799View Article
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.