13 Feature Comparison
This method allows users to compare the distributions of a single numerical feature across groups. It can help users spot overarching trends between distributions, identify within-distribution shape-related characteristics such as skewness or bimodality that might signal sub-populations, and diagnose potential outliers.
13.2 Separate by
Separate by takes effect before Color by. Separate by divides the plot into sections separated by a dashed line, with each section labeled by one of the available categories of the selected categorical feature after filtering. The selected feature of Separate by is automatically removed from available features in Color by. The section order is determined by the order of the categories which are sorted numeric-alphabetically.
Then, visual channels (color, shape, opacity) and effect size annotation are applied within each section. For example, the visualization below was separated by cell_line, with each section labeled by the bold text of a cell line. Within each section, the x-axis rendered the Color by categories (treatments) so colors were consistent across sections. Effect size calculation and annotation (only effect size \(> 0.5\) is shown) were performed within each section.
If Separate by is left blank and both cell_line and treatment are selected in Color by, then colors are assigned to each cell line and treatment combination, and effect sizes are calculated across all combinations of group pairs (only pairs with effect size \(> 0.5\) are shown).
13.3 Comparison Widget
Statistical tests and effect size calculations are performed on the selected comparison pairs. FLIM Playground uses the groups on the x-axis populated by Color by to create comparison groups, starting from the leftmost group. For example, if there are 5 groups, then \(\binom{5}{2} = 10\) comparisons will be created. The number of groups can get combinatorially large and many of the comparisons are not meaningful. Therefore, two widgets are provided to filter the comparison groups:
- A selection widget that shows all possible comparisons by default and users can deselect groups they do not need.
- A threshold by effect size widget that filters out comparisons below the threshold.
13.4 Effect Size
Complementary to statistical tests, which address “is there a difference?”, effect size metrics answer “how large is the difference?”, enabling cross-study comparison. FLIM Playground offers two effect size calculation methods, Glass’s Delta and Cohen’s d, each can be in mean or median form.
13.4.1 Glass’s Delta
Mean (\(\bar{X}\)) form: \[ \Delta_G = \frac{\bar{X}_{\text{treatment}} - \bar{X}_{\text{control}}} {s_{\text{control}}},\qquad s_{\text{control}} = \sqrt{\frac{1}{\,n_c-1\,}\sum_{i=1}^{n_c} \bigl(X_{control_i}-\bar{X}_{\text{control}}\bigr)^{2}} \]
Median (\(\tilde{X}\)) form: \[ \widetilde{\Delta}_G = \frac{\tilde{X}_{\text{treatment}} - \tilde{X}_{\text{control}}} {\operatorname{MAD}_{\text{control}}},\qquad \operatorname{MAD}_{\text{control}} = 1.4826 \times \operatorname{median}\,\bigl\lvert X_{control_i}-\tilde{X}_{\text{control}} \bigr\rvert, \qquad i = 1, \ldots, n_c \]
\(\operatorname{MAD}\) stands for median absolute deviation. In order to make \(\operatorname{MAD}\) asymptotically consistent with the standard deviation under normality, it is scaled by \(C = 1 / \Phi^{-1}(0.75) \approx 1.4826\). Implementation-wise, it uses median_abs_deviation from scipy.stats, and scale option is set to "normal" to account for the constant multiplier \(C\). THE MEDIAN FORM of both Glass’s Delta and Cohen’s d IS NOT REPORTED IN THE LITERATURE.
Which group is treatment and which is control matters! From the formula of \(\Delta_G\), switching the order not only flips the sign of the result, but also changes the denominator, which is \({s_{\text{control}}}\), the standard deviation of the control. Currently, FLIM Playground treats the group on the left as control and the group on the right as treatment (the negative \(\Delta_G\) below implies this). FLIM Playground now supports user-customizable x-axis order so that users can arrange the groups to make sure the control group is on the left. Or users can use Cohen’s d that does not assume the treatment-control structure.
To keep the point-based visualization style that supports interactive hover while showing the violin-plot-like distribution, Sina plot is used: it uses gaussian_kde from scikit-learn to make sure the width of the point distribution is proportional to the kernel density.
13.4.2 Cohen’s d
Mean (\(\bar{X}\)) form:
\[ d = \frac{\bar{X}_{1}-\bar{X}_{2}} {s_p},\qquad s_p = \sqrt{\frac{(n_1-1)s_1^{2} + (n_2-1)s_2^{2}} {n_1 + n_2 - 2}} \]
Median (\(\tilde{X}\)) form:
\[ \tilde{d} = \frac{\tilde{X}_{1}-\tilde{X}_{2}} {s_{\tilde{p}}},\qquad s_{\tilde{p}} = \sqrt{\frac{(n_1-1)\operatorname{MAD}_{1}^{2} + (n_2-1)\operatorname{MAD}_{2}^{2}} {n_1 + n_2 - 2}} \]
13.5 Statistical Test
Statistical tests are performed on the selected comparison pairs that exceed the effect size threshold (if users choose to calculate effect sizes). The statistical tests supported are:
- Independent t-test (student’s t-test, for equal variances)
- Welch’s t-test (for unequal variances)
FLIM Playground uses scipy.stats.ttest_ind to perform the independent t-test and scipy.stats.ttest_ind with equal_var=False to perform the Welch’s t-test.
13.6 Comparison Annotation
For each selected comparison pair that exceeds the effect size threshold (if any), the statistical significance, effect size, or both are annotated on the plot. It selects the lowest eligible position (over all points in both groups and over existing annotations) to annotate. If both statistical significance and effect size calculation are performed, the annotation is shown in the format of effect size followed by p-value. If only one is performed, then only the performed method’s result is shown.
