21 Classification
A machine learning classifier learns patterns from categorized data and then predicts the category of new, unseen data. This module provides a set of machine learning classifiers and interactive widgets to help users perform classification.
All classifiers use scikit-learn, with a fixed random seed of 42 for reproducibility. The app evaluates predictions on a held-out test set and reports performance metrics, a confusion matrix, and ROC curves.
An overview of the pipeline:
- Choose one or more numerical features on the left.
- Choose the classifier, train-test split, and hyperparameters.
- Filter the dataset.
- Choose categorical features to form classes.
- Choose how to handle class imbalance.
- Choose the classes to be classified.
- Look at the performance metrics, confusion matrix, and ROC curve.
- Compare the results using the metrics that matter for your classification task, then export the analysis to reproduce it.
SVM and Logistic Regression standardize features with a z-score within their training pipeline; test observations use that training-fitted transformation. Random Forest and Gradient Boosting use the original feature scale. Classification does not automatically remove rows with missing selected measurements, so a classifier that cannot accept them reports an error. Clean those measurements or use the numerical filters to select complete observations.
21.2 Method-specific Components
- Choose a Classifier and Train size (proportion of training data) on the left. The slider ranges from 0.5 to 0.9 in steps of 0.1, with a default of 0.7. The remaining observations form the held-out test set. Available classifiers are:
Random ForestGradient Boosting- Support Vector Machine (
SVM) Logistic Regression
The train-test split is stratified to preserve class proportions. The app does not create a separate validation set for choosing model hyperparameters; those are set with the controls below the classifier. Optional threshold tuning uses cross-validation within the training data.
Both the train-test split and the cross-validation used in threshold tuning operate on individual rows (i.e. single cells), stratified by class. They do not group rows by field of view, so cells segmented from the same image can end up in both the training set and the test set.
Cells from one image are usually more similar to each other than to cells from a different image, because they share a field of view, an acquisition session, and often a well. When such cells appear on both sides of the split, the reported metrics can look better than the classifier’s true performance on a genuinely new image. Keep this in mind when interpreting the performance metrics, particularly when the dataset contains many cells drawn from relatively few fields of view.
- Classify by forms target classes from one or more categorical columns. With multiple columns, each distinct combination becomes a class, with values joined by underscores. Classification requires at least one categorical column with two distinct values after filtering.
- Select a way to classify chooses the classes included in a run. With two classes, the app offers their binary comparison. With more classes, it offers combinations of two or more classes, including the full multiclass problem, and a
class VS the restoption for each class. A one-versus-rest choice combines all other classes intothe rest. If the number of combinations is too large, the app asks you to narrow the categories or filters.
21.2.1 Classifier Hyperparameters
The main hyperparameters appear below the classifier; open Advanced settings for additional controls. The tables below describe parameters using three roles:
Model-Form/Capacity: defines the hypothesis family or expressive power.Regularization: constrains model complexity to reduce overfitting.Optimization: affects training convergence and runtime behavior.
For Random Forest, max_depth (0=None) uses 0 for an unrestricted depth. SVM shows degree only for the polynomial kernel and coef0 for polynomial or sigmoid kernels. Logistic Regression updates its regularization choices when the solver changes and shows l1_ratio for saga with elasticnet.
Random Forest
| Param | Role | Effect | Priority |
|---|---|---|---|
n_estimators |
Model-Form/Capacity | More trees increase ensemble capacity; gains usually show diminishing returns after enough trees. | High |
max_depth |
Regularization | Caps tree depth to reduce overfitting; also directly limits capacity. | High |
min_samples_split |
Regularization | Requires more samples before splitting, reducing fragmentation on small partitions. | Medium |
min_samples_leaf |
Regularization | Enforces minimum leaf size, smoothing predictions. | Medium |
max_features |
Regularization | Fewer features per split increase tree diversity and reduce variance/overfitting. | Medium-High |
Gradient Boosting
| Param | Role | Effect | Priority |
|---|---|---|---|
n_estimators |
Model-Form/Capacity | More stages increase additive model capacity; strongly interacts with learning_rate. |
High |
learning_rate |
Regularization | Shrinks each stage contribution; lower values typically need more trees and often generalize better (also affects optimization dynamics). | High |
max_depth |
Regularization | Limits base learner complexity; also limits capacity. | High |
subsample |
Regularization | < 1.0 introduces stochastic boosting, reducing variance and overfitting risk. |
High |
Support Vector Machine (SVM)
| Param | Role | Effect | Priority |
|---|---|---|---|
C |
Regularization | Higher C penalizes training errors more, which usually weakens regularization. |
High |
kernel |
Model-Form/Capacity | Chooses hypothesis family (linear vs non-linear kernels). |
High |
gamma |
Model-Form/Capacity | Controls influence radius of samples for rbf/poly/sigmoid; higher values create tighter boundaries. |
High |
degree |
Model-Form/Capacity | Polynomial order for poly kernel; higher degree increases flexibility. |
Medium-High |
coef0 |
Model-Form/Capacity | Kernel offset for poly/sigmoid, shifting boundary behavior. |
Medium |
tol |
Optimization | Convergence tolerance; mostly affects runtime/termination behavior. | Low |
Logistic Regression
| Param | Role | Effect | Priority |
|---|---|---|---|
regularization |
Regularization | Selects regularization type; available options depend on solver (see table below). |
High |
C |
Regularization | Inverse regularization strength; higher C means weaker regularization. |
High |
l1_ratio |
Regularization | Balances l1 vs l2 when solver='saga' and regularization='elasticnet'. |
Medium |
fit_intercept |
Regularization | Disabling intercept is a mild structural constraint. | Low |
solver |
Optimization | Optimization algorithm; affects convergence speed and which regularization options are available. | Low-Medium |
max_iter |
Optimization | Iteration budget; too low can stop before convergence. | Low-Medium |
tol |
Optimization | Convergence threshold; mainly runtime-sensitive. | Low |
Regularization options by solver:
| Solver | Available regularization |
|---|---|
lbfgs |
l2, none |
liblinear |
l1, l2 |
newton-cg |
l2, none |
newton-cholesky |
l2, none |
sag |
l2, none |
saga |
l1, l2, elasticnet, none |
21.2.2 Class Imbalance Handling
Class imbalance occurs when some classes have far fewer observations than others, which can bias a classifier toward the majority class. The app offers three ways to address it, all applied only to the training set to prevent data leakage.
- Sampling methods:
None: stratified random sampling of the data on classification classes, regardless of the class size. It is implemented usingtrain_test_splitfromsklearn.model_selection.Undersampling: All classes are downsampled to the size of the smallest class. It is implemented usingRandomUnderSamplerfromimbalanced-learn.Oversampling: All classes are upsampled to the size of the largest class by randomly duplicating samples from minority classes. It is implemented usingRandomOverSamplerfromimbalanced-learn.
- Class weights:
None: no class weights.Balanced: the class weights are inversely proportional to the class size (support). Note: Gradient Boosting does not support class weights in the scikit-learn implementation, and thus this option is not available for it.
- Threshold tuning: Adjusts the decision rule that converts predicted probabilities into class labels, while leaving the probability estimates unchanged.
None: uses the classifier’s default decision rule without tuning a threshold.Balanced Accuracy: the threshold is set to maximize the balanced accuracy score, which is the unweighted average of each class’s recall (i.e., the minority class’s recall is as important as the majority class’s recall).F1 Score: the threshold is set to maximize the unweighted average of each class’s F1 score.
K-fold cross-validation for the probability model: After any selected resampling, stratified k-fold cross-validation (default k=5) divides the training data into k subsets. Each fold trains a probability model on k-1 subsets and predicts the remaining subset. Together, the folds provide one validation prediction for each row used in cross-validation. The held-out test set is not used to choose thresholds.
Threshold optimization: A single threshold optimization procedure is performed once using all cross-validation folds. For binary classification, a single threshold parameter \(\theta \in [0, 1]\) is optimized, where class 1 is predicted when \(P(\text{class } 1 \mid x) \geq \theta\), and class 0 otherwise. For multi-class problems, a threshold vector \(\theta = [\theta_1, \theta_2, \ldots, \theta_k]\) is optimized simultaneously for all \(k\) classes, where thresholds are applied through a normalized probability scheme: predictions are made by selecting the class with the highest normalized probability \(P(\text{class } i \mid x) / (\theta_i + \epsilon)\), where \(\epsilon = 10^{-10}\) prevents division by zero.
Each candidate threshold or threshold vector is applied to every fold’s validation probabilities. The app scores the resulting labels using the chosen metric, averages the scores across folds, and minimizes the negative mean score with the Nelder-Mead simplex algorithm.
Finally, the app trains the probability model on the entire training set and applies the optimized thresholds to its test-set probabilities to produce the final predictions.
Keep the features, classifier, hyperparameters, and train size fixed when comparing class-imbalance settings. Sampling and class weights change model training, so they can change the predicted probabilities and ROC curve. Changing only the decision threshold preserves the probabilities and ROC curve, but can move the operating point and change the confusion matrix.
Lowering a binary class’s probability threshold makes it easier to assign that class. Compare recall, specificity, precision, and accuracy to understand the trade-off. Thresholds are optimized within the training data; improvement on the held-out test set is not guaranteed. Choose the settings according to the metrics most important to the task.
21.2.3 Confusion Matrix
FLIM Playground evaluates the trained classifier on the held-out test set using its default decision rule or the tuned probability thresholds. The resulting predicted labels are summarized with sklearn.metrics.confusion_matrix. In the binary case, it is represented as:
| Predicted Positive | Predicted Negative | |
|---|---|---|
| Actual Positive | True Positive (TP) – correct hits | False Negative (FN) – missed positives |
| Actual Negative | False Positive (FP) – false alarms | True Negative (TN) – correct rejections |
21.2.4 Performance Metrics
The Overall Metrics table reports accuracy and N, the number of held-out test observations. Per-Class Metrics reports each class’s test-set support and the measures below, with an unweighted average across classes. Recall is highlighted when threshold tuning uses Balanced Accuracy, and F1 Score is highlighted when it uses F1 Score.
Overall accuracy is the number of correct predictions divided by the number of test observations: the sum of the confusion matrix’s diagonal divided by the sum of all its entries. For binary classification, this is:
- Accuracy: \(\frac{TP + TN}{TP + FP + TN + FN}\)
Other metrics are calculated for each class based on the confusion matrix:
- Precision: \(\frac{TP}{TP + FP}\)
- Recall (Sensitivity): \(\frac{TP}{TP + FN}\)
- Specificity: \(\frac{TN}{TN + FP}\)
- Youden’s Index: \(\text{Sensitivity} + \text{Specificity} - 1\)
- F1 Score: \(\frac{2 \times \text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}\)
Precision, recall, and specificity reveal trade-offs that accuracy alone can hide. See this visual explanation of precision and recall.
21.2.5 ROC Curve
The Receiver Operating Characteristic (ROC) curve shows how true-positive and false-positive rates change as the probability threshold varies. The confusion matrix summarizes one decision rule; the ROC curve lets you explore the trade-offs across thresholds.
At a threshold above the highest predicted probability, every observation is classified as negative, so the false-positive rate \(\text{FPR} = \frac{\text{FP}}{\text{FP} + \text{TN}}\) and true-positive rate \(\text{TPR} = \frac{\text{TP}}{\text{TP} + \text{FN}}\) are both 0. At a threshold of 0, every observation is classified as positive, giving the endpoint (1, 1).
The area under the ROC curve (AUC) is the probability that a randomly chosen positive observation receives a higher score than a randomly chosen negative observation, with tied scores contributing one-half.
The ROC curve displays the operating point (FPR, TPR) associated with the confusion matrix on the right.
In a binary problem, the curve’s legend names the class treated as positive. Read the displayed threshold in terms of that class’s probability; the app converts it to match the class shown on the curve.
See this visual explanation of ROC curves and AUC.
Multi-class ROC Curve
In multi-class classification, each class’s ROC curve is computed as a one-vs-rest binary classification problem. The operating point (FPR, TPR) associated with the confusion matrix is displayed for each class.
21.2.6 Feature Importance
Random Forest and Gradient Boosting classifiers provide feature importance scores.
21.3 Export the analysis
Click Export the entire analysis as a Python script below the results to retain the selected features, filters, target classes, classifier, hyperparameters, train size, sampling, class weights, threshold tuning, and font sizes. The script runs the classification workflow from the original input table, prints the metrics, and saves roc_curve.svg and confusion_matrix.svg. Random Forest and Gradient Boosting also save feature_importance.svg.
Follow the shared script workflow to run it or change its settings. The script repeats the row-based split and classification decisions used in the app.





