Technical review of a clinician-driven low-code workflow for anatomical segmentation in radiologic imaging

Step I: Preparing the data

FastMONAI offers the DecathlonDataset for direct data download and item generation. The images in this dataset are provided in the Neuroimaging Informatics Technology Initiative (NIfTI) format (.nii.gz). While clinical images are typically stored and transferred in the Digital Imaging and Communications in Medicine (DICOM) format, the NIfTI format is commonly used in research for its convenience in analysis and processing. To enable users to examine their own datasets, we demonstrate how to download and investigate files, particularly the JSON (JavaScript Object Notation) file. The JSON structure here consisted of two main components: image and label. The JSON contains image and label information, with labels distinguishing background (0) from left atrium (1), crucial for semantic segmentation models in medical imaging.

Further, we created a Pandas DataFrame with complete file paths for images and labels, facilitating data manipulation and integration into machine learning models. Using sklearn’s ‘train_test_split’ function, we divide the data into training (90%) and test (10%) sets. This split ensured that all images from a single patient were assigned exclusively to either the training or the test set to prevent data leakage and provide a robust evaluation of the model’s generalization capabilities.

STEP II: Preparing the datablock and loading the data

We enhanced data diversity using preprocessing and augmentation techniques. The MedDataset class analyzes label distribution, providing data dimensions, voxel size, orientation, and counts. We defined resampling, reordering, and image size standardization. Data augmentation included random rotation and normalization, improving model generalization. This step is crucial for significantly improving the model’s ability to generalize to unseen data, enriching the overall training process.

fastMONAI provides ‘MedDataBlock’ container to quickly build dataloaders. which formats data for neural network training, specifying input/output blocks (images/labels), splitting strategies (for validation sets), and augmentation methods. We defined the batch size, specifying the number of training examples processed in one iteration. A smaller batch size, such as 4, offers advantages like enhanced memory efficiency. For efficient data loading, we use random shuffling before creating mini-batches each epoch. FastMONAI’s DataLoader class automates shuffling and mini-batch collation, streamlining the data handling workflow.

STEP III: Preparing the learner

A Learner in deep learning consolidates essential components for model training: dataloader, model architecture, loss function, optimizer, and metrics.

We imported the U-Net model from MONAI for our segmentation task. U-Net excels in biomedical image segmentation by capturing local features (such as object shape or texture) and global features (such as the object’s position within the image).¹¹ We initialized U-Net for 3D MRI images with three spatial dimensions, one input channel, and one output channel for binary segmentation. The architecture includes encoder pathway (contracting path) and a decoder pathway (expansive path), with the ‘channels’ parameter dictating feature map presence at each level during encoding and decoding.¹² The ‘stride’ parameter controls down-sampling by determining the shift of the filter/kernel over input data during convolution operations. The number of residual units aids in learning complex patterns by facilitating direct gradient flow between layers, enhancing convergence during training.

We used DiceLoss from MONAI as our loss function, measuring overlap between predicted and target segmentation masks. The ‘sigmoid’ parameter allows interpretation as probabilities. Setting the ‘sigmoid’ parameter to True applies a sigmoid activation function to the model’s output before calculating the Dice Loss, allowing interpretation as probabilities or confidence scores. The Dice loss ranges from 0 (perfect overlap) to 1 (no overlap). It is defined as 1 - Dice coefficient.

$\text{Loss}=1-\frac{2\left|y\cap \hat{y}\right|}{\left|y\right|+\left|\hat{y}\right|+\in }$

Our optimizer function was set to ‘ranger’, an optimization algorithm combining RAdam and LookAhead mechanisms. Ranger is lauded for its ability to achieve rapid convergence and robust performance by adjusting neural network parameters such as weights and learning rate to minimize losses.¹³ For evaluating model performance during training or validation, the binary dice score was chosen as metric. It measures the overlap between predicted and actual binary masks, with higher scores indicating better alignment between predictions and ground truth labels.

This Learner instance efficiently manages tasks such as forwarding input data through network layers, calculating losses based on predictions versus actual labels, adjusting network parameters using the specified optimizer based on calculated losses, and tracking progress using specified metrics.

STEP IV: Training

Before training, it is crucial to find the optimal learning rate. The learn.lr_find() method helps identify this rate by training the model at incrementally increasing rates, plotting losses against learning rates on a logarithmic scale. As the learning rate increases, the model learns faster, reducing loss. However, an excessively high learning rate can cause overshooting or divergence, resulting in increased losses. This process yielded a plot with a distinct shape, highlighting a value (10^-2) where the model learned efficiently without divergence or overshooting [Figure 2].

Close

Figure 2:

Determination of optimal learning rate.

Export to PPT

We initiated training using the fit_flat_cos method, employing ‘Flat and Cosine Annealing’ scheduling. This maintains a constant learning rate initially, then gradually reduces it following a cosine schedule. This strategy aids in fine-tuning parameters and achieving optimal solutions without excessive oscillations. Furthermore, to iteratively refine its parameters and converge towards optimal performance, we trained for 200 epochs (complete passes through our dataset) at the designated learning rate.

To visualize our model’s training process, we utilized the `learn.recorder.plot_loss()` function. This function leverages the recorder attribute within a Learner object, which tracks various metrics throughout training, including losses at each step or epoch. Insights gained from these plots include:

• If both training and validation losses decrease, it indicates continual improvement on both seen (training) and unseen (validation) data.

• If training loss decreases while validation loss increases, it suggests potential overfitting to training data—improved performance on seen data but worse on unseen data.

• If both losses remain high or do not decrease, the model might be underfitting, requiring adjustments such as a more complex architecture or an enhanced training strategy.

Alternatively, ‘matplotlib.pyplot’ can be used to plot training, validation losses, and dice scores from ‘learn.recorder.values’ [Figure 3].

Close

Figure 3:

Visualization of metrics obtained after training.

Export to PPT

After training, we saved the model using ‘learn.save()’ to preserve the learner’s internal state, including acquired parameters. This step conserves time and computational resources invested in training. For visual inspection, we used ‘learn.show_results()’ to generate images comparing model predictions with ground truth for validation set samples. This visual inspection provides insights into potential shortcomings in specific image types or data patterns.

STEP V: Evaluate model with test data

To see how well our model has been training, we now evaluated it with our test data. For evaluation, the test data generated during the splitting process in step I was utilized. To facilitate this, a dataloader named ‘test_dl’ was created for the test data DataFrame, `test_df`.

For numerical evaluation, the Dice score was employed as a metric. The model’s predictions were compared against the expert-provided segmentation masks from the public dataset, which served as the gold standard. This yielded a Dice score of approximately 0.9232, indicating a significant overlap between the predicted and actual segmented areas. This score indicated a significant overlap between the predicted and actual segmented areas. Furthermore, the `learn.show_results` method was employed to visually present the model’s outcomes on the test data at specific anatomical planes [Figure 4]. This visual inspection of the final model’s performance on unseen test data is the standard for evaluation and is representative of similar validation checks performed throughout the training process using the same function on the validation set. This dual approach, integrating both quantitative and visual assessments, supplied valuable insights into the model’s efficacy, allowing for a comprehensive evaluation of its performance.

Close

Figure 4:

Prediction on test images with segmentation of the left atrium by the trained model.

Export to PPT

STEP VI: Export and deploy

We saved the crucial variables related to data processing and model configuration into a pickle file. Furthermore, the trained model itself was exported using the learn.export(‘heart_model.pkl’) method. This process encapsulated everything about the learner object, not just the model parameters, but also architectural details. The resultant ‘heart_model.pkl’ file serves as a comprehensive package for deployment. We deployed our trained model using Gradio on Hugging Face, which can be publicly accessed at https://drankush-ai-left-atrium-heart-segmentation.hf.space/ [Figure 5].

Close

Figure 5:

Deployment using Gradio for demonstration, can be accessed at https://drankush-ai-left-atrium-heart-segmentation.hf.space/.

Export to PPT

Democratizing deep learning education and research

A European survey of 1041 radiologists and residents revealed that limited AI expertise correlated with fear of AI replacing roles and less proactive attitudes. This suggests that unfamiliarity with AI limitations may lead to overestimation of its threat and undervaluation of one’s role in AI integration. Conversely, advanced AI knowledge fostered a more open stance.¹⁵ The survey identified a lack of knowledge as a significant hurdle in AI implementation, with 79% supporting AI education in radiology residency programs.¹⁶ Recent discussions emphasize the need for AI curricula that enable students to apply machine learning without delving into complex algorithms.¹⁷ Richardson and Odeja’s curriculum for teaching deep learning to radiology residents using a No-code system demonstrated effectiveness in conveying insights and increasing interest.¹⁸ The advantages and challenges of employing Low-Code/No-code (LCNC) AI for teaching machine learning in higher education are increasingly recognized.¹⁹

There is scarcity of beginner-friendly literature on AI model training in radiology partly also due to rapid advancements rendering libraries obsolete. For instance, the popular “Magician’s Corner” tutorial series, designed to teach U-net model building for image segmentation, now encounters errors due to Google Colab no longer supporting TensorFlow 1.²⁰ Our Simplified-six-step workflow based on an actively maintained LC library offers a structured pathway for efficient foundational model preparation for radiologists and residents interested in learning model training through a hands-on approach. Through a series of intuitive function calls and concise steps, this workflow navigates beginners through data handling, augmentation, loss function selection, model architecture utilization, hyperparameter tuning, and training monitoring. These sequence of steps provide an iterative process in a human-centric manner by being more balanced in learning curve and intuitively explaining the design choice.²¹ This streamlined process allows professionals to focus on refining models rather than wrestling with intricate coding, thereby significantly enhancing overall efficiency. Customized LCNC AI platforms in health informatics can alleviate technical knowledge challenges in various domains and democratizing AI access for non-coders.^22,23 Apart from our case study of segmentation of left atrium, fastMONAI has been successfully used for tumor segmentation in cervical cancer, endometrial cancer, spine segmentation and pulmonary nodule classification.^24-27

Furthermore, this approach also cultivates continuous learning opportunities within medical organizations. Enabling non-technical team members to engage in AI model development fosters a culture of interdisciplinary collaboration.^28,29 LC also reduces the occurrence of unpredictable or inconsistent requirements, making it easier to construct initial prototypes for requirement validation. This helps prevent resource wastage on unnecessary features or functionalities and avoids unnecessary development cycles.³⁰

Comparative advantages and limitations

Although LCNC solutions could present generalizable workflow for training a model, the foundation of training an effective and practical generalizable model however rests upon the quality of input data; requiring a diverse set of MRI images encompassing various demographics, pathologies, and image qualities necessitating meticulous preprocessing to address inconsistencies and artifacts, ensuring the model’s ability to generalize effectively across varied scenarios [Table 1].³¹ This technical review focuses on the feasibility and efficiency of the low-code training workflow itself. A full clinical validation was beyond the scope of this paper. Before any clinical deployment, a crucial next step would be a rigorous co-evaluation of the model’s output with a team of expert clinicians and a statistical assessment against manual segmentations to account for inter- and intra-rater variability. LC also presents challenges in terms of scalability, as it may enable project-based solutions but might not materialize at the enterprise level.³⁰ Although LC libraries provide greater room for adjustment compared to No-code libraries, a key trade-off exists. These platforms prioritize ease-of-use and rapid development, which may come at the cost of the fine-grained control needed for highly novel research. For example, designing a completely new neural network layer or a bespoke loss function from first principles would be difficult or impossible. Therefore, LC solutions are ideal for empowering clinicians to apply established, state-of-the-art architectures to new clinical problems, while traditional, code-intensive workflows remain essential for pushing the boundaries of machine learning research.

While this review focuses on the U-Net architecture due to its widespread adoption, proven efficacy, and straightforward implementation within LCNC frameworks, it is important to acknowledge recent advancements in model architectures. Newer models, Beyond convolutional baselines, transformer-based architectures such as Vision Transformers and hybrid variants (e.g., TransUNet, Swin-UNet) have advanced performance in medical segmentation by capturing long-range dependencies and global context. In parallel, foundation models like the Segment Anything Model (SAM) enable promptable, zero-/few-shot segmentation and can accelerate annotation, though careful domain adaptation is needed for grayscale modalities, small structures, and domain shift.³²

Regulatory and privacy considerations

It is critical to address the regulatory landscape surrounding AI in medicine. Any software tool intended for clinical diagnosis or treatment planning is considered a medical device and is subject to rigorous oversight and approval from regulatory bodies like the U.S. Food and Drug Administration (FDA) or to receive a CE mark in Europe. While low-code platforms are excellent for research, rapid prototyping, and developing institution-specific, non-diagnostic tools, any model intended for widespread clinical use must undergo a formal validation and documentation process that meets these stringent regulatory standards. This remains true regardless of the platform it was built on. Developers must also ensure that all aspects of data handling, from training to deployment, are compliant with data privacy regulations such as HIPAA.

Future development

Research efforts have been initiated towards the development of a semi-automatic annotation loop for fastMONAI with an active learning pipeline by providing expert annotators with automatically segmented lesion/target instances for refinement.⁴ Future endeavours also seek to expand on PACS (Picture Archiving Communications System) integration and to provide more detailed and extensive documentation.