External Validation of the Hiveomics Radiomic Model at the Saint-Petersburg State Phthisiopulmonology Research Institute

At the 2025 annual congress of the National Alliance of Phthisiatrists (NASPH-2025), researchers from the Saint-Petersburg State Phthisiopulmonology Research Institute of the Russian Federation Healthcare Ministry presented the results of an independent external validation of the Hiveomics radiomic model designed to classify pulmonary nodules as typical intrapulmonary lymph nodes (ILNs). This work represents one of the most rigorous third-party evaluations of an ILN classifier to date and provides strong evidence supporting the clinical reliability of the Hiveomics system.

Study Objective

The primary goal of the study was to independently validate the performance of the Hiveomics AI-based radiomic system in distinguishing typical intrapulmonary lymph nodes from other pulmonary nodules using real-world clinical data not included in model training.

This external validation was conducted entirely by the Saint-Petersburg State Phthisiopulmonology Research Institute, without participation from the model developers, ensuring an unbiased assessment.

Materials and Methods

A total of 167 pulmonary nodules were analyzed:

• 44 nodules with histological verification
• 123 nodules classified by expert consensus Two certified thoracic radiologists, with 2 years and 25 years of experience respectively, independently assessed the cases.

The AI system evaluated each nodule and automatically classified it as:

• ILN or
• non-ILN

Ambiguous or controversial cases (n = 13) underwent a repeated expert review to establish a final reference standard.

Performance metrics included:

• Sensitivity
• Specificity
• Overall accuracy

Results

Initial evaluation

• Sensitivity: 80%
• Specificity: 87.9%
• Accuracy: 85.3%

These results already demonstrated that the model performs well in real clinical conditions, including complex and borderline cases.

Final evaluation (after re-review of ambiguous cases)

Following expert reassessment, the final reference standard allowed recalculation of model performance:

• Sensitivity: 91.7%
• Specificity: 97.9%
• Accuracy: 91.7%

These metrics indicate that the Hiveomics radiomic classifier is highly reliable, especially in distinguishing benign ILNs with near-perfect specificity. Importantly, the AI correctly identified several nodules that experts initially labeled as indeterminate, and subsequent consensus confirmed that the model's classification was accurate.

Clinical Significance

The high specificity of 97.9% is particularly meaningful in clinical decision-making. When the model identifies a nodule as a typical ILN, the likelihood of it being benign is extremely high.

For radiologists working in high-volume screening or oncology centers, such performance offers practical benefits:

• Reduced number of unnecessary follow-up CT scans
• Lower rates of unwarranted invasive procedures
• Improved diagnostic confidence in borderline cases
• Greater consistency in ILN recognition, addressing the well-documented variability between human readers

Given that ILNs account for a substantial proportion of pulmonary nodules encountered in routine chest CT, automated and reliable classification can meaningfully streamline workflow and reduce the burden of overdiagnosis.

Conclusions

The independent evaluation conducted at the Saint-Petersburg State Phthisiopulmonology Research Institute confirmed that the Hiveomics radiomic model:

• Performs with high sensitivity and exceptional specificity,
• Accurately identifies typical intrapulmonary lymph nodes, including cases difficult for human readers,
• Supports consistent and reproducible nodule assessment,
• Has significant potential to reduce unnecessary follow-up and invasive interventions.

These findings demonstrate that the model is well-suited for integration into clinical practice as a reliable decision-support tool for pulmonary nodule classification.