Ten Years of Experience with an Iterative, Points-based Variant Classification Framework
Laboratory Genetics and Genomics
-
Primary Categories:
- Laboratory Genetics
-
Secondary Categories:
- Laboratory Genetics
Introduction:
The science of variant classification is integral to clinical genetics and relies on classification frameworks to systematically review evidence. As new data sources and tools become available, it is important to incorporate them into existing frameworks expeditiously. Ten years ago we developed a points-based system for clinical variant classification, based on the ACMG/AMP guidelines, that evaluates variant type, allele frequency, and clinical, functional, and computational evidence. Since its first version, the framework has undergone many iterations to refine and expand its usage and incorporate new evidence. We describe how this framework has changed through years of iteration and the impact these changes have had on the classification and reclassification of a large dataset of variants.
Methods:
In this classification framework, criteria are given a weighted points value and are assigned to an evidence-type group. Only one criterion in a group can be applied per variant, which prevents any single line of evidence from being overcounted as more criteria are added. Criteria were added to the framework in a versioned manner, prompted by feedback from variant classification scientists and newly available data sources. We reviewed 18 versions of the classification framework and assigned added criteria to categories. The impact of the added criteria on approximately 65,000 variants of uncertain significance (VUS) that changed classification between 2015 and 2023 was analyzed. To determine how frequently the added criteria led to reclassification, we compared the presence of these criteria in the initial versus updated classifications. In addition, approximately 75,000 variants that were recently classified using the latest framework version were examined. In this dataset, the impact of removing criteria added since the initial version of the classification framework was analyzed.
Results:
Throughout the versions analyzed, the number of criteria in the classification framework increased from 134 to 300. The added criteria were refinements to existing groups (variant type, clinical, functional), incorporated new data sources (machine learning algorithms, genome aggregation database (gnomAD), RNA experimental data), or expanded the framework to accommodate new classes of variants (cytogenetic events, mitochondrial variants, repeat expansions). We found that most variant reclassifications during this timeframe were due to new criteria added to the framework. Specifically, 71% of reclassifications were driven by the addition of criteria that incorporated novel data sources, predominantly machine learning (55%) and gnomAD criteria (14%). In contrast, only 24% of reclassifications were due to new data captured by evidence criteria present in the original classification framework (e.g. new clinical data or changes in gene curation). Separately, when recent variant classifications were recalculated without the newly added criteria, 22% of the classifications changed. 98% of the changes in classification resulting from this simulated removal of criteria were shifts from a pathogenic, likely pathogenic, likely benign, or benign classification to variants of uncertain significance. Most classification changes were due to removing machine learning criteria. Removing new RNA, clinical, and variant type criteria resulted in fewer changes to classifications.
Conclusion:
The iterative addition of criteria had a large impact on variant classification, resulting in more variants reaching clinically impactful classifications. Criteria that incorporated new data sources were the most impactful, while refinements to existing criteria resulted in smaller numbers of classification changes. These findings suggest that iteration should be a primary consideration when designing a variant classification framework. An iterative framework must allow for the incorporation of new criteria without perturbing the relationship between evidence types or overcounting certain lines of evidence. We have found that a weighted points-based framework with evidence-type groups provides the structure and flexibility needed for continual progress.
The science of variant classification is integral to clinical genetics and relies on classification frameworks to systematically review evidence. As new data sources and tools become available, it is important to incorporate them into existing frameworks expeditiously. Ten years ago we developed a points-based system for clinical variant classification, based on the ACMG/AMP guidelines, that evaluates variant type, allele frequency, and clinical, functional, and computational evidence. Since its first version, the framework has undergone many iterations to refine and expand its usage and incorporate new evidence. We describe how this framework has changed through years of iteration and the impact these changes have had on the classification and reclassification of a large dataset of variants.
Methods:
In this classification framework, criteria are given a weighted points value and are assigned to an evidence-type group. Only one criterion in a group can be applied per variant, which prevents any single line of evidence from being overcounted as more criteria are added. Criteria were added to the framework in a versioned manner, prompted by feedback from variant classification scientists and newly available data sources. We reviewed 18 versions of the classification framework and assigned added criteria to categories. The impact of the added criteria on approximately 65,000 variants of uncertain significance (VUS) that changed classification between 2015 and 2023 was analyzed. To determine how frequently the added criteria led to reclassification, we compared the presence of these criteria in the initial versus updated classifications. In addition, approximately 75,000 variants that were recently classified using the latest framework version were examined. In this dataset, the impact of removing criteria added since the initial version of the classification framework was analyzed.
Results:
Throughout the versions analyzed, the number of criteria in the classification framework increased from 134 to 300. The added criteria were refinements to existing groups (variant type, clinical, functional), incorporated new data sources (machine learning algorithms, genome aggregation database (gnomAD), RNA experimental data), or expanded the framework to accommodate new classes of variants (cytogenetic events, mitochondrial variants, repeat expansions). We found that most variant reclassifications during this timeframe were due to new criteria added to the framework. Specifically, 71% of reclassifications were driven by the addition of criteria that incorporated novel data sources, predominantly machine learning (55%) and gnomAD criteria (14%). In contrast, only 24% of reclassifications were due to new data captured by evidence criteria present in the original classification framework (e.g. new clinical data or changes in gene curation). Separately, when recent variant classifications were recalculated without the newly added criteria, 22% of the classifications changed. 98% of the changes in classification resulting from this simulated removal of criteria were shifts from a pathogenic, likely pathogenic, likely benign, or benign classification to variants of uncertain significance. Most classification changes were due to removing machine learning criteria. Removing new RNA, clinical, and variant type criteria resulted in fewer changes to classifications.
Conclusion:
The iterative addition of criteria had a large impact on variant classification, resulting in more variants reaching clinically impactful classifications. Criteria that incorporated new data sources were the most impactful, while refinements to existing criteria resulted in smaller numbers of classification changes. These findings suggest that iteration should be a primary consideration when designing a variant classification framework. An iterative framework must allow for the incorporation of new criteria without perturbing the relationship between evidence types or overcounting certain lines of evidence. We have found that a weighted points-based framework with evidence-type groups provides the structure and flexibility needed for continual progress.