Skip to main content

Conference Program

Subpage Hero

Loading

VEPerform: A Web Resource for Evaluating the Performance of Variant Effect Predictors 

Education and Research Strategies
  • Primary Categories:
    • Basic Research
  • Secondary Categories:
    • Basic Research
Introduction:
Computational variant effect predictors (VEPs) are providing increasingly-strong evidence to classify the pathogenicity of missense variants. Many VEP comparisons have aggregated performance across genes and diseases, but there is a need for a standardized platform to assess performance at the individual gene level, in order not only to choose the best VEP, but also to begin to evaluate the strength of evidence it can provide and identify genes for which experimental functional evidence is most important. VEPerform is a web-based application that allows users to evaluate VEP performance for individual genes.

Methods:
VEPerform assesses performance by evaluating, across the range of potential VEP score thresholds, the tradeoff between precision (fraction of variants below a given threshold that are truly pathogenic) vs recall (fraction of known pathogenic variants that are identified using that threshold). Because precision depends on the balance of the reference set used, VEPerfom transforms precision values into balanced precision values (the precision that would have been observed if the test set had been balanced to contain exactly 50% pathogenic variants). This provides a balanced precision-recall curve (BPRC), with two useful quantitative measures of performance: area under the balanced precision vs recall curve (AUBPRC) and recall at 90% balanced precision (R90BP).

VEPerform provides a default set of VEPs and a reference set of ClinVar curated variants annotated as pathogenic, likely-pathogenic, benign, or likely-benign. Users can select a gene, choose one or more predictors from VARITY, REVEL, or AlphaMissense, and apply a common variant filter based on gnomAD allele frequencies (>0.005). Alternatively, users can manually filter this variant list or augment it with additional VEPs. VEPerform supports custom user-provided reference sets with any number of predictor scores. In the case of augmentation, VEPerform can fetch VEP scores from OpenCRAVAT. Supported VEPs include AlphaMissense, VARITY (R and ER), REVEL, PROVEAN, SIFT, and PolyPhen-2.

Results:
The above functionalities were implemented in a Shiny web app. Apart from generating the BPRC, VEPerform provides multiple data export options, including a PDF with the BPRC plot and metadata, or a CSV of the variants used in generating the BPRC.

Conclusion:
VEPerform is a powerful tool for evaluating and comparing the performance of variant effect predictors. By leveraging a combination of pre-stored data and custom user-uploaded variant processing, VEPerform streamlines variant annotation and performance assessment workflows. It is an essential resource for genomic researchers and clinicians seeking to make informed decisions about variant effect predictors.

Agenda

Sponsors