Clinical guidelines for interpreting genetic variants in the context of Mendelian disease require converting the outputs of pathogenicity prediction tools into well-calibrated probabilities. However, the existing calibration method is only valid when pre-committing to one tool, preventing clinical laboratories from using multiple tools with complementary strengths. To lift this restriction, we introduce Pathogenicity K-Nearest Neighbors (P-KNN), a flexible method that jointly calibrates any set of tools.