Patient records contain repeated visits per person; which split best prevents leakage?
A Random row split
B Stratified row split
C Patient-level group split
D Single-example split
Answer: C. Patient-level group split. A group split keeps every visit from one patient in the same partition.
In PCA, each later component is constrained to be what?
A Orthogonal to earlier components
B Equal to the first component
C A class label
D A target probability
Answer: A. Orthogonal to earlier components. Principal components are mutually orthogonal directions ordered by explained variance.
Repeatedly tuning against the same validation set can cause what?
A Under-sampling
B Label encoding
C Feature scaling
D Validation overfitting
Answer: D. Validation overfitting. Repeated selection can tailor a model to that validation set and inflate its apparent performance.
External validation at a different hospital mainly tests what?
A Training speed
B Transfer to a new population
C Feature count
D Label balance
Answer: B. Transfer to a new population. Evaluation on a distinct site checks whether performance transfers beyond the development population.
For binary scores, ROC AUC has which ranking interpretation?
A Negative outranks positive
B Classes are equally common
C Positive ranks above negative
D Predictions are calibrated
Answer: C. Positive ranks above negative. ROC AUC can be interpreted as the chance that a random positive receives a higher score than a random negative.
At a fixed recall, higher specificity generally means what?
A Fewer false positives
B More false negatives
C More positive predictions
D Higher training accuracy
Answer: A. Fewer false positives. Specificity is the fraction of actual negatives correctly identified, so higher specificity means fewer false positives.
What do class weights change inside a weighted loss?
A Number of input features
B Training-set size
C Predicted probabilities only
D Each class’s loss contribution
Answer: D. Each class’s loss contribution. Class weights change how strongly errors from each class contribute to the objective.
When every class counts equally despite imbalance, which F1 averaging is suitable?
A Micro average
B Macro average
C Weighted accuracy
D Binary threshold
Answer: B. Macro average. Macro averaging computes a score for each class and gives classes equal weight.
How should a decision threshold be chosen when error costs differ?
A Maximize training accuracy
B Use threshold 0.5 always
C Minimize expected deployment cost
D Ignore false negatives
Answer: C. Minimize expected deployment cost. A cost-aware threshold should reflect the expected costs of the different prediction errors.
Calibration differs from discrimination because it assesses what?
A Training-set size
B Ranking ability
C Feature relevance
D Probability reliability
Answer: D. Probability reliability. Calibration checks whether predicted probabilities agree with observed event frequencies.
To prevent target-encoding leakage, where should category statistics be estimated?
A Within each training fold
B Across the full dataset
C From test labels
D After final scoring
Answer: A. Within each training fold. Estimate target-encoding statistics using only the training portion of each validation fold.
Why place scaling inside a cross-validation pipeline?
A To remove every outlier
B Fit separately per fold
C To use test labels
D To avoid all missing values
Answer: B. Fit separately per fold. A pipeline refits scaling within each training fold, preventing held-out data from influencing the transform.
Under covariate shift, which distribution changes?
A Label names
B Model parameters
C Input distribution P(X)
D Number of classes
Answer: C. Input distribution P(X). Covariate shift changes the input distribution while the conditional target relationship is assumed stable.
Concept drift specifically means a change in what relationship?
A Input file format
B Class order
C Feature units
D Input-target relationship
Answer: D. Input-target relationship. Concept drift refers to a changing relationship between inputs and outcomes over time.
With expanding-window validation, which observations train the next fold?
A Earlier observations only
B Future observations only
C Randomized future rows
D Test labels
Answer: A. Earlier observations only. An expanding window trains on earlier observations and evaluates on a later period.
In nested cross-validation, what does the outer loop estimate?
A Training loss only
B Generalization after model selection
C The best feature name
D The final class label
Answer: B. Generalization after model selection. The outer loop estimates performance of the full model-selection procedure on held-out folds.
Permutation importance measures which change?
A Training time after scaling
B Number of selected features
C Score drop after shuffling
D Class balance after splitting
Answer: C. Score drop after shuffling. It measures how model performance changes when one feature’s values are shuffled.
Why can correlated features make individual importance unstable?
A They increase the target size
B They always cause leakage
C They remove all variance
D They substitute for each other
Answer: D. They substitute for each other. Correlated features can carry overlapping information, so shuffling one may leave its signal in another.
High ROC AUC but poor precision may reflect what?
A Rare positives at chosen threshold
B Perfect probability calibration
C A balanced test set
D No false positives
Answer: A. Rare positives at chosen threshold. With rare positives, even a strong ranking model may have low precision at a given threshold.
For F-beta with beta greater than one, which measure receives more weight?
A Specificity
B Recall
C Accuracy
D Training speed
Answer: B. Recall. An F-beta score with beta above one weights recall more heavily than precision.