API Reference

Samplers for Evaluation

Classes

OASISSampler(alpha, predictions, scores, oracle) Optimal Asymptotic Sequential Importance Sampling (OASIS) for estimation of the weighted F-measure.
ImportanceSampler(alpha, predictions, ...[, ...]) Importance sampling for estimation of the weighted F-measure
DruckSampler(alpha, predictions, scores, oracle) Stratified sampling for estimation of the weighted F-measure
PassiveSampler(alpha, predictions, oracle[, ...]) Passive sampling for estimation of the weighted F-measure

Stratification

Classes

stratification.Strata(allocations) Represents a collection of strata and facilitates sampling from them

Functions

stratification.stratify_by_scores(scores[, ...]) Stratify by binning the items based on their scores
stratification.stratify_by_features(...) Stratify by clustering the items in feature space
stratification.auto_stratify(scores, **kwargs) Generate Strata instance automatically