cross_val_predict
A scikit-learn function that returns a prediction for every training row from a model that never saw that row. It fits a copy of the model on all folds but one, predicts the held-out fold, and repeats, so the predictions can be used to choose a threshold without touching the test set.