Cross-Validation > Single Validation Set
Yes, you should have a test set, which you never touch, only when evaluating your final model. In a sense, we could call this the "measuring set", as its only purpose should be to give you a performance number of unseen data.
However, for the model building, hyperparameter tuning and feature engineering that leads β¦