Add evaluate_best_model option to skip final RMSE/MAE evaluation in mtt train

[zyxwwxyz]

Summary

• Adds an optional evaluate_best_model boolean to the mtt train YAML input (default true), following the pattern proposed in Add the possibility to avoid printing stats #1189 for print_stats.
• Also answers Add optional final_evaluation predictions dump to mtt train #1177 : by setting evaluate_best_model to false and subsequently running mtt eval on the training, validation and test sets, metrics are evaluated and predicted labels are printed in a resource efficient way.
• When set to false, mtt train skips reloading and evaluating the best model (the one exported to .pt) on the training, validation and test sets after training, avoiding the RMSE/MAE computation. Training, checkpointing and model export are unaffected.
• Useful for very large datasets where this final evaluation pass is expensive.

---

📚 Documentation preview 📚: https://metatrain--1192.org.readthedocs.build/en/1192/

[pfebrer]

Thanks! The fact that is the best model is a choice by each architecture. All we do is to export the model (here most architectures choose to export the best model) and evaluate it. Could you reword to capture this? Maybe the input is better called final_eval or something like this.

[pfebrer]

Perfect, I would say this is good to go 👍 We will merge #1189 first and then come to this one. The hyper will need to be moved to the new doc section introduced in that PR.

(right now everything is blocked because CSCS is not available to run the tests)