Decision trees are supervised machine learning models used for classification and regression tasks. They work by splitting data into branches based on decision rules. Each split aims to improve prediction accuracy or reduce uncertainty. Decision trees are easy to interpret and visualize.
Hyperparameter optimization (HPO) is used in machine learning to find the best settings (hyperparameters) for a learning algorithm to maximize model performance, accuracy, and generalization on a specific dataset. Grid search and random search are methods for hyperparameter optimization in machine learning models. They evaluate model performance across different parameter combinations.
- Grid search tests all predefined combinations systematically.
- Random search samples parameter values randomly within given ranges.
Both help improve model performance and generalization.
In this case it was applied to the Breast Cancer Wisconsin (diagnostic) dataset and another synthetic dataset.