In current implementation of orion.benchmark, the holdout parameter is expected to be a boolean. Indicating whether to use a predefined train-test split or not.
The suggested improvement is to have holdout be of two types:
Following this change, I recommend changing holdout to be test_split such that the user can understand the affected dataset size.
Example
benchmark(test_split=True): uses a predefined test split in path.benchmark(test_split=False): does use split and thus data used for training will be used for testing.benchmark(test_split=0.2): the dataset will be split into 80% training and 20% testing.RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4