Skip to content
This repository was archived by the owner on Sep 18, 2024. It is now read-only.

HPO Benchmark Fixes and New Features#3925

Merged
QuanluZhang merged 31 commits into
microsoft:masterfrom
xiaowu0162:hpo-benchmark-v1-fix
Jul 26, 2021
Merged

HPO Benchmark Fixes and New Features#3925
QuanluZhang merged 31 commits into
microsoft:masterfrom
xiaowu0162:hpo-benchmark-v1-fix

Conversation

@xiaowu0162

@xiaowu0162 xiaowu0162 commented Jul 11, 2021 •
edited
Loading

Copy link
Copy Markdown
Contributor

This pr fixes several issues found in the first version of the HPO Benchmark.
Update: the new feature "new search space for MLP" is also added in this pr.

  • Fix serialized run
  • Retry if tuners return None for generate_parameters
  • Check time limit
  • Check search space design
  • Cast params to int for sklearn
  • Split nnismall into three benchmarks
  • Update and expand documentation
  • New search space for MLP
  • Docs for new features

@xiaowu0162 xiaowu0162 changed the title HPO Benchmark V1 fixes HPO Benchmark V1 Fixes Jul 11, 2021
@xiaowu0162 xiaowu0162 closed this Jul 12, 2021
@xiaowu0162 xiaowu0162 reopened this Jul 12, 2021
@xiaowu0162 xiaowu0162 marked this pull request as ready for review July 12, 2021 03:32
QuanluZhang
QuanluZhang reviewed Jul 12, 2021
View reviewed changes
Comment thread docs/en_US/hpo_benchmark.rst Outdated
QuanluZhang
QuanluZhang reviewed Jul 12, 2021
View reviewed changes
Comment thread docs/en_US/hpo_benchmark.rst Outdated
@QuanluZhang

QuanluZhang commented Jul 12, 2021

Copy link
Copy Markdown
Contributor

@xiaowu0162 suggest to further refine the doc

@xiaowu0162 xiaowu0162 marked this pull request as draft July 13, 2021 02:34
@xiaowu0162

This comment has been minimized.

Sign in to view
@xiaowu0162 xiaowu0162 changed the title HPO Benchmark V1 Fixes HPO Benchmark Fixs and New Features Jul 13, 2021
@xiaowu0162 xiaowu0162 marked this pull request as ready for review July 14, 2021 02:33
@QuanluZhang QuanluZhang mentioned this pull request Jul 14, 2021
Closed
@xiaowu0162 xiaowu0162 requested a review from QuanluZhang July 14, 2021 02:53
@xiaowu0162 xiaowu0162 changed the title HPO Benchmark Fixs and New Features HPO Benchmark Fixes and New Features Jul 16, 2021
J-shang
J-shang reviewed Jul 19, 2021
View reviewed changes
Comment thread docs/en_US/hpo_benchmark.rst Outdated
Comment thread ...es/trials/benchmarking/automlbenchmark/nni/extensions/NNI/architectures/run_random_forest.py
Comment thread ...es/trials/benchmarking/automlbenchmark/nni/extensions/NNI/architectures/run_random_forest.py
Comment thread docs/en_US/hpo_benchmark.rst Outdated
Comment thread ...es/trials/benchmarking/automlbenchmark/nni/extensions/NNI/architectures/run_random_forest.py
@QuanluZhang QuanluZhang requested a review from ultmaster July 21, 2021 02:45
@QuanluZhang QuanluZhang merged commit 5bf2cb1 into microsoft:master Jul 26, 2021
@xiaowu0162 xiaowu0162 deleted the hpo-benchmark-v1-fix branch July 26, 2021 02:15
@acured acured mentioned this pull request Aug 16, 2021
Closed
78 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

3 more reviewers
@ultmaster ultmaster ultmaster approved these changes
@QuanluZhang QuanluZhang QuanluZhang approved these changes
@J-shang J-shang J-shang approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@xiaowu0162 @QuanluZhang @ultmaster @J-shang