Skip to content

v2.0.0
Compare
Choose a tag to compare
@hvy hvy released this 29 Jul 00:13
v2.0.0
bb91b3c
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

This is the release note of v2.0.0.

Highlights

The second major version of Optuna 2.0 is released. It accommodates a multitude of new features, including Hyperband pruning, hyperparameter importance, built-in CMA-ES support, grid sampler, and LightGBM integration. Storage access is also improved, significantly speeding up optimization. Documentation has been revised and navigation is made easier. See the blog for details.

Hyperband Pruner

The stable version of HyperbandPruner is available with a simpler interface and improved performance.

Hyperparameter Importance

The stable version of the hyperparameter importance module is available.

  • Our implementation of fANOVA, FanovaImportanceEvaluator is now the default importance evaluator. This replaces the previous requirement for fanova with scikit-learn.
  • A new importance visualization function visualization.plot_param_importances.

image7

Built-in CMA-ES Sampler

The stable version of CmaEsSampler is available. This new CmaEsSampler can be used with pruning for major performance improvements.

Grid Sampler

The stable version of GridSampler is available through an intuitive interface for users familiar with Optuna. When the entire grid is exhausted, the optimization stops automatically, so you can specify n_trials=None.

LightGBM Tuner

The stable version of LightGBMTuner is available. The behavior regarding verbosity option has been improved. The random seed was fixed unexpectedly if the verbosity level is not zero, but now the user given seed is used correctly.

Experimental Features

  • New integration modules: TensorBoard integration, Catalyst integration, and AllenNLP pruning integration are available as experimental.
  • A new visualization function for multi-objective optimization: multi_objective.visualization.plot_pareto_front is available as an experimental feature.
  • New methods to manually create/add trials: trial.create_trial and study.Study.add_trial are available as experimental features.

Breaking Changes

Several deprecated features (e.g., Study.study_id and Trial.trial_id) are removed. See #1346 for details.

  • Remove deprecated features in optuna.trial (#1371)
  • Remove deprecated arguments from LightGBMTuner (#1374)
  • Remove deprecated features in integration/chainermn.py (#1375)
  • Remove deprecated features in optuna/structs.py (#1377)
  • Remove deprecated features in optuna/study.py (#1379)

Several features are deprecated.

  • Deprecate optuna study optimize command (#1384)
  • Deprecate step argument in IntLogUniformDistribution (#1387, thanks @nzw0301!)

Other.

  • BaseStorage.set_trial_param to return None instead of bool (#1327)
  • Match suggest_float and suggest_int specifications on step and log arguments (#1329)
  • BaseStorage.set_trial_intermediate_valute to return None instead of bool (#1337)
  • Make optuna.integration.lightgbm_tuner private (#1378)
  • Fix pruner index handling to 0-indexing (#1430, thanks @bigbird555!)
  • Continue to allow using IntLogUnioformDistribution.step during deprecation (#1438)
  • Align LightGBMTuner verbosity level to the original LightGBM (#1504)

New Features

  • Add snippet of API for integration with Catalyst (#1056, thanks @VladSkripniuk!)
  • Add pruned trials to trials being considered in CmaEsSampler (#1229)
  • Add pruned trials to trials being considered in SkoptSampler (#1431)
  • Add TensorBoard integration (#1244, thanks @VladSkripniuk!)
  • Add deprecation decorator (#1382)
  • Add plot_pareto_front function (#1303)
  • Remove experimental decorator from HyperbandPruner (#1435)
  • Remove experimental decorators from hyperparameter importance (HPI) features (#1440)
  • Remove experimental decorator from Study.stop (#1450)
  • Remove experimental decorator from GridSampler (#1451)
  • Remove experimental decorators from LightGBMTuner (#1452)
  • Introducing optuna.visualization.plot_param_importances (#1299)
  • Rename integration/CmaEsSampler to integration/PyCmaSampler (#1325)
  • Match suggest_float and suggest_int specifications on step and log arguments (#1329)
  • optuna.create_trial and Study.add_trial to create custom studies (#1335)
  • Allow omitting the removal version in deprecated (#1418)
  • Mark CatalystPruningCallback integration as experimental (#1465)
  • Followup TensorBoard integration (#1475)
  • Implement a pruning callback for AllenNLP (#1399)
  • Remove experimental decorator from HPI visualization (#1477)
  • Add optuna.visualization.plot_edf function (#1482)
  • FanovaImportanceEvaluator as default importance evaluator (#1491)
  • Reduce HPI variance with default args (#1492)

Enhancements

  • Support automatic stop of GridSampler (#1026)
  • Implement fANOVA using sklearn instead of fanova (#1106)
  • Add a caching mechanism to make NSGAIIMultiObjectiveSampler faster (#1257)
  • Add log argument support for suggest_int of skopt integration (#1277, thanks @nzw0301!)
  • Add read_trials_from_remote_storage method to Storage implementations (#1298)
  • Implement log argument for suggest_int of pycma integration (#1302)
  • Raise ImportError if bokeh version is 2.0.0 or newer (#1326)
  • Fix the x-axis title of the hyperparameter importances plot (#1336, thanks @harupy!)
  • BaseStorage.set_trial_intermediate_valute to return None instead of bool (#1337)
  • Simplify log messages (#1345)
  • Improve layout of plot_param_importances figure (#1355)
  • Do not run the GC after every trial by default (#1380)
  • Skip storage access if logging is disabled (#1403)
  • Specify stacklevel for warnings.warn for more helpful warning message (#1419, thanks @harupy!)
  • Replace DeprecationWarning with FutureWarning in @deprecated (#1428)
  • Fix pruner index handling to 0-indexing (#1430, thanks @bigbird555!)
  • Load environment variables in AllenNLPExecutor (#1449)
  • Stop overwriting seed in LightGBMTuner (#1461)
  • Suppress progress bar of LightGBMTuner if verbosity == 1 (#1460)
  • RDB storage to do eager backref "join"s when fetching all trials (#1501)
  • Overwrite intermediate values correctly (#1517)
  • Overwrite parameters correctly (#1518)
  • Always cast choices into tuple in CategoricalDistribution (#1520)

Bug Fixes

RDB Storage Bugs on Distributed Optimization are Fixed

Several critical bugs are addressed in this release with the RDB storage, most related to distributed optimization.

  • Fix CMA-ES boundary constraints and initial mean vector of LogUniformDistribution (#1243)
  • Temporary hotfix for sphinx update breaking existing type annotations (#1342)
  • Fix for PyTorch Lightning v0.8.0 (#1392)
  • Fix exception handling in ChainerMNStudy.optimize (#1406)
  • Use step to calculate range of IntUniformDistribution in PyCmaSampler (#1456)
  • Avoid exploding queries with large exclusion sets (#1467)
  • Temporary fix for problem with length limit of 5000 in MLflow (#1481, thanks @PhilipMay!)
  • Fix race condition for trial number computation (#1490)
  • Fix CachedStorage skipping trial param row insertion on cache miss (#1498)
  • Fix _CachedStorage and RDBStorage distribution compatibility check race condition (#1506)
  • Fix frequent deadlock caused by conditional locks (#1514)

Installation

  • [Backport] Add packaging in install_requires (#1561)
  • Set python_requires in setup.py to clarify supported Python version (#1350, thanks @harupy!)
  • Specify classifiers in setup.py (#1358)
  • Hotfix to avoid latest keras 2.4.0 (#1386)
  • Hotfix to avoid PyTorch Lightning 0.8.0 (#1391)
  • Relax sphinx version (#1393)
  • Update version constraints of cmaes (#1404)
  • Align sphinx-rtd-theme and Python versions used on Read the Docs to CircleCI (#1434, thanks @harupy!)
  • Remove checking and alerting installation pfnopt (#1474)
  • Avoid latest sphinx (#1485)
  • Add packaging in install_requires (#1561)

Documentation

  • Fix experimental decorator (#1248, thanks @harupy!)
  • Create a documentation for the root namespace optuna (#1278)
  • Add missing documentation for BaseStorage.set_trial_param (#1316)
  • Fix documented exception type in BaseStorage.get_best_trial and add unit tests (#1317)
  • Add hyperlinks to key features (#1331)
  • Add .readthedocs.yml to use the same document dependencies on the CI and Read the Docs (#1354, thanks @harupy!)
  • Use Colab to demonstrate a notebook instead of nbviewer (#1360)
  • Hotfix to allow building the docs by avoiding latest sphinx (#1369)
  • Update layout and color of docs (#1370)
  • Add FAQ section about OOM (#1385)
  • Rename a title of reference to a module name (#1390)
  • Add a list of functions and classes for each module in reference doc (#1400)
  • Use .. warning:: instead of .. note:: for the deprecation decorator (#1407)
  • Always use Sphinx RTD theme (#1414)
  • Fix color of version/build in documentation sidebar (#1415)
  • Use a different font color for argument names (#1436, thanks @harupy!)
  • Move css from _templates/footer.html to _static/css/custom.css (#1439)
  • Add missing commas in FAQ (#1458)
  • Apply auto-formatting to custom.css to make it pretty and consistent (#1463, thanks @harupy!)
  • Update CONTRIBUTING.md (#1466)
  • Add missing CatalystPruningCallback in the documentation (#1468, thanks @harupy!)
  • Fix incorrect type annotations for catch (#1473, thanks @harupy!)
  • Fix double FrozenTrial (#1478)
  • Wider main content container in the documentation (#1483)
  • Add TensorBoardCallback to docs (#1486)
  • Add description about zero-based numbering of step (#1489)
  • Add links to examples from the integration references (#1507)
  • Fix broken link in plot_edf (#1510)
  • Update docs of default importance evaluator (#1524)

Examples

  • Set timeout for relatively long-running examples (#1349)
  • Fix broken link to example and add README for AllenNLP examples (#1397)
  • Add whitespace before opening parenthesis (#1398)
  • Fix GPU run for PyTorch Ignite and Lightning examples (#1444, thanks @arisliang!)
  • Add Stable-Baselines3 RL Example (#1420, thanks @araffin!)
  • Replace suggest_*uniform in examples with suggest_(int|float) (#1470)

Tests

  • Fix plot_param_importances test (#1328)
  • Fix incorrect test names in test_experimental.py (#1332, thanks @harupy!)
  • Simplify decorator tests (#1423)
  • Add a test for CmaEsSampler._get_trials() (#1433)
  • Use argument of pytorch_lightning.Trainer to disable checkpoint_callback (#1453)
  • Install RDB servers and their bindings for storage tests (#1497)
  • Upgrade versions of pytorch and torchvision (#1502)
  • Make HPI tests deterministic (#1505)

Code Fixes

  • Introduces optuna._imports.try_import to DRY optional imports (#1315)
  • Friendlier error message for unsupported plotly versions (#1338)
  • Rename private modules in optuna.visualization (#1359)
  • Rename private modules in optuna.pruners (#1361)
  • Rename private modules in optuna.samplers (#1362)
  • Change logger to _trial's module variable (#1363)
  • Remove deprecated features in HyperbandPruner (#1366)
  • Add missing __init__.py files (#1367, thanks @harupy!)
  • Fix double quotes from Black formatting (#1372)
  • Rename private modules in optuna.storages (#1373)
  • Add a list of functions and classes for each module in reference doc (#1400)
  • Apply deprecation decorator (#1413)
  • Remove unnecessary exception handling for GridSampler (#1416)
  • Remove either warnings.warn() or optuna.logging.Logger.warning() from codes which have both of them (#1421)
  • Simplify usage of deprecated by omitting removed version (#1422)
  • Apply experimental decorator (#1424)
  • Fix the experimental warning message for CmaEsSampler (#1432)
  • Remove optuna.structs from MLflow integration (#1437)
  • Add type hints to slice.py (#1267, thanks @bigbird555!)
  • Add type hints to intermediate_values.py (#1268, thanks @bigbird555!)
  • Add type hints to optimization_history.py (#1269, thanks @bigbird555!)
  • Add type hints to utils.py (#1270, thanks @bigbird555!)
  • Add type hints to test_logging.py (#1284, thanks @bigbird555!)
  • Add type hints to test_chainer.py (#1286, thanks @bigbird555!)
  • Add type hints to test_keras.py (#1287, thanks @bigbird555!)
  • Add type hints to test_cma.py (#1288, thanks @bigbird555!)
  • Add type hints to test_fastai.py (#1289, thanks @bigbird555!)
  • Add type hints to test_integration.py (#1293, thanks @bigbird555!)
  • Add type hints to test_mlflow.py (#1322, thanks @bigbird555!)
  • Add type hints to test_mxnet.py (#1323, thanks @bigbird555!)
  • Add type hints to optimize.py (#1364, thanks @bigbird555!)
  • Replace suggest_*uniform in examples with suggest_(int|float) (#1470)
  • Add type hints to distributions.py (#1513)
  • Remove unnecessary FloatingPointDistributionType (#1516)

Continuous Integration

  • Add a step to push images to Docker Hub (#1295)
  • Check code coverage in tests-python37 on CircleCI (#1348)
  • Stop building Docker images in Pull Requests (#1389)
  • Prevent doc-link from running on unrelated status update events (#1410, thanks @harupy!)
  • Avoid latest ConfigSpace where Python 3.5 is dropped (#1471)
  • Run unit tests on GitHub Actions (#1352)
  • Use circleci/python for dev image and install RDB servers (#1495)
  • Install RDB servers and their bindings for storage tests (#1497)
  • Fix dockerimage.yml format (#1511)
  • Revert #1495 and #1511 (#1512)
  • Run daily unit tests (#1515)

Other

  • Add TestPyPI release to workflow (#1291)
  • Add PyPI release to workflow (#1306)
  • Exempt issues with no-stale label from stale bot (#1321)
  • Remove stale labels from Issues or PRs when they are updated or commented on (#1409)
  • Exempt PRs with no-stale label from stale bot (#1427)
  • Update the documentation section in CONTRIBUTING.md (#1469, thanks @harupy!)
  • Bump up version to 2.0.0 (#1525)
Assets 2