Pasky’s Log

In part of my research, I have been heavily involved with building portfolios of optimization algorithms. Optimization algorithms stay at the root of many computational tasks, from designing laser mirror systems to neural network training. We want to find a minimum (or maximum) of some mathematical function, and for some functions it’s easier than for others.

For very many fairly hairy functions, the best state-of-art optimization algorithm is based on genetic algorithms and it’s called CMA-ES. It also has a very nice Python implementation by its original author, Nikolaus Hansen.

CMA-ES is still not as good as it could be on some functions with many local optima, but its performance can be much improved by establishing a restart strategy that will repeatedly restart it with varying population size and parameters. The best performing restart strategy is BIPOP-CMA-ES and unfortunately, it had no Python implementation so far. I took care of that more than a month ago, but since it’s taking some time to get my modifications upstreamed, if anyone would find that useful,

here is a patch for CMA-1.1.02 adding BIPOP restart strategy