cloud_ml_engine scaleType implementation

Question

I'm trying to implement the functionality of google cloud ml engine's hyperparameter tuning using scikit-optimize (skopt). I'm unsure how to consistently convert ml-engine's scaleType to skopt.space.Real's priors.

Uniform is straight forward enough, and log-uniform looks to have an equivalent in each - but I'm not entirely sure the implementation is consistent. I'm also unsure how to implement ml engine's UNIT_REVERSE_LOG_SCALE if the LOG_SCALE is consistent with skopt's log-uniform prior. The log-uniform distribution doesn't seem to behave as one might like for parameters away from 0 - e.g. if you want to scale between 0.9 and 0.999 the distribution is close to uniform (see first plot below).

Code and visualization using skopts log-uniform and a couple of custom transformations below.

#!/usr/bin/python
import numpy as np
import matplotlib.pyplot as plt
import skopt


def sample_custom_log_uniform(min_val, max_val, n_samples):
    sample = skopt.space.uniform(0, 1).rvs(n_samples)
    return min_val + (10 ** sample - 1) / 9 *(max_val - min_val)


def sample_custom_reverse_log_uniform(min_val, max_val, n_samples):
    sample = skopt.space.uniform(0, 1).rvs(n_samples)
    return max_val - (10 ** sample - 1) / 9 *(max_val - min_val)


def sample(min_val, max_val, prior='log-uniform', n_samples=100000):
    if prior == 'custom-log-uniform':
        return sample_custom_log_uniform(min_val, max_val, n_samples)
    elif prior == 'custom-reverse-log-uniform':
        return sample_custom_reverse_log_uniform(min_val, max_val, n_samples)
    else:
        return skopt.space.Real(min_val, max_val, prior=prior).rvs(n_samples)


priors = (
    'log-uniform', 'custom-log-uniform', 'custom-reverse-log-uniform')
fig, axes = plt.subplots(1, len(priors))
for (prior, ax) in zip(priors, axes):
    ax.hist(sample(0.9, 0.999, prior))
    ax.set_title(prior)
    ax.set_yticklabels([])

plt.show()

My questions are:

• does ml engine implement LOG_SCALE as log-uniform or custom-log-uniform from the above, or something else?
• does ml engine implement REVERSE_LOG_SCALE as custom-reverse-log-uniform above, or something else?

Answer 1

For a parameter with feasible region [a, b]: UNIT_LOG_SCALE scales the feasible space logarithmically to [0, 1]. This maps a value x to log(x / a) / log(b / a). UNIT_REVERSE_LOG_SCALE scales the feasible space "reverse" logarithmically to [0, 1]. This maps a value x to 1.0 - log((b + a - x) / a) / log(b / a).