搜索
您的当前位置:首页正文

高斯过程Gaussian Processes

来源:意榕旅游网

https://zhuanlan.zhihu.com/p/75589452

http://bridg.land/posts/gaussian-processes-1

RW2: https:///weixin_40056577/article/details/112364503

Exploring Bayesian Optimization: https://distill.pub/2020/bayesian-optimization/

kernel

Kernel function kernel, in another word, covariance function is crucial for Gaussian process to represent a large repotoire of distributions of function. A common practice is that people use the automatic relevance determination (ARD) squared exponential kernel for Gaussian process regression. However, the concern is that sample functions with this covariance function are too smooth for real world optimization problems. As suggested by the author of this paper, we’ll choose Matern52 as our kernel, which is twice differentiable. The rationale behind this is this kernel can make sample functions corresponds to those made by quasi-Newton methods, and more close to real-world situation.

在这里插入代码片
from scipy.optimize import minimize
import numpy as np
import matplotlib.pyplot as plt

class GPR:

    def __init__(self, optimize=True):
        self.is_fit = False
        self.train_X, self.train_y = None, None
        self.params = {"l": 0.5, "sigma_f": 0.2}
        self.optimize = optimize

    def fit(self, X, y):
        # store train data
        self.train_X = np.asarray(X)
        self.train_y = np.asarray(y)
        self.is_fit = True

    def predict(self, X):
        if not self.is_fit:
            print("GPR Model not fit yet.")
            return

        X = np.asarray(X)
        Kff = self.kernel(self.train_X, self.train_X)  # (N, N)
        Kyy = self.kernel(X, X)  # (k, k)
        Kfy = self.kernel(self.train_X, X)  # (N, k)
        Kff_inv = np.linalg.inv(Kff + 1e-8 * np.eye(len(self.train_X)))  # (N, N)

        mu = Kfy.T.dot(Kff_inv).dot(self.train_y)
        cov = Kyy - Kfy.T.dot(Kff_inv).dot(Kfy)
        return mu, cov

    def kernel(self, x1, x2):
        dist_matrix = np.sum(x1 ** 2, 1).reshape(-1, 1) + np.sum(x2 ** 2, 1) - 2 * np.dot(x1, x2.T)
        return self.params["sigma_f"] ** 2 * np.exp(-0.5 / self.params["l"] ** 2 * dist_matrix)


def y(x, noise_sigma=0.0):
    x = np.asarray(x)
    y = np.cos(x) + np.random.normal(0, noise_sigma, size=x.shape)
    return y.tolist()


train_X = np.array([3, 1, 4, 5, 9]).reshape(-1, 1)
train_y = y(train_X, noise_sigma=1e-4)
test_X = np.arange(0, 10, 0.1).reshape(-1, 1)

gpr = GPR()
gpr.fit(train_X, train_y)
mu, cov = gpr.predict(test_X)
test_y = mu.ravel()
uncertainty = 1.96 * np.sqrt(np.diag(cov))
plt.figure()
plt.title("l=%.2f sigma_f=%.2f" % (gpr.params["l"], gpr.params["sigma_f"]))
plt.fill_between(test_X.ravel(), test_y + uncertainty, test_y - uncertainty, alpha=0.1)
plt.plot(test_X, test_y, label="predict")
plt.scatter(train_X, train_y, label="train", c="red", marker="x")
plt.legend()
plt.show()

Acquisition Functions for Bayesian Optimization

1. Probability of Improvement

2. Expected Improvement

3. GP Upper Confidence Bound


因篇幅问题不能全部显示,请点此查看更多更全内容

热门图文

Top