Skip to content

Commit be2e6aa

Browse files
lawrenndlawrennd
committed
Merge remote-tracking branch 'origin/master'
2 parents 063f217 + 068681b commit be2e6aa

38 files changed

+786
-208
lines changed

‎README.md

+31
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
2+
Matlab Noise Models Toolbox
3+
===========================
4+
5+
6+
Release Information
7+
-------------------
8+
9+
**Current release is 0.141**.
10+
11+
Added noiseReadFromFile and noiseReadFromFID for compatability with CPP releases.
12+
13+
#### Version 0.14
14+
15+
As of version 0.14, the NCNM noise model has been incorporated into this toolbox.
16+
17+
This toolbox allows computation of several different noise models and their gradients. You can add noise models to the toolbox by creating versions of the relevant files. Once added, they can be tested using the `noiseTest`. For example you can test the probit noise model by writing
18+
19+
```
20+
>> noiseTest('probit')
21+
```
22+
23+
There are several noise models implemented, the mains ones that are being maintained for the latest release are:
24+
25+
`gaussian ordered probit ncnm`
26+
27+
These noise models are Gaussian (standard Gaussian noise), probistic regression for binary classificaiton and ordered categorical for ordered categories (ordinal regression).
28+
29+
Page updated on Tue Oct 6 17:28:55 2009
30+
31+

‎matlab/additionalfiles.txt

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# These are files that should be packaged for the drawing software
2+
dir: html
3+
~/mlprojects/noise/html/index.html
4+
dir: util
5+
~/mlprojects/noise/matlab/noiseGenerator.py

‎matlab/clutterDir/noiseVers.m

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
function [vers, depend] = noiseVers
2+
3+
% NOISEPATH Brings dependent toolboxes into the path.
4+
5+
vers = 0.13;
6+
if nargout > 2
7+
depend(1).name = 'ndlutil';
8+
depend(1).vers = 0.13;
9+
depend(1).required = 0;
10+
end

‎matlab/cmpnd.txt

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
% The compound noise model is a wrapper noise model for allowing each output
2+
% of a the model to have a different noise model.
3+
%
4+
% SEEALSO : cmpndKernParamInit

‎matlab/cmpndNoiseNuG.template

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
function [g, nu] = cmpndNoiseNuG(noise, x)
2+
3+
% CMPNDNOISENUG Compute nu and g for CMPND noise model.
4+
% FORMAT
5+
% DESC computes the values nu and g for the compound noise given the mean and variance inputs as well as the output of the noise model.
6+
% ARG noise : the noise structure for which the nu and g are computed.
7+
% ARG mu : input mean to the noise model.
8+
% ARG varSigma : input variance to the noise model.
9+
% ARG y : target output for the noise model.
10+
% RETURN g : the vector g, which is the gradient of log Z with respect to the input mean.
11+
% ARG y : target output for the noise model.
12+
% RETURN nu : the vector nu, see equation 10 of "Extensions of the Informative Vector Machine".
13+
%
14+
% SEEALSO : cmpndNoiseParamInit, noiseUpdateNuG, noiseCreate
15+
%
16+
% COPYRIGHT : Neil D. Lawrence, 2004, 2005
17+
18+
% NOISE
19+

‎matlab/cmpndNoiseSites.template

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
function [m, beta] = cmpndNoiseSites(noise, g, nu, mu, varSigma, y)
2+
3+
% CMPNDNOISESITES Update the site parameters for the CMPND noise mode.
4+
% FORMAT
5+
% DESC updates the site parameters for the compound
6+
% noise model.
7+
% ARG noise : the noise structure for which the site parameters are to
8+
% be updated.
9+
% ARG g : values of g as retuned by cmpndNoiseNuG.
10+
% ARG nu : values of nu as retuned by cmpndNoiseNuG.
11+
% ARG mu : the mean value of the Gaussian input to the noise structure.
12+
% ARG varSigma : the variance of the Gaussian input to the noise structure.
13+
% ARG y : the target value.
14+
% RETURN m : the site mean parameters.
15+
% RETURN beta : the site precision parameters.
16+
%
17+
% SEEALSO : cmpndNoiseParamInit, noiseUpdateSites
18+
%
19+
% COPYRIGHT : Neil D. Lawrence, 2004, 2005
20+
21+
% NOISE
22+

‎matlab/diagnostics.dat

+57
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
Files not tagged as in toolbox:
2+
gaussianNoiseExpectationLogLikelihood.m
3+
4+
Files not commented:
5+
6+
Files not in CVS or SVN:
7+
8+
Dependent toolbox problems
9+
File Comments Removed
10+
if any(isnan(dlnZ_dmu))
11+
removed from exported ncnmNoiseGradVals.m
12+
warning('dlnZ_dmu is NaN')
13+
removed from exported ncnmNoiseGradVals.m
14+
end
15+
removed from exported ncnmNoiseGradVals.m
16+
if any(isnan(dlnZ_dvs))
17+
removed from exported ncnmNoiseGradVals.m
18+
warning('dlnZ_dvs is NaN')
19+
removed from exported ncnmNoiseGradVals.m
20+
end
21+
removed from exported ncnmNoiseGradVals.m
22+
if any(isnan(dlnZ_dmu))
23+
removed from exported orderedNoiseGradVals.m
24+
warning('dlnZ_dmu is NaN')
25+
removed from exported orderedNoiseGradVals.m
26+
end
27+
removed from exported orderedNoiseGradVals.m
28+
if any(isnan(dlnZ_dvs))
29+
removed from exported orderedNoiseGradVals.m
30+
warning('dlnZ_dvs is NaN')
31+
removed from exported orderedNoiseGradVals.m
32+
end
33+
removed from exported orderedNoiseGradVals.m
34+
if any(isnan(dlnZ_dmu))
35+
removed from exported noiseGradVals.m
36+
warning('Gradient of mu is NaN')
37+
removed from exported noiseGradVals.m
38+
end
39+
removed from exported noiseGradVals.m
40+
if any(isnan(dlnZ_dvs))
41+
removed from exported noiseGradVals.m
42+
warning('Gradient of varsigma is NaN')
43+
removed from exported noiseGradVals.m
44+
end
45+
removed from exported noiseGradVals.m
46+
if any(isnan(dlnZ_dmu))
47+
removed from exported mgaussianNoiseGradVals.m
48+
warning('dlnZ_dmu is NaN')
49+
removed from exported mgaussianNoiseGradVals.m
50+
end
51+
removed from exported mgaussianNoiseGradVals.m
52+
if any(isnan(dlnZ_dvs))
53+
removed from exported mgaussianNoiseGradVals.m
54+
warning('dlnZ_dvs is NaN')
55+
removed from exported mgaussianNoiseGradVals.m
56+
end
57+
removed from exported mgaussianNoiseGradVals.m

‎matlab/gaussian.txt

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
% The noiseless Gaussian noise model is simply the Gaussian noise model with
2+
% the variance of the Gaussian set very small.
3+
%
4+
% SEEALSO : gaussianParamInit

‎matlab/gaussianNoiseExpectationLogLikelihood.m

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
function L = gaussianNoiseExpectationLogLikelihood(noise, mu, varsigma, y)
2+
3+
% GAUSSIANNOISEEXPECTATIONLOGLIKELIHOOD Likelihood of the data under the GAUSSIAN noise model.
4+
% FORMAT
5+
% DESC returns the likelihoods for data points under the Gaussian noise model.
6+
% ARG noise : the noise structure for which the likelihood is required.
7+
% ARG mu : input mean locations for the likelihood.
8+
% ARG varSigma : input variance locations for the likelihood.
9+
% ARG y : target locations for the likelihood.
10+
%
11+
% SEEALSO : gaussianNoiseParamInit, gaussianNoiseLogLikelihood, noiseLikelihood
12+
%
13+
% COPYRIGHT : Neil D. Lawrence, 2004, 2005
14+
15+
16+
17+
N = size(y, 1);
18+
D = size(y, 2);
19+
varsigma = varsigma + noise.sigma2;
20+
for i = 1:D
21+
mu(:, i) = mu(:, i) + noise.bias(i);
22+
end
23+
arg = (mu - y)./sqrt(varsigma);
24+
L = (2*pi*varsigma).^(-1/2).*exp( - .5*arg.*arg);

‎matlab/heavisideNoise/heavisideLikelihood.m

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
function L = heavisideLikelihood(noise, mu, varsigma, y)
2+
3+
% HEAVISIDELIKELIHOOD Likelihood of data under heaviside noise model.
4+
5+
% IVM
6+
7+
D = size(y, 2);
8+
L = zeros(size(mu));
9+
for i = 1:D
10+
mu(:, i) = mu(:, i) + noise.bias(i);
11+
L(:, i) = (1-2*noise.eta)...
12+
*cumGaussian((y(:, i).*mu(:, i))...
13+
./(sqrt(varsigma(:, i))))+noise.eta;
14+
end

‎matlab/heavisideNoise/heavisideLogLikelihood.m

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
function L = heavisideLogLikelihood(noise, mu, varsigma, y)
2+
3+
% HEAVISIDELOGLIKELIHOOD Log-likelihood of data under heaviside noise model.
4+
5+
% IVM
6+
7+
% limVal = 36;
8+
% fact = sqrt(2)/2;
9+
% D = size(y, 2);
10+
% L = 0;
11+
% for i = 1:D
12+
% u = mu(:, i) + noise.bias(i);
13+
% u = (y(:, i).*u)./(sqrt(varsigma(:, i)));
14+
% index = find(u > limVal);
15+
% L = L + length(index)*log(noise.eta);
16+
% u(index) = [];
17+
% u2 = u.*u;
18+
% L = L - sum(.5*u2) + sum(log((1-2*noise.eta)...
19+
% *.5*erfcx(-fact*u)...
20+
% +exp(.5*u2)*noise.eta));
21+
% end
22+
L = sum(sum(log(heavisideLikelihood(noise, mu, varsigma, y))));

‎matlab/heavisideNoise/heavisideNoiseDisplay.m

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
function heavisideNoiseDisplay(noise)
2+
3+
% HEAVISIDENOISEDISPLAY Display the parameters of the Heaviside noise model.
4+
5+
% IVM
6+
7+
for i = 1:noise.numProcess
8+
fprintf('Heaviside bias on process %d: %2.4f\n', i, noise.bias(i))
9+
end
10+
fprintf('Heaviside label noise: %2.4f\n', noise.eta);

‎matlab/heavisideNoise/heavisideNoiseExpandParam.m

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
function noise = heavisideNoiseExpandParam(noise, params)
2+
3+
% HEAVISIDENOISEEXPANDPARAM Expand heaviside noise structure from param vector.
4+
5+
% IVM
6+
7+
8+
noise.eta = 0.5*sigmoidBound(params(1));
9+
noise.bias = params(2:end);
10+

‎matlab/heavisideNoise/heavisideNoiseExtractParam.m

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
function [params, names] = heavisideNoiseExtractParam(noise)
2+
3+
% HEAVISIDENOISEEXTRACTPARAM Extract parameters from heaviside noise model.
4+
5+
% IVM
6+
7+
params = [invSigmoid(2*noise.eta) noise.bias];
8+
9+
if nargout > 1
10+
names{1} = 'inv logistic of eta';
11+
for i = 1:noise.numProcess
12+
names{1+i} = ['bias ' num2str(i)];
13+
end
14+
end

‎matlab/heavisideNoise/heavisideNoiseGradVals.m

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
function [dlnZ_dmu, dlnZ_dvs] = heavisideNoiseGradVals(noise, mu, varsigma, y)
2+
3+
% HEAVISIDENOISEGRADVALS Gradient wrt mu and varsigma of log-likelihood for heaviside noise model.
4+
5+
% IVM
6+
7+
D = size(mu, 2);
8+
c = y./sqrt(varsigma);
9+
u = zeros(size(c));
10+
dlnZ_dmu = zeros(size(c));
11+
for i = 1:D
12+
u(:, i) = c(:, i).*(mu(:, i) + noise.bias(i));
13+
dlnZ_dmu(:, i) = c(:, i).*ngaussian(u(:, i))...
14+
./(cumGaussian(u(:, i))+ ...
15+
noise.eta/(1-2*noise.eta));
16+
end
17+
dlnZ_dvs = -.5*c.*u.*dlnZ_dmu;

‎matlab/heavisideNoise/heavisideNoiseGradientParam.m

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
function g = heavisideNoiseGradientParam(noise, mu, varsigma, y)
2+
3+
% HEAVISIDENOISEGRADIENTPARAM Gradient of the heaviside noise model's parameters.
4+
5+
% IVM
6+
7+
c = y./sqrt(varsigma);
8+
denom = zeros(size(c));
9+
u = zeros(size(c));
10+
for i = 1:size(mu, 2)
11+
u(:, i) = c(:, i).*(mu(:, i) + noise.bias(i));
12+
denom(:, i) = ((1-2*noise.eta)*cumGaussian(u(:, i))+noise.eta);
13+
end
14+
15+
gnoise.bias = (1-2*noise.eta).*sum(c.*ngaussian(u)./denom, 1);
16+
gnoise.eta = sum(sum((-2*cumGaussian(u)+1)./denom, 1));
17+
gnoise.eta = gnoise.eta.*(noise.eta.*(1-2*noise.eta));
18+
g = [gnoise.eta gnoise.bias];

‎matlab/heavisideNoise/heavisideNoiseOut.m

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
function y = heavisideNoiseOut(noise, mu, varsigma)
2+
3+
% HEAVISIDENOISEOUT Output from heaviside noise model.
4+
5+
% IVM
6+
D = size(mu, 2);
7+
for i = 1:D
8+
mu(:, i) = mu(:, i) + noise.bias(i);
9+
end
10+
y = sign(mu);

‎matlab/heavisideNoise/heavisideNoiseParamInit.m

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
function noise = heavisideNoiseParamInit(noise, y)
2+
3+
% HEAVISIDENOISEPARAMINIT Heaviside classification model's parameter initialisation.
4+
5+
% IVM
6+
7+
if nargin > 1
8+
nClass1 = sum(y==1);
9+
nClass2 = sum(y==-1);
10+
noise.bias = invCumGaussian(nClass1./(nClass2+nClass1));
11+
noise.numProcess = size(y, 2);
12+
else
13+
noise.bias = zeros(1, noise.numProcess);
14+
end
15+
16+
noise.eta = 0.01;
17+
noise.nParams = length(noise.bias) + length(noise.eta);
18+

‎matlab/heavisideNoise/heavisideNoiseUpdateParams.m

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
function [nu, g] = heavisideNoiseUpdateParams(noise, mu, varsigma, y, index)
2+
3+
% HEAVISIDENOISEUPDATEPARAMS Update parameters for heaviside noise model.
4+
5+
% IVM
6+
7+
c = y(index, :)./sqrt(varsigma(index, :));
8+
u = zeros(size(c));
9+
for i = 1:size(mu, 2)
10+
u(:, i) = c(:, i).*(mu(index, i) + noise.bias(i));
11+
g(:, i) = c(:, i).*ngaussian(u(:, i))...
12+
./(cumGaussian(u(:, i))+ ...
13+
noise.eta(i)/(1-2*noise.eta(i)));
14+
end
15+
nu = g.*(g + c.*u);
16+

‎matlab/mgaussian.txt

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
% This noise model is simply a Gaussian noise that allows different
2+
% variances when multiple outputs are used.
3+
%
4+
% SEEALSO : gaussianNoiseParamInit

‎matlab/mgaussianNoiseNuG.template

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
function [g, nu] = mgaussianNoiseNuG(noise, x)
2+
3+
% MGAUSSIANNOISENUG Compute nu and g for MGAUSSIAN noise model.
4+
% FORMAT
5+
% DESC computes the values nu and g for the multiple output Gaussian noise given the mean and variance inputs as well as the output of the noise model.
6+
% ARG noise : the noise structure for which the nu and g are computed.
7+
% ARG mu : input mean to the noise model.
8+
% ARG varSigma : input variance to the noise model.
9+
% ARG y : target output for the noise model.
10+
% RETURN g : the vector g, which is the gradient of log Z with respect to the input mean.
11+
% ARG y : target output for the noise model.
12+
% RETURN nu : the vector nu, see equation 10 of "Extensions of the Informative Vector Machine".
13+
%
14+
% SEEALSO : mgaussianNoiseParamInit, noiseUpdateNuG, noiseCreate
15+
%
16+
% COPYRIGHT : Neil D. Lawrence, 2004, 2005
17+
18+
% NOISE
19+

‎matlab/mgaussianNoiseSites.template

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
function [m, beta] = mgaussianNoiseSites(noise, g, nu, mu, varSigma, y)
2+
3+
% MGAUSSIANNOISESITES Update the site parameters for the MGAUSSIAN noise mode.
4+
% FORMAT
5+
% DESC updates the site parameters for the multiple output Gaussian
6+
% noise model.
7+
% ARG noise : the noise structure for which the site parameters are to
8+
% be updated.
9+
% ARG g : values of g as retuned by mgaussianNoiseNuG.
10+
% ARG nu : values of nu as retuned by mgaussianNoiseNuG.
11+
% ARG mu : the mean value of the Gaussian input to the noise structure.
12+
% ARG varSigma : the variance of the Gaussian input to the noise structure.
13+
% ARG y : the target value.
14+
% RETURN m : the site mean parameters.
15+
% RETURN beta : the site precision parameters.
16+
%
17+
% SEEALSO : mgaussianNoiseParamInit, noiseUpdateSites
18+
%
19+
% COPYRIGHT : Neil D. Lawrence, 2004, 2005
20+
21+
% NOISE
22+

0 commit comments

Comments
 (0)