# Difficult to build learning curve for a simple linear regression

In some part of my online ML course, I should build a code for plotting a learning curve, to be more specific, it's just a linear regression implementation.

So the first thing to consider is the expressions of training and Cross-validation error:

$$J_{train} = \frac{1}{2.m_{train}}\sum_{i=1}^n (h_{\theta}(x^{(i)} - y^{(i)})^2$$ $$J_{CV} = \frac{1}{2.m_{CV}}\sum_{i=1}^n (h_{\theta}(x^{(i)} - y^{(i)})^2$$

the instructor give for my the following instruction:

%Note: You should evaluate the training error on the first i training
%       examples (i.e., X(1:i, :) and y(1:i)).
%
%       For the cross-validation error, you should instead evaluate on
%       the _entire_ cross validation set (Xval and yval).
%
% Note: If you are using your cost function (linearRegCostFunction)
%       to compute the training and cross validation error, you should
%       call the function with the lambda argument set to 0.
%       Do note that you will still need to use lambda when running
%       the training to obtain the theta parameters.
%
% Hint: You can loop over the examples with the following:
%
%       for i = 1:m
%           % Compute train/cross validation errors using training examples
%           % X(1:i, :) and y(1:i), storing the result in
%           % error_train(i) and error_val(i)
%           ....
%
%       end


And then, in accordance what I understood, I build my MATLAB code:

function [error_train, error_val] = ...
learningCurve(X, y, Xval, yval, lambda)
grad = 0;
theta = trainLinearReg(Xval,yval,lambda);
m = size(X,1);
h_theta = X*theta;

for i = 1:m

error_train(i) = linearRegCostFunction(X(1:i,:), y(1:i,:), theta, 0);
error_val(i) = linearRegCostFunction(Xval(1:i,:), yval(1:i,:), theta, 0);

end


And I got the following result as learning curves:

I think that there's something wrong and I got some misunderstanding about this subject because second the instructor the learning curves should look like:

Could someone help me understand what I did or understood wrongly?

LinearRegFunc:

function [J, grad] = linearRegCostFunction(X, y, theta, lambda)
J = 0;
grad = zeros(size(theta));

h_theta  = X*theta;

J = (1/(2*m))*sum((h_theta - y).^2,1) + (lambda/(2*m))*sum(theta.^2,1);

grad(1) = (1/m)*sum((h_theta - y)'*X(:,1));
grad(2) = (1/m)*(h_theta - y)'*X(:,2)  + sum((lambda/m).*theta);


## 1 Answer

Look a the CV description I just posted at SE CV. In the first sentence there is a link to Kohavi, which explains bootstrap bias, or estimating error as a function of increasing sample size -- which is what you want.