To find out which algorithm is buggy I have run linear regression algorithm of data mining software Weka on the same data. New normalEque function on new data that are not close to singular gives different theta as gradientDesc. Then I changed normalEque code as suggested by woodchips so now it looks like: function theta = normalEque(X, y) So I hope that X is not close to singular any more. This time inv(X) did not display the warning and using pinv and inv gave same results. To be sure that that is not the problem I obtained much larger dataset and run tests with this new dataset. So I replaced pinv by inv - and when run it I really got warning Matrix is close to singular or badly scaled. Edit 1:īased on answers and comments I checked few my code and run some tests.įirst I want to check if the problem can be cause by X beeing near singular as suggested by answer. I pass exactly the same data X and y to both functions (I do not normalize X). Options = optimset('GradObj', 'on', 'MaxIter', 9999) Here is code for gradient descent: function theta = gradientDesc(X, y) Here is my implementation of normal equation in Matlab: function theta = normalEque(X, y) What can cause the difference in theta(1, 1) returned by gradient descent compared to theta(1, 1) returned by normal equation? Do I have bug in my code? Here is how the thetas look like (fist column is output of Gradient descent, second output of Normal equation): Grad desc Norm eq That is the one used to multiply vector of all 1 added to the data. However they do not.īoth theta vectors are very similar on all elements but the first one. On the same data they should both give approximately equal theta vector. I have implemented 2 different methods to find parameters theta of linear regression model: Gradient (steepest) descent and Normal equation. I'm working on machine learning problem and want to use linear regression as learning algorithm.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. Archives
January 2023
Categories |