What am I doing wrong with hw5.3?

14
1

I wont place my code, but I basically just took over what was shown in the video for hw5-3.

Funnily enough I get:

[[0, 0], 
[5.1257557637014316e+307, 3.6054863890688511e+306], 
[2.3213664369024311e+307, 1.6328626407199667e+306], 
[6.7824048353802897e+305, 4.7707829596773796e+304], 
[-2.235757895364848e+307, -1.5726450954407239e+306], 
[-5.2222802968343177e+307, -3.673382306976488e+306], 
[6, 0], 
[8.9205188647975054e+307, 6.27474480580988e+306], 
[8.8580231942965764e+307, 6.2307849880222052e+306], 
[6, 3], 
[-5.2032199824328205e+307, -3.6599751710659726e+306], 
[-2.3564486463998559e+307, -1.6575396710368467e+306], 
[-6.8849055672202959e+305, -4.8428825966235518e+304], 
[2.2695463250431335e+307, 1.596412072914244e+306],
[5.3012032656660389e+307, 3.7288971814432291e+306], 
[0, 3], 
[-9.0553323547470735e+307, -6.3695733972873715e+306], 
[nan, -6.3249492261720942e+306]]

I have tried to vary the style a bit to get different results, but I keep getting similar issues.

If I just check Thrun's formula I think there is nothing wrong with it, but a direct implementation seem to be not working.

Anyone that has seen something similar or just has a general tip for what I am doing wrong?

asked 20 Mar '12, 12:22

Ben%20Haanstra's gravatar image

Ben Haanstra
5003417
accept rate: 50%

edited 20 Mar '12, 14:04

Gundega's gravatar image

Gundega ♦♦
44.0k70170315

Could you format your output please? Select your output and hit the 1010 button.

(20 Mar '12, 12:38) yoshi yoshi's gravatar image
6

Alright, I see what I missed there. Tip for the next person: look in the notes for the word: AND.

D'oh!

(20 Mar '12, 13:23) Ben Haanstra Ben%20Haanstra's gravatar image

Just a clarification needed - when Prof says 'points like these' did he mean points either side of the fixed points? He speaks faster at times and I am listening it repeatedly but he remains elusive ;)

(20 Mar '12, 16:57) rakesh kumar rakesh%20kumar's gravatar image
1

No, rukacity, he means ALL non-fixed points. I admit, at first it was a little confusing.

(24 Mar '12, 17:01) Gibgezr Gibgezr's gravatar image

9 Answers:

Okay, I just spent a large amount of time facing the same problem and I finally solved it. The new update procedures that Sebastian talks about in the homework video are to be added to the old update procedures.

NOTE: The new equations have to be added, they don't replace either of the original update steps

If you do what I did, which was to replace one of the original steps, then your answer will blow up to a really large number. Judging by your output, that's exactly what happened.

link

answered 20 Mar '12, 13:49

yoshi's gravatar image

yoshi
4.0k82659

And I just noticed you solved your own problem 25 mins before I did. Yay! So much time I will never get back.

(20 Mar '12, 14:03) yoshi yoshi's gravatar image
1

The weight_data is set to 0.0 by default. Does that need to be changed?

(20 Mar '12, 19:09) james gallagher james%20gallagher's gravatar image

@bayowulf... I belive no, but I'm not sure.

(20 Mar '12, 22:21) yoshi yoshi's gravatar image
1

@yoshi ... weight_data=0.0 seems to work. I passed the 2 test cases using the default for weight_data=0.0. I probably failed a few attempts because I had changed it to 0.1. I suppose that since four data points are totally fixed there is no need for the remainder to influence the curve.

(21 Mar '12, 08:48) james gallagher james%20gallagher's gravatar image

Thanks @bayowulf, @yoshi! I have the habit of copying other examples and modifying them to work, but I tend to miss tiny differences like that. Spent a bunch of time trying to figure out what was wrong when I saw your comments.

(23 Mar '12, 16:40) Ron Cococcia Ron%20Cococcia's gravatar image

I thought I was modifying the new data points, but found out I was clobbering my old value and not actually updating properly. This caused my answers to grow very quickly. This helped me find the bug in my logic!

(24 Mar '12, 13:38) JakeL JakeL's gravatar image

@yoshi - I wish I could buy you two tickets to that thing you like - thank you. I was so close to the answer before this dawned on me. Thank you thank you thank you! This question was tricky for me because it was hard to debug the intermediate steps, but what I did managed to do was print them out for say, step 3, 4, or 5 incrementally (as the whole output was unwieldy) and watch them approach the correct vales.

(25 Mar '12, 07:47) Aaron Newton Aaron%20Newton's gravatar image

Thank you, thank you.

That makes all the difference. Why didn't I check the forum before pulling my hair out for hours trying to see where I had gone wrong!

(25 Mar '12, 23:33) Howard Howard's gravatar image

Thank you, thank you!

I should learn to check the forum earlier when I am tearing my hair out over a problem for hours.
And I should read the comments that Sebastian puts at the top of the code!

(25 Mar '12, 23:43) Howard Howard's gravatar image

Ditto.
Thank You. But Sebastian's comments are a bit cryptic…. the word AND - ouch.

(26 Mar '12, 14:08) DaveAppleton DaveAppleton's gravatar image

Read the comments on top of the code, they say apply ... and the formulas presented in the video. Also consider http://www.udacity-forums.com/cs373/questions/24337/homework-5-3-update-formulas

link

answered 20 Mar '12, 12:35

Ruslan%20Ciurca's gravatar image

Ruslan Ciurca
2.9k72125

Thanks!! I didn't read it carefully. That was precisely the problem. I hate when I do this :)

(20 Mar '12, 12:41) Guillermo Go... Guillermo%20Gonzalez's gravatar image
1

Lol, Ruslan, you saved me again! Minus one for me for lack of reading skills.

To the original person who wrote the question: I got similar numbers until I read the line "# You will need to use ..." and followed those instructions carefully.

(20 Mar '12, 12:54) Steve Carlin-1 Steve%20Carlin-1's gravatar image

Do we need to use the equations from gradient descent for fixed points also?
And I understand from other thread that new equations doesn't apply for fixed points.
Is both those statements correct?

(20 Mar '12, 13:30) Balaji Sunda... Balaji%20Sundaresan's gravatar image

Sebatian implements that in his code as well with

if fix[i]==0
(24 Mar '12, 12:58) yoshi yoshi's gravatar image

Answer is to look in the notes of the program itself.

link

answered 20 Mar '12, 13:25

Ben%20Haanstra's gravatar image

Ben Haanstra
5003417

So can somebody explain why halving the factor (smoothing weight) makes such a huge difference? I know that a gradient descent can wildly oscillate if the descent factor is large, where it keeps missing the local minima and climbing up - is this what's happening here? If that's the case, it shouldn't converge - but it manages to converge with such large values.

link

answered 20 Mar '12, 22:25

Max%20Mir's gravatar image

Max Mir
42451018

1

Because since the updates are not happening simultaneously, you end up minimising a different function for each different weight parameter

(21 Mar '12, 02:06) Alejandro Du... Alejandro%20Dubrovsky-1's gravatar image

I've got a similar problem. My error starts decreasing but, when it reaches around 0.33 it starts to diverge. I've actually plotted the results and it looks like the cornering moving points are being pulled away more or less as you would expect but the points at x=3 are not moving away. So, instead of looking like an oval it appears to be converting to a horizontal "8" so to speak. Beyond that point, everything starts to get crazy and error goes to "infinity and beyond".

I've watched the video quite a few times and I just don't get what I may have overlooked. It looks like a simple change in the formulas when updating the non-fixed points but it is not reaching a solution. Any idea what could cause it to diverge?

link

answered 20 Mar '12, 12:32

Guillermo%20Gonzalez's gravatar image

Guillermo Go...
301213

You have the formulas wrong! Check all the signs carefully and check that you are using the right beta. Many times in my testing, it would diverge and never stop when I had the formulas wrong. When you have them correct, it converges.

(20 Mar '12, 19:29) Curt Welch Curt%20Welch's gravatar image

Could anyone explain why in hw5-3 we need to compare the sum of change in both coordinate deviations with the tolerance, but in hw5-2 we do not

link

answered 21 Mar '12, 13:05

Anh%20Nguyen's gravatar image

Anh Nguyen
3317

I believe, this is the private case. In hw5-3 we added smoothing around the fixed points and the deviation of y coordinate of the path last point became an inappropriate convergence criterion (less iterations, insufficient smoothing). Therefore, the sum of coordinate errors (but for the last point i=17 as before) has been chosen. And because we have to match the test, we should use the same criterion in our code :) I think there are many ways to improve the code for the real life :)

(22 Mar '12, 07:03) Sergey Radzh... Sergey%20Radzhabov's gravatar image

I suppose that we do this in hw5-2 too.

(23 Mar '12, 19:09) Svyatoslav Z... Svyatoslav%20Zarutskiy's gravatar image

If you are getting huge numbers or nan values, make sure you ADDED the new code to the old code and didn't REPLACE the old code with the new code.

link

answered 24 Mar '12, 17:00

Mikey's gravatar image

Mikey
33469

can u tell me the new equations which were given in class...
the video didn't run completely in my comp due to .dll error....

(25 Mar '12, 01:14) vipul divyanshu vipul%20divyanshu's gravatar image

I had exactly the same problem - took me a while to figure out what's wrong.
It could have taken much less time if I read forums!

link

answered 25 Mar '12, 13:10

Vitaly%20Lishchenko's gravatar image

Vitaly Lishc...
231

Guys, I understand that we have to apply BOTH the new gradient descent AND the old equations, and I also understand that we ONLY APPLY THEM TO THE NON FIXED POINTS, but my question is:

In what order do I apply the new gradient descent equations with the old ones?

In that, in the old way, I have eq1, eq2 being done.

Now new equations are eq3 and eq4.

So we just apply eq1, eq2, eq3, eq4 in that order??

Thanks

link

answered 25 Mar '12, 21:08

Terran's gravatar image

Terran
2.8k244769

1

Yes, in that order. But don't forget to update newpath after each equation and not just at the end.

(25 Mar '12, 21:35) Mikey Mikey's gravatar image

and take care of the weights. Prof does mention that the new euations need to split the weights so that they add to 1.

(25 Mar '12, 22:24) rakesh kumar rakesh%20kumar's gravatar image

@rukacity ... the weights?...

(25 Mar '12, 22:56) Terran Terran's gravatar image
Your answer
Question text:

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "Title")
  • image?![alt text](/path/img.jpg "Title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Tags:

×5,185
×69

Asked: 20 Mar '12, 12:22

Seen: 2,203 times

Last updated: 06 Apr '12, 13:41