Homework Grading - Correct code graded as incorrect

I had correct code within the YOUR CODE block, however it was graded as incorrect. I copied this block of code out, reset the question, and pasted it back, and it passed, although now my mark is lower. Frustrating!

The only thing I changed was uncommenting the different scenario lines as required to evaluate different measurements, as shown in the video. In the end, I think I put a comment back with different content within the comment. Should this matter?

Originally:

# measurements = [[1., 4.], [6., 0.], [11., -4.], [16., -8.]]
# initial_xy = [-4., 8.]

When I submitted it (missing leading white space):

#measurements = [[1., 4.], [6., 0.], [11., -4.], [16., -8.]]
#initial_xy = [-4., 8.]

The observed results were identical to those shown by Sebastian during the explanatory video.

Could the homework evaluator please be changed to pull my block out, and run it within the pristine question context?


Here's my code block:

P = matrix([[0., 0., 0., 0.], 
            [0., 0., 0., 0.],
            [0., 0., 1000., 0.],
            [0., 0., 0., 1000.]]) # initial uncertainty
F = matrix([[1., 0., dt, 0.], 
            [0., 1., 0., dt],
            [0., 0., 1., 0.],
            [0., 0., 0., 1.]]) # next state function
H = matrix([[1., 0., 0., 0.],
            [0., 1., 0., 0.]]) # measurement function
R = matrix([[dt, 0.],
            [0., dt]]) # measurement uncertainty
I = matrix([[1., 0., 0., 0.],
            [0., 1., 0., 0.],
            [0., 0., 1., 0.],
            [0., 0., 0., 1.]]) # identity matrix

asked 07 Mar '12, 03:42

Drew%20Noakes's gravatar image

Drew Noakes
1.7k61616
accept rate: 66%

edited 07 Mar '12, 03:45

post all your code, maybe there is something else ...

(07 Mar '12, 03:45) jimgb-2 jimgb-2's gravatar image

@jimgb -- well, I copied my block out (as added above), reset the question, and pasted it back in. The question now passes, and the rest of the code is from the reset version. Unfortunately I cannot get back to the exact version I submitted.

(07 Mar '12, 03:47) Drew Noakes Drew%20Noakes's gravatar image

As @xyzzy said, the problem "almost" for sure is "dt" in matrix R.
I believe that if they put "dt" in the "restricted" area, is because they use it for the testings

(07 Mar '12, 04:30) jimgb-2 jimgb-2's gravatar image

@jimgb, the grader accepted my code with the dt in the R matrix (after the deadline). I know that there's no correlation between dt and noise, but I'm still not clear on exactly what went wrong. It would be nice if there were more transparency in how the auto-grading worked.

(07 Mar '12, 04:34) Drew Noakes Drew%20Noakes's gravatar image

You will not get an error for using "dt" in R, the problem is that you are forcing the change on R every time the automatic grading program tried to change only F (by means of changing "dt"), in that case the results from your matrices where not as expected by the grading program

(07 Mar '12, 04:43) jimgb-2 jimgb-2's gravatar image

3 Answers:

Measurement uncertainty isn't by definition equal to dt, though in this case it happened to be. If the grading program varied dt to test your matrix under multiple conditions, your uncertainty figures would be off. That might explain your issue.

link

answered 07 Mar '12, 03:56

xyzzy-2's gravatar image

xyzzy-2
1.8k21037

I don't think that explains it, as the system seemed to accept my answer as correct when I submitted it the second time around. The exact same code was not accepted minutes earlier with changes outside the 'your code' block. I never changed the value of dt.

(07 Mar '12, 04:04) Drew Noakes Drew%20Noakes's gravatar image
1

I think having more transparency around how the grading system works would be useful, actually. As a software developer, you find out how the system you're coding for works and code to that spec.

(07 Mar '12, 04:05) Drew Noakes Drew%20Noakes's gravatar image
1

The test cases run against the homework for grading may have more tests built in than the one that's run from the web interface. I know that for HW1, lots of people had code that looked good for a 20 element matrix, but the homework hit them with lots of different varieties of matrix sizes that could cause folks to fail. I don't know if those multiple test cases came along for the homework check that you can run yourself. If one of the official Udacians could comment if the test suite that can be run on demand is the same as the ones run on the back end, that would help clarify a lot, I think.

(07 Mar '12, 04:09) xyzzy-2 xyzzy-2's gravatar image

Since the value for the measurement uncertainty is arbitrary, I guess using dt doesn't matter much.

(07 Mar '12, 04:38) lrq3000 lrq3000's gravatar image

It does if the dt changes to say .05, but uncertainty is supposed to stay the same at .1! I'm not sure why you voted me down...

(07 Mar '12, 13:55) xyzzy-2 xyzzy-2's gravatar image

Hello, I submitted the following as my answer to homework 2.6 and it was graded as incorrect:

P = matrix([[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 1000, 0],
[0, 0, 0, 1000]]) # initial uncertainty

F = matrix([[1, 0, dt, 0],
[0, 1, 0, dt],
[0, 0, 1, 0],
[0, 0, 0, 1]]) # next state function

H = matrix([[1, 0, 0, 0],
[0, 1, 0, 0]]) # measurement function

R = matrix([[0.1, 0],
[0, 0.1]]) # measurement uncertainty

I = matrix([[1, 0, 0, 0],
[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 0, 1]]) # identity matrix

All the results were EXACTLY equal to what was shown in the explanation video.
Also when I repost the exact some code into the interpreter and run it now, it tells me that it is correct!

Is there a bug in the auto-grader? Because I'm sure that my results are correct!

And also, wouldn't there be a possibility to make the auto-grader a little more flexible? It seems that your whole homework is graded wrong, if you insert just a single character in a place where you're not supposed to.

In my opinion it matters that the code+results are correct. And not that you have every single space exactly like in the solution!!!

link

answered 07 Mar '12, 04:16

Lukas_Fuerler's gravatar image

Lukas_Fuerler
116148

Did you make your code in an external interpreter such as IDLE, before copying it in the Udacity editor?

If so, maybe you are suffering from the EOL bug I described here:

http://www.udacity-forums.com/cs373/questions/14844/hw-26-graded-incorrectly-end-of-line-bug

link

answered 07 Mar '12, 04:39

lrq3000's gravatar image

lrq3000
342259

Yes, I used IDLE and I think my problem was created by the EOL bug. Thanks!

(07 Mar '12, 20:54) Lukas_Fuerler Lukas_Fuerler's gravatar image
Your answer
Question text:

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "Title")
  • image?![alt text](/path/img.jpg "Title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Tags:

×5,185
×1,718
×515
×298
×123

Asked: 07 Mar '12, 03:42

Seen: 272 times

Last updated: 07 Mar '12, 20:54