NASA is coding in Python?

I wonder in what language the software for Mars Climate Orbiter was written. Python/C++/C/Assembler/w.h.y.?

Regarding the mis-specification/-communication of units used in the software (e.g. ‘m/s’ vs. ‘ft/s’), I would like to direct your attention to Bjarne Stroustrups keynote at Channel9. The recent standard of C++11 permits the creation of classes resp. objects of classes that hold units in their type.

Regarding the ‘queue’ example I would also like to point to C++, which many years ago acquired such a prefabricated type in its standard library. Just #include <queue> and off you go.

asked 27 Jun '12, 01:36

Andreas%20Scherer's gravatar image

Andreas Scherer
32113

accept rate: 100%

retagged 23 Nov '12, 12:37

Retagbot's gravatar image

Retagbot ♦
1518171

1

Yes, NASA uses several languages, including Python. You can even download some of that code at http://code.nasa.gov/language/python/.

(27 Jun '12, 09:26)

Clinton Morell

Clinton%20Morell's gravatar image

As it is said in the video, in Python you can just use lists as queues. Reimplementing it serves as an example, not because Python doesn't have queues. They also say that implementing it using lists would be too trivial to serve as a good example, so they use arrays. If you need more advanced queuey stuff, there is http://docs.python.org/library/queue.html

(27 Jun '12, 11:05)

Christian

Christian's gravatar image

7 Answers:

The short answer:

NASA is not using in Python in any substantial way in any flight project or military deliverable. However, as JIffyClub and Clinton point out, there are some projects that use Python, particularly on the science side of the equation, where the requirements of flight projects or military deliverables are not present. LIkewise, I would expect that you could find it in some personal projects or replacing automation scripts that used to be done with shell scripts or Perl.

The very long (but interesting read to some) answer from an insider's point of view:

Although anyone can use anything they like for their daily work, the majority of people use C. To a much much lesser extent you see C++ and to an even lesser extent, Java. Few-to-none use ADA, Fortran or Lisp. Assembler is used if you happen to be working on optimization or writing a driver. For prototyping, at least in JPL, Matlab is the favorite by a long shot although a few people use Mathematica (I would expect someone might be using Maple too but I never saw anyone using it). In general, no large or medium project uses any scripting language, i.e., no python, ruby, perl, etc., because these projects tend to have special requirements.

The preference for C makes a lot of sense. By default, all large flight projects are written in C. For example, the code for the Mars rovers is in C with a single exception (Gestalt, the visual navigation software, which is written in C++). Thus, it makes little sense to develop the code in a language other than C knowing that for the code to be accepted by either a flight project or adopted by your peers, it will need to be ported to C anyways. Also, by default, drivers of commercial hardware that you might need to interact with are written in C. C is fast, there are very mature libraries for it and It allows you to write at very low-level, close to the hardware, which is useful. A huge plus is that there are many compilers that can write code that follows the C standard. Thus, If you develop your code in, say, gcc, it would compile without any major problems in a watcom or green hills or any other C compiler and you can be pretty confident that it will behave the same (you still need to test it, of course, but the expectation is that the surprises will be few to none). You could not say the same of many other languages. In particular, I am not sure whether there is any C++ compiler that can produce code that follows the C++ standard today, but I know that there wasn't any not many years ago.

C/C++ is the preferred language of military applications so if your project happens to have a military sponsor (fairly common at NASA), your C code would work too. I know of no project that uses ADA, even for military sponsors. One of the possible reasons for this is the close interaction between university research and military sponsors. UNIX and C/C++ are (or at least used to be) ubiquitous in academia and thus, graduate research projects from universities like CMU that had a direct impact on military projects were developed in C/C++. Likewise, an EE/CS/Math/Physics new graduate is bound to know either C or C++, so s/he can start contributing to any C/C++ project without being hit with the learning curve of a new language.

IMHO, for the rovers, C++ is not a good language without restricting it a lot, in which case, you might as well use C (the following scenario fits satellites as well). Most projects that you encounter on a daily basis that use C++ are completely oblivious of resource constraints. These are very important in a flight project where the processing power and amount of memory available, in both RAM and disk, are quite limited. Long gone are the days when military and space hardware requirements drove the chip and computer memory industries; nowadays, they are driven by the mass market. Thus we have powerful chips and inexpensive hard drives that are great for our laptops but cannot be used in outer space, where special radiation-hardened parts are required. The combination of the need of both rad-hard parts and throughly tested parts often leads managers to prefer parts that severely lag the capabilities of the state of the art parts that we are used to in our daily lives. As an example, the MERs had a 20 MHz RAD6000 CPU with 128 MB of DRAM. My laptop, already 4 years old, is 100 times faster (2.4Hhz) and has 32 times more RAM (4Gb). Likewise, each rover has 256 MB of flash memory, dwarfed by my tiny 180Gb HD, not to mention my 1TB external HD. So.. with this in mind you can see why managers have problems using C++, with it reputation of generating bloated code. In addition, certain features of the language, like templates, have a large tendency of producing large code footprints. In comparison, C code tends to be very streamlined.

Again, IMHO, another problem with C++ is that it takes a long time to master. You can say that you can start using the basic features of the language, but in that case, just use C. If you use more features, then you get into the problem of requiring people that master the language AND also the subject that they are contributing too. However, consider a typical case. An engineer who spends all his time working on, say, antennas or communications. This person is very likely to know C but very unlikely to know C++ (less alone master it). You could expect that s/he would deliver the driver for the antenna in C. The code of this driver could be easily reviewed by other communication engineers which, again, are all likely to know C very well and not know C++. After all, for them the language is just a means to get the job done, which is to get the antenna working. Their efforts go into making the antenna and using the least common denominator that can get it to work fast, reliably and safely and that is C, not C++. This scenario repeats across the board: the person that is working in image compression, in visual odometry, in firing the parachute, ... on and on. So why use C++? C++ is an amazing language and I can see why it is the preferred language for very complicated pieces of software but I do not see it as an option that stands out in flight software.

As an addendum pertinent to this class, I wrote the dust devil and cloud detectors for the Mars rovers - all in C. My code was loaded with assertions to a much larger extent than the 1:100 ratio discussed in class. A second person - Alex Fukunaga, now a prof. at Tokyo University - had the sole job to test my code against test sets provided by the scientists - Mark Lemmon (Texas A&M) and Ron Greeley (Arizona State) - to meet their hit and miss ratios. We did this to-and-fro, me working on the code, Alex testing it and providing feedback, for several months. During all this time, Steve Chien, the manager of the project, was passing updates to the QA team of the MERs. After the code met the scientists specs, Alex review the whole code. At this point my job was done.. the algorithms were running at specs and my only duty was to be available if something unexpected happened. However, the completion of the job, which now Alex was in charge of, was far from over. The code was reviewed by a red team, whose job was to to comb it, line by line, for any problems. All of the people in the red team were very familiar with C/C++ so reviewing my C code was no problem. After that Alex and Jeff Biesiadecki, our interface to the core of the MER project, integrated the code into the flight software. Jeff generated the calling commands and Alex tested the code again in a simulator, this time to check its interaction with the code of the rovers and vxworks, the rovers's OS, e.g., its behavior with respect to memory, timings, interfaces to cameras and buffers, interruptions, etc. Once integrated and tested, it was downloaded into a functional copy of the rovers kept at JPL, and tested again. This is as close as you can get on Earth to the real thing because the rover at JPL is an exact copy of the ones in Mars. This is the time where you get to put on white lab coats and shoe protectors and do your test in an air-filtered room: although the rover copy is on Earth, it is treated as flight hardware because it very much affects flight hardware (i.e., the rovers in Mars). For example, this same rover was used to figure out how to free a rover that has gotten stuck in soft sand. Again, Alex had to retest everything and now, we add the tests of actually processing input with the rover cameras. After all of this, the software was transmitted to Mars and added to the code of Spirit and Opportunity. So as you can see, Alex's effort of testing the software to flight-qualified level was several times the effort that it took me to write it.

I have always been a huge fan of assertions which helped the code pass the red team review without comments pertinent to errors but there was an error in our integration to the Mer code: a misunderstanding in the interface between our code and the MER code (what we discussed in class as a specification error.. we did have a detailed document that described exactly what we were going to give them.. or so we thought.. there was a minor issue that was wrong, though.. I don't remember what it was but the issue is that we had written something and they thought it meant something else). After that, the code was uploaded and has been running on Mars for several years now without a hiccup and my understanding is that the same code is now part of the code of MSL, the rover presently on its way to Mars.

So, 3 observations with respect to this class:

  • Been defensive about your code pays off big time. Load your code with assertions. Do not use fancy code. Be clear. Keep your routines short. Name your variables sensibly. refactor. Be organized. In essence, do what you learnt in kindergarten: reduce the possible causes of mistakes and catch them early in development.

  • you should never underestimate the time that you will need to test, particularly if you want to convince someone else that your code is correct and/or safe.

  • in my experience, the main source of problems show up in the specification. It is extremely difficult for two people to work out the details of a spec and actually get them right, i.e., to end up with the case in which each person understands what the other one is saying. In general, managers budget very little time and effort to the process of putting code from different people/teams together (a.k.a. the 'glue logic' process).. after all, both people/teams met beforehand, agreed on a 'specification' and 'followed' it, right? It is here, though, where a lot of problems show up.. this is similar to what happened with the Mars Climate Orbiter.. both the JPL and the Lockheed Martin codes were correct.. the problem was that, in spite of the specs, they were passing each other the wrong data, i.e., both 'thought' that had understood what the other wanted but they hadn't.

link

answered 27 Jun '12, 09:02

Goldsong's gravatar image

Goldsong
26.8k30213126

edited 28 Jun '12, 22:27

3

The short answer:
No, NASA is most definitely not coding in Python

The real answer:

NASA is definitely using Python. They may not use Python on spacecraft, but they certainly use it on Earth. You can even download Python code from NASA at
http://code.nasa.gov/language/python/

See also:

http://www.python.org/about/success/usa/
https://modelingguru.nasa.gov/docs/DOC-1762

(27 Jun '12, 09:26)

Clinton Morell

Clinton%20Morell's gravatar image
1

Hi CLinton,

The first project that you mention, Sunpy, is a small and isolated case. The second is not even from NASA but from United Space Alliance, a NASA contractor. Either way, I can certainly find people at NASA that have isolated Python projects.. or LISP projects.. or IDL projects.. or Mathematica.. or PERL.. but that does not mean that there is a trend to move toward any of those languages or that any of them is a language of choice for any medium-to-large project.

(27 Jun '12, 09:47)

Goldsong

Goldsong's gravatar image
1

Very generous of you to take the time to provide this insight into real world application of the concepts being taught. I enjoyed reading and learning from your post.

I've worked in industry for 26 years, all of that in enterprise/consumer software. That's a very different world from the one you describe, but I agree fully with your 3 concluding assertions.

To your point regarding the main source of problems originating in specification:
It can be particularly challenging to specify interactions with users in a GUI world, and I am looking forward to learning about best mindsets/practices to help in this regard.
Often (usually) we don't know the best ways to interact with users when the project starts. Only after creating usable solutions and seeing them in action do we learn the conceptual flaws in our interaction and visual designs. This leads us to iterate and improve continually based on user feedback.
During this time, the specifications are fluid...which presents great challenge and responsibility to developers and testers. Generally, developers and testers with strong domain knowledge (and very good understanding of our users) end up providing significant inputs into the interaction/visual design evolution.

(27 Jun '12, 09:59)

Bruce

Bruce's gravatar image

Hi Bruce,

Thanks for the comment. I was hoping someone would find it interesting.

Your comment reminds me of a meeting many years ago, with the Apple User Interface Development group, at one of their worldwide developer's conferences. They were discussing their challenges designing interfaces: the way that the users would react to a feature of the gui was not always the expected one. They used to have large teams of people studying user reactions. I guess as users, we take GUIs for granted many times are not aware of the tremendous efforts paid to every detail to make the experience of using them as effortless as possible.

(27 Jun '12, 10:15)

Goldsong

Goldsong's gravatar image
1

@Goldsong this is really cool, thanks for sharing your experiences! I hope you and others keep sharing examples from what you've encountered in the real world--really rounds out the stuff we are being exposed to in the class. Btw, what prompted you to take this class? Seems like you have a lot of experience already.

(27 Jun '12, 12:29)

Jeffrey Tratner

Jeffrey%20Tratner's gravatar image
1

@Goldsong , do you have any suggestions for good books for learning C. I have a sense that K&R is the gold standard, but I'm more interested in problem sets (e.g. introduce a few concepts, now figure out how to get them to work to accomplish some predefined result). I've been working through Learn C The Hard Way at the moment, which I like a lot because it focuses on getting you to read correct code, figuring out how to break it and error check it, and also asks you to do research on your own to accomplish tasks. But if you have the time to make some suggestions, I would really appreciate them!

(27 Jun '12, 12:40)

Jeffrey Tratner

Jeffrey%20Tratner's gravatar image
2

Hi Jeffrey,

I'm glad you found the answer interesting. Although have a lot of hands-on experience, my background is in Electrical Engineering, not in Computer science. I took many CS classes (data structures, algorithms, AI-related, numerical analysis, on and on) but I also missed many (languages, databases, software engineering, testing, etc.) which fortunately did not impact my work. Thus, with this class I am trying to fill out that gap and also pick up techniques that I might not have come across before.

I would call my testing skills in C fair.. I use lint, assertions, valgrind (at least until I moved to a mac) and load the compiler with warnings. I compensate my lack of formal training asking questions from people I see know a lot, reading books and being careful and methodical. Some of the books that I found useful in improving my code writing skills are 'Code Complete' by S.McConnel, 'Write portable code' by B. Hook and 'Code Craft' by P. Goodliffe.

w.r.t. C books, k&r is very good but I prefer much better "C, A Reference Manual" by Harbison and Steele, which aside from being an excellent read, covers the modern ANSI C that is not covered in k&r. I used to keep a copy at home and another one at work. I think that if you know how to program, Harbison and Steele is all you need to learn C.

Finally, the most useful theory book that I found was 'Intro. to Algorithms' by Cormen et. al. It is a thick book that will teach you about everything you need to tackle any general problem and used to be the standard text for Algorithms in many universities (it probably still is..). Cormen's book teaches everything with pseudo-code which you can them implement with the langauge of your choice. However, be aware that some of Cormen's problems can be extremelly challenging.

I hope this was helpful.

(27 Jun '12, 13:28)

Goldsong

Goldsong's gravatar image
1

@Jeffrey Tratner - you might find 'The C puzzle book' useful for self study problem sets

(27 Jun '12, 13:33)

ydnayabe

ydnayabe's gravatar image
1

@Goldsong
Very interesting stuff, thanks for taking the time to put such a good post online. I work in automation software and we also use C and C++ only

(27 Jun '12, 21:32)

Michele Calgaro

Michele%20Calgaro's gravatar image

Hi Michele,

No problem. It seems that C/C++ is going to be around for a long time; I do not think there is any other language that compares in flexibility for these types of tasks that we work on. Thank you for your comment.

(27 Jun '12, 23:59)

Goldsong

Goldsong's gravatar image
1

@Goldsong: Very well thought out answer. Quite interesting, thank you for sharing. I think I'm seeing a pattern: NASA using solely C for flight software (required characteristics: robustness, efficiency, small footprint, extensive testing, precise control), the Linux writers using solely C for the kernel (Linus Torvalds seems to hate C++ - he makes good points, though)...

It seems that whenever you need to get close to the hardware, when you can't afford to be wasting CPU cycles, when you need precision control of memory and predictable, deterministic realtime algorithms, then all of the benefits of the standard high-level abstractions (which we hear about all the time, what with IAbstraction this and public sealed class Abstraction that, inheriting from some BaseAbstraction...) - all those benefits become drawbacks, and the answer ends up being to "loosen up", and get back down to the low level details, which usually ends up being C.

Is this a correct conclusion? I ask because, to me, the thought that abstractions may actually be a bad idea sometimes is something I hadn't considered. I mean, sure, you wouldn't try to write realtime mission-critical software in Python ("It's too slow, right?"), but I used to think, "Well, C++ is plenty fast, isn't it?", but "Nope, it's too huge of a language - too much cruft, and it's easy to make the complexity skyrocket. We need low-level." So that means, no need for objects, no need for some gigantic templates library, no need for massive inheritance hierarchies... Just good old C.

This is actually pretty cool, in my opinion. It's not just the programming language that is the tool - it's the architecture, the paradigm, the entire collection of techniques at one's disposal, which are the tools for software development and, as such, can be interchanged at will. Thoughts?

(28 Jun '12, 00:54)

Zaven Murady...

Zaven%20Muradyan-1's gravatar image

Hi Voithos,

I think your point of the benefits turning into drawbacks as you approach the hardware hits the nail on the head. A driver, very close to the hardware, most likely will be written in C or assembler: very limited resources, need for a very high speed, small and focused application. An OS, that lies between the hardware drivers and the apps, could be written in either (I just googled it and got this from stackoverflow:

Windows: C++, kernel is in C
Mac: Objective C, kernel is in C (IO PnP subsystem is Embedded C++)
Linux: Most things are in C
KDE: C++
Gnome: C
Unix: C

Again, the kernel, which requires a high speed and needs to have a small footprint, is in C, while the rest can be done in either C or C++. Finally, purely 'high-level' software packages, like mozilla and the adobe products, are written in C++ or other high level language. Here there are few constraints in either memory or speed and at the same time the need for abstraction becomes much more important to handle the sheer size of the project.

I did not know that L. Torvalds disliked C++ but I am not overly surprised. I played with C++, found it fairly difficult and didn't see the point of investing time in it for the type of programming that I did. Still, I had some experience with some projects that have embraced the potential benefits from C++. It comes to mind one called Claraty that provides a general framework for a robotic mobile robot. Who knows.. in the future it might be the case that a flight project might have the luxury in memory to adopt the Claraty framework but presently that is not the case.

(28 Jun '12, 02:56)

Goldsong

Goldsong's gravatar image
1

Thanks for this insight in to real world applications. Your 3 final observations are something I'm going to keep in mind whenever I code in future.

(29 Jun '12, 03:42)

Will Bickers...

Will%20Bickerstaff's gravatar image
1

I would give this answer +100 if the system permitted it.

(29 Jun '12, 12:21)

John Regehr ♦♦

John%20Regehr's gravatar image

@Goldsong: Very interesting. I'll be sure to take a look at Claraty at some point. Thank you for your insight!

(29 Jun '12, 13:54)

Zaven Murady...

Zaven%20Muradyan-1's gravatar image

Thank you very much for all your comments. Clearly NASA remains a place for one-ok-a-kind fascinating work involving both Earth and outer space missions that, in spite of our getting more and more used to daily breakthroughs in the tech world, still stirs imaginations and inspires kids and adults alike. I very much appreciate the warm reception and the time you took reading this fairly long answer and am glad you found it interesting and pertinent to what we are learning in this class.

(29 Jun '12, 14:42)

Goldsong

Goldsong's gravatar image

I'm pretty shocked NASA isn't using Ada. It's the predominant language for flight management systems.

(29 Jun '12, 21:26)

Matt-29

Matt-29's gravatar image
showing 10 of 17 show 7 more comments

I work (and have worked) as a contractor to NASA and my perspective is a little different than that of @Goldsong. Python usage in science is growing fast, including at NASA. SunPy is one example, and there are others.

The Hubble Space Telescope exposure time calculators (http://etc.stsci.edu) are written entirely in Python, as is most of the data analysis software written by STSCI (http://www.stsci.edu/institute/software_hardware). Python is the primary language used for writing the data processing pipelines for the upcoming James Webb Space Telescope, and for some of the testing of JWST: http://conference.scipy.org/scipy2010/slides/warren_hack_smoke_mirrors.pdf.

So, while you may not find Python running on spacecraft, don't be surprised to find it in use at NASA (and elsewhere) for other tasks, especially data processing and analysis. Python, NumPy, SciPy, matplotlib, and the wealth of third-party libraries are a powerful and free stack of software for science.

Some other links:

https://modelingguru.nasa.gov/community/tools

http://agiletesting.blogspot.com/2009/07/python-well-represented-in-nasas-nebula.html

link

answered 28 Jun '12, 14:36

jiffyclub's gravatar image

jiffyclub
2.4k31148

edited 28 Jun '12, 14:39

Hi Jiffyclub,

You are correct, of course. Clearly, NASA is not immune to the impact that Python has had and I can see engineers (me included) are starting to use Python more actively in projects that do not have the requirements of fligth projects, specially in research and data analysis. I modified my answer to make clear that it reflected the specific requirements of flight projects or military projects.

(28 Jun '12, 22:34)

Goldsong

Goldsong's gravatar image

This is from the JPL, but they released a big suite of tools for working with data in Python. You can check them out here on Sourceforge. Mostly just command line wrappers around numpy, etc.

(The source code is inside the tarball)

link

answered 27 Jun '12, 12:38

Jeffrey%20Tratner's gravatar image

Jeffrey Tratner
4.1k136

While C++ does indeed provide a queue type, I can easily see said queue not being optimal for plenty of situations. For example, the queue may reallocate when adding elements, which is something we wish to avoid. Not that this is relevant -- the queue is but an example, and there are many more exotic data structures that are not readily available.

Even in the case of data structures and algorithms that are available, these must still have been implemented by someone, and are not guaranteed to be bug-free. The most obvious historical example is the bug in common implementations of binary search which was discovered decades after the algorithm was first conceived.

link

answered 27 Jun '12, 04:38

Anton%20Golov's gravatar image

Anton Golov ♦
13.8k2175174

i have no idea exactly, but I guess it may have been ADA which AFAIK was the language of choice for this kind of applications many years ago (even though the Mars orbiter was not that old)

link

answered 27 Jun '12, 04:09

Michele%20Calgaro's gravatar image

Michele Calgaro
6.4k41252

edited 27 Jun '12, 04:10

in python among other languages (c, cpp, perl, java...). but i have no idea what they used for mars climate orbiter, probably a lot languages. i wouldn't be surprised if lisp and fortran are there with bunch of obscure domain specific languages.

link

answered 27 Jun '12, 02:08

Aleksandar%20Rodi%C4%87-2's gravatar image

Aleksandar R...
2.1k1445

I think it depends, there's no universal programming language that NASA people use. For the scientific stuff, they may need to use languages like FORTRAN; maybe C++ or JAVA for other applications. It really depends.

This would give you a good idea of how is work like in NASA.

:)

link

answered 27 Jun '12, 01:51

Ericat's gravatar image

Ericat
3146

Your answer
Question text:

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "Title")
  • image?![alt text](/path/img.jpg "Title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Tags

×13,075
×9,100
×1,046

Asked: 27 Jun '12, 01:36

Seen: 16,935 times

Last updated: 29 Jun '12, 21:26