## Is Python more than a scripting/prototyping language? Discuss.

This isn't really a question that I have. I just thought that it would be an interesting subject to talk about.

Python - a language you would use to develop a complex program or just a really nice scripting language.

Discuss.

Realistically I can not see myself using Python to solve complex problems. But what I do see myself using python for is to rapidly build small to medium sized prototypes or mini-projects. A quick proof of concept if you will. If I have an idea about a widget I want to build and want to build a quick dirty model or prototype of my ideas python is a good fit for that. At a high level I can use Python to see how different classes (or objects) might work together without implementing all the gory details under the hood. Perfect for a visual learner like myself. This does not get rid of the need to diagram my ideas out on paper (uml or otherwise) but I like to think of it as more fun way to map out my ideas using Python. Just my 2 cents. What do you think?

jksdrum
3.6k102058

accept rate: 18%

There are quite a lot of old wisdom banging around here. Some of it is older than it is usable :-D I'll add my 5 cents.

All computer languages are equal, since they are all Turing-complete

True, but irrelevant. The ultimate version of this is: In the end, all code you're writing will be run by the processor in your computer. It talks in a machine language that consists mainly of moving numbers about, basic arithmetic, testing for a few numeric conditions and jumping about. Thus, every program can be written in machine language. We don't do that, though, because expressing the kind of programming ideas we want is damn hard in the frugal environment of pure machine code. What matters is the expressivity of the language, not its turing-completeness. What kind of code is easy to write in the language?

The canonical reference here is Paul Grahams Beating the averages, where he is talking about building production code in another "Too theoretical, too slow, too highlevel" language - Common Lisp.

Python is too slow, because it is interpreted

This is also half-true at best. The standard way of developing in Python is to do it in the interactive, interpreted environment. This is for good reasons - This kind of incremental development environment is very effective in the speed of the write-test-debug-cycle and the in-place testing. However, there is nothing that stops you from compiling the finished code with something like Cython when it is time for release, adding things like type hints and so on to increase effectiveness. There are tools like PyChecker that you would use to go through the code and warn for potential problems. It is just that these are not forced upon you from the start.

Java, by the way, is an interpreted language in exactly the same way as Python is. I do believe that it is used in one or another big system.

Python can do everything that any other language can do

Not true, either. In Python, we have a lot of very powerful constructs on the high-level-end: mapping over sequences, list comprehensions, a dynamic object model. On the other hand, we are lacking stuff at the low-level end - If we want to bang hard on memory adresses or control the storage models for our complex numbers, we cant. Python is a fairly unashamed highlevel language. If you want to do lowlevel stuff stuff, use a lowlevel language like C.

(This taps into another misunderstanding: A highlevel language is better than a lowlevel language, because it is highlevel - Not so. High and low level are in relation to how far the language is from the "metal" of the computer. It expresses its concepts on a different abstraction level.)

However: From Python, you can easily call procedures and functions written in C, for instance (and many other languages). So, if you need to get lowlevel stuff done, you write it in C. Then you want to do advanced operations on the data you send to the lowlevel procedures. Those bits you write in Python.

So, what is the answer, then

The answer is that the question is broken. Or, rather, the question is the headline is simple to answer: Yes, Python is a lot more than a scripting and prototyping language. The related question "Would you use Python for a largescale commercial project?" however, is wrong. The proper question would be "Where, in a large-scale, complex commercial project would you be able to use python?" And there, the answer would be "In the high-level parts of it, where we need to do complex manipulations on intricate data structures, or use flexible generic procedures on varying data sets." Would you write an OS in Python? No, but I'd write a computer music system, or a big CRPG, or the user interface of a big database application. I'd farm away the low-level audio and graphics stuff (that is, audio buffering, triangle rendering and so on) to a lowlevel language that does that better, though.

So, why isn't everyone using Python, then?

Well, that one is easy. As Paul Graham points out - If everyone knew what they were doing, everyone would be using LISP :-) But they don't. Large scale projects are handled by politics and management, and management reads stuff they understand, where words like "Market share" and "Brand visibility" is important. From that, they learn that Microsoft is important, that everyone is using Visual Dot Net Letter-of-the-month-#, and so that is what must be used in the project, top to bottom. A few years back, it was Java, and a few years before that, it was C++. A lot of years back, it was COBOL.

Would you, or could you, use Python in a big commercial project? Sure you could - As you could with almost any language, as long as it allows you to express the kind of data structures and manipulation that you need. Would you use it as the single language? Depends on the project, of course - The user part of a big database application thing could without problem be only Python. Should I learn only Python and use it for everything? No. If you want to be a proficient programmer, you should at least have a lowlevel language such as C under your belt, too. If you want to work at a specific place, you need to know the language used there - If you're going to do major iOS work, you'll want Objective C. This, however, is no reflection on whether one language or another is more suitable in itself for a particular style or scale of project, but has everything to do with the corporate culture and politics of the place you're going to work at.

Are you wasting your time learning Python? No. Every programming language you learns adds concepts and patterns to your repertoire, and allows you new and different ways to think about a problem. When you've learned about Pythons list comprehensions, you'll know about those as an elegant way of solving certain problems. Then, you might get stuck on an only-C project. Still, when you see the problem that you're up against, and realise that you could solve it easily using list comprehensions, then you can quickly code up a simulation of them in C, and use that to solve the problem.

Every language gives you tools. Every language allows you to use them with more or less difficulty. Every language is useful.

Are you bored yet? :-)

JohanG-Sweden
9.8k1242101

3

You, sir, win an internet. This should be the top answer.

(16 Mar '12, 06:01)

1

Have an upvote. And my hat

(16 Mar '12, 06:29)

1

takes a bow :)

(16 Mar '12, 06:33)

3

re:" we are lacking stuff at the low-level end - If we want to bang hard on memory adresses or control the storage models for our complex numbers, we cant. Python is a fairly unashamed highlevel language. If you want to do lowlevel stuff stuff, use a lowlevel language like C."

I use python to bang on H/W registers. I work on a team that ships embedded systems, and I use python for both prototyping and production to access hardware at the bit level. We are running on an OS, so we go through a device driver (written in C), but my python code does have lines such as hwoject.poke(address,value) and much higher level access as well.

(16 Mar '12, 16:02)

Why is this question still open? JohanG just shut this biatch down (well about 13 hours ago he did)!

(16 Mar '12, 18:49)

Church. Nuff said. (as the mic drops)

(16 Mar '12, 19:26)

George - Well, then, so much the better :-)

(16 Mar '12, 19:34)

This is the first time someone gives an answer as complete and great as this to this question... It would be great to read your opinion about C++ in the same way

(04 Feb '13, 03:34)

Python is as capable as any language out there. If something can be done in another language, it can be done in Python.

There are many considerations that go into choosing a language, but the perception that python is only a "scripting" language, or only suitable for small projects, or prototyping, or that it lacks the power or versatily to do what language x can do, is not correct.

Rob Barnes-3 ♦
19.4k1068205

I was not saying you couldn't use Python to do any project imaginable. I am not knowledgeable about Java. But dare I say that Python could be, maybe already is, a major competitor to Java. Both require "interpreters" (Java uses JVM). @dshmif already showed me a way to convert python programs to an installable windows app:

dshmif post re: py2exe

(15 Mar '12, 18:31)

1

Not only can you dare to say...feel free to say it. I will tell you point blank that the group I worked with HATED having to go to Java. The productivity gains when using Python are truly great and at least in the web application space frameworks such as django really give Python some firepower. Again it's tough to argue against the productivity that Python provides. However for the Enterprise space I'm sure there will always be a need for Java. For some reason, at least in my experience, Python is looked at as a toy language (ridiculous) and as I've mentioned in another thread managements perceived ability to obtain resources easily to handle ramp up and turn over plays a roll in these things.

(15 Mar '12, 18:43)

This isn't true. There's a bunch of things you can do in some languages that you really can't do in Python without libraries (and even with libraries: try accessing your processor's flags register). The point is that it isn't a matter of big project vs little project, but of what you want to express with it.

(15 Mar '12, 19:48)

1

Indeed...one of the tools in a toolbox that's it...just another tool.

(15 Mar '12, 21:00)

@jesyspa you miss his point. You can get the same operations done with Python as you could with any language that could operate on the processor's flags register. As is pointed out in Unit 1 Python is just as Turing Complete as any other language out there.

(15 Mar '12, 23:58)

@tbartels: Sure. So is brainfuck. However, nobody is going to claim you can do everything in brainfuck that you can in another language, because you can't -- the language simply doesn't provide networking capabilities, and you're not going to get around that without changing it fundamentally. Similarly, writing certain programs in Python is not possible, either due to the language providing no support for it, or the current implementations not offering it.

(16 Mar '12, 04:31)

Neither Python nor C can access the processor flag register without an assembler library or a device driver. (unless you are on an embedded processor and the flag register is memory mapped). I use python to control hardware registers in my daily work - and much easier than the corresponding C/C++ code.

(16 Mar '12, 16:23)

@GeorgeBullis: C and C++ have inline assembly as a fairly common feature (although it is not particularly convenient, as far as I've seen). How do you access the registers? Googling gives this, which doesn't seem very affirmative.

(16 Mar '12, 16:32)

@jesyspa -For my register access, I use a home grown version of the PortIO library (like the one you found) that accesses our custom hardware via our custom device driver. www.SWIG.org is an invaluable tool for taking a DLL or .so (shared library) and wrapping it for python access.

(17 Mar '12, 00:31)

...which is exactly what I said above, isn't it? Taking some low level code and calling it from Python?

(17 Mar '12, 15:11)

jksdrum, don't get snookered into this vestigial stigma of Python not being suited for large-scale projects.
With all humility, folks who shrug off Python as, somehow, inadequate are laggards from the past, when hardware
was painfully slow.

Exactly because Python's level of abstraction is so much higher than the conventional "Enterprise" languages,
it is very well suited for large-scale development. It supports multiple paradigms and nimble tools that the "big"
languages can't catch up to for the past 15 years.

In addition to that, Python - and it's habitat - will teach you good engineering practices.

I'm a Ruby, Scala and C person generally, so I'm not touting Python's virtues out of religious motivations (i think) ;)

Cheers.

Eli Burmin
2.4k1231

It can and is used for large scale projects. I've personally been involved only up to medium scale web applications, one of which was used 24/7 to handle legal requests for information (can't say much more than that on that one). A colleague of mine worked for PIMCO for sometime and though I don't know for sure but I'm guessing they were working on decent sized projects. Heck I get the vibe that some of the people responsible for udacity are googlers and there's mad love for Python there. I'm sure they've done some substantial work with it.

Hopefully mathprof finds this thread because he might have some specific examples.

David Smith
1.2k61236

Depends on what you call a "complex problem". Python is by itself not suitable for writing heavy-weight programs because it will not be able to run fast enough. However, with the right libraries backing it it can be used to make some impressive things that work at a decent speed (such as Anki), and if you talk about "problems" in general, as opposed to "programming problems", you'll find plenty of uses (see SciPy, Sage, etc.). It isn't good raw material, but it is wonderful glue.

Anton Golov ♦
13.8k2175174

I like your use of the word "glue". I read somewhere else that Python is heavily used as glue code between modules.

I'll say one thing about python it is so much easier to type than other languages (I'm referring to C++ and the like).

(15 Mar '12, 18:41)

@jksdrum: And read. And test. And (thus) maintain. And I think getting the AST and doing awesome things with that is quite doable, too.

(15 Mar '12, 18:44)

AST? Help a n00b out.

(15 Mar '12, 18:47)

@jksdrum: Abstract syntax tree. Well, I mean the parse tree, really. The code, but represented in the form of a tree, so that it's easy to manipulate.

(15 Mar '12, 19:03)

Thanks @jesyspa.

So this question is coming out of left field but here it goes....

My first exposure to programming was the c language followed up by c++. The world of int's and char[]'s was fairly straightforward to understand.

Is python, and other higher level languages, really just making clever use of pointers in essence abstracting away the data just one more level? Because everything is an object in python.

(15 Mar '12, 19:11)

@jksdrum: Python and other high level languages make use of a lot of things to abstract things away. Pointers are important (although I think it's just because shallow copying everything by default is simpler that they do that); however, there's also all kinds of syntactic sugar that isn't present even in C++, such as list comprehensions and inline tuple creation.

On the other hand, you've also got the power of the language being dynamic, and therefore being able to change things that would usually be compile-time at run-time. Class definitions are a great example of this: you can dynamically select what you want your class to inherit from, or what functions it should have.

(15 Mar '12, 19:51)

2

C is easy to write for certain coding approaches and somewhat harder to write for others. Pointers are easy to code, but also easy to screw up. I've written a lot of object-oriented code in C. It's "supported," but not enforced or made easy. If you want readable, type-checked and range-checked code, you soon start adding "syntactic sugar" via macros or carefully named function calls.

Bjarne Stroustrup said, "C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off."

My limited exposure to Python leads me to believe that its strength comes from natural, almost-English expression of coding intentions. You write code to be the way you'd like it read back to you. This was tried at least once before, with COBOL, but we understand general-purpose computing much better now. (COBOL reads like instructions to a filing clerk: "add a to b and store the result in c." I heard [then Capt.] Grace Hopper say this was so supervisors could read the code. She also said the design was a mistake; supervisors don't read code.)

I'm sure there is power in Python that we haven't yet encountered, for functional programming, error handling, recursion, and a lot more. It's impressive when multiple styles are well supported in a single language.

(15 Mar '12, 20:06)

Sorry to beat a dead horse...

I did not phrase my initial question well:

Is python, and other higher level languages, really just making clever use of pointers in essence abstracting away the data just one more level? Because everything is an object in python.

Let's analyze a concrete example:

Python has a list type. It can append anything you want: strings, numbers, other objects.

If I initialize a list like this:

list = ['text',6,'more text']

It is clear that the list contains a string, a number, and another string. What isn't clear is what is going on "under the hood".

My guess is if you were to look into memory the python list is actually just a linked list. And each node in the list is a pointer to an object. For example, list[0] holds the string 'text'. But under the hood list[0] contains a pointer to a string object somewhere else in memory. Likewise list[1] holds the integer 6. But under the hood list[1] contains a pointer to an integer object somewhere else in memory.

Is this even close to being correct?? Or am I wrong in thinking of the Python language in c/c++-like terms?

(15 Mar '12, 20:27)

1

You are wrong in thinking of the Python language in C++ terms, or are at least thinking in the wrong terms.

Python is a language, not an implementation. Mapping Python to C++ when writing it is like mapping C++ to assembly: useful in a few cases, but only applicable to one implementation, and unlikely to improve your understanding. First and foremost, think about semantics, then think about how it will be executed if you need to.

A list may be a linked list. I doubt it -- I think something along the lines of a C++ vector is more likely. However, once you start talking about that, you are no longer talking about Python; you're talking about CPython, or PyPy, or IronPython. Those three may do it in completely different ways, and you should focus on one if you want to understand these things.

For example, CPython will first compile your code down into bytecode, and then interpret that instead of the original code. Therefore, there's actually two mappings you should be looking at: Python code to (optimised) bytecode, and bytecode to instructions. I do advise looking at the bytecode, it's an interesting thing to see, but it's no t necessary for writing Python code.

One exception to this is knowing the complexity of operations on different types: if lists were indeed represented as linked lists, then random access would be awfully slow, and therefore it would pay off to avoid it. I doubt this is the case; on the other hand, you may want to know that strings are immutable, and that this can lead to concatenation being more expensive.

In brief, make sure you know the difference between a language and an implementation, and look things up if you want to make a mapping to another system.

(15 Mar '12, 20:40)

Guido van Rossum (the creator of Python - http://en.wikipedia.org/wiki/Guido_van_Rossum) has just addressed this question in an interview:

http://developers.slashdot.org/story/12/03/16/1947252/van-rossum-python-not-too-slow

"Van Rossum: What people of course do is they pick a tool and they build something incredible, and as part of the incredible thing they're building, they're doing crazy stuff, and sometimes some of the crazy stuff involves doing a whole bunch of computation, crawling through a graph of a billion social contacts or analyzing a trillion email messages or whatever.

At some point, you end up with one little piece of your system, as a whole, where you end up spending all your time. If you write that just as a sort of simple-minded Python loop, at some point you will see that that is the bottleneck in your system. It is usually much more effective to take that one piece and replace that one function or module with a little bit of code you wrote in C or C++ rather than rewriting your entire system in a faster language, because for most of what you're doing, the speed of the language is irrelevant."

Update: More commentary on Linux Weekly News: https://lwn.net/Articles/486908/

Karl J. Smith
8422516

1

"for most of what you're doing, the speed of the language is irrelevant"

Agreed. Take our webcrawler for example. Network delays far outweigh string processing delays or list parsing delays.

If and when those delays do become critical path re-write those parts using c/c++.

(16 Mar '12, 17:44)

Go is kind of like a mix of C and Python if you care more about performance.

Here's an interesting anecdote:

Here's an interactive go compiler, kind of like the one used for the Udacity class:

http://tour.golang.org/#1

Karl J. Smith
8422516

I have really enjoyed seeing both sides of the coin. Thanks to all who have contributed their experiences/opinions.

I talked with one of the SW architects at the company I work for and this is what our team uses Python for:

1. parsing API messages on a server
2. accessing Compact Flash
3. Internal test harnesses
4. Code generation during the Build for client api
5. Help file generation for APIs

jksdrum
3.6k102058

Eve Online is an MMORPG that uses stackless python for its client and server software, that's a pretty huge and complex database system. Python also is the language used for Dropbox client and server, and backends for things GMail, Google Maps. It's heavily used in web programming I'm pretty sure.

It may not make sense to write a given product completely in Python, but it certainly has a lot of uses.

curiousborg
67421031

Learn python, youre life as a programmer will be so much easier.

MathGuru
285119

Question text:

Markdown Basics

• *italic* or _italic_
• **bold** or __bold__
• image?![alt text](/path/img.jpg "Title")
• numbered list: 1. Foo 2. Bar
• to add a line break simply add two spaces to where you would like the new line to be.
• basic HTML tags are also supported

×30,528
×2,226
×40
×1