Why_Python
已恢复历史版本
当前版本 00000017 在备份中缺失；以下内容来自最新可读取版本 00000016。
Why Python?
By Eric Raymond on Mon, 2000-05-01 01:00.
英文原文 http://www.linuxjournal.com/article/3882
中文翻译 http://blog.chinaunix.net/article.php?articleId=37422&blogId=8618
在憋出这篇自白之前，Cardinal Biggles让Eric坐在舒适的椅子上思考了超过4小时…
我第一次看到Python是一次偶然，而那时我并不十分喜欢我所看到的。那时是1997年初，Mark Lutz 的书 Programming Python 刚由O’Reilly & Associates出版。O’Reilly的一些新书会不时被送到我家门口，这些书是经过一些神秘的赠送人用我也无法理解的随机方法挑选出来的。
其中一本就是Programming Python。由于我喜欢收集不同的计算机语言，我发现它比较有趣。我知道数十种（超过两打）通用语言，把写编译器和解释器当作乐趣，也曾自己设计过许多种专用语言和标记形式语言。在我写这篇文章时我完成的最近一个项目是一个用来操作PNG(Portable Network Graphic)图像的叫做SNG的专用语言。感兴趣的读者可以参考SNG的主页 http://www.catb.org/~esr/sng 。我也曾在我的Retrocomputing Museum网页（ http://www.catb.org/retro ）上写过一些罕见的通用语言的实现。
我已经听说过许多关于Python的评论，知道它是一种现在所谓的“脚本语言”，一种内建内存管理、并能够很好的与其他程序相互调用和配合的解释型语言。于是，带着头脑中一个最迫切的问题：“Python什么地方优于Perl？”我深入研究了Programming Python。
众所周知，Perl是现代脚本语言的始祖。它在很大程度上取代了shell成为用于系统管理的脚本语言的最佳选择。 这一部分归功于它易于理解的Unix库和系统调用，一部分归功于活跃的Perl社区构建的巨大的Perl模块集合。这个语言通常被认为是网上85%的内容背后被使用的CGI语言。它的创建者Larry Wall被公认为开源社区最重要的领导人之一，在黑客名人堂上仅次于Linus Torvalds和Richard Stallman。
那个时候，我已经用 Perl做过不少小型项目。我发现这个语言非常强大，虽然它的语法和其它一些方面显得有点混乱，而且容易给粗心的人带来麻烦。在我看来，Python作为另一种脚本语言似乎还有不小的山峰需要翻越，因此在我读《Programming Python》的时候，我首先注意了它和Perl的区别。
我很快地浏览到了大家都注意到的Python的第一个新奇的特性：空白符号（缩进）在这个语言的语法里是有实际意义的。这个语言没有使用类似于C和Perl中括号的语法，而是通过缩进的改变来为语句块划分了界限。我和大多数黑客最初了解这一特性时一样，本能的反应是非常的反感，不能接受。
作为一个老资格的程序员，我有幸在20世纪70年代有过数月在批处理的FORTRAN下编程的经验。虽然大部分的黑客并没有经历那个时代，但我们的文化中还是保留了一些对于那些丑陋的固定排版格式的语言的记忆。事实上，那时常用自由格式（Free Format）一词来形容当时出现在Pascal和C中的面向Token的新的语法风格，而现在这个词差不多已被人们遗忘；而接下来几十年里，所有的语言都被以这种自由格式的方式设计。如果不是所有，那也可以说几乎所有。而现在看到Python的这一特性，也难怪人们一开始有那样的反应，就好像他们不小心掉进了一堆热气腾腾的恐龙粪（比喻过时的、丑陋的东西）里一样。
那当然是我个人的感觉。我并不是很感兴趣地浏览了剩下的对这个语言的描述。除了比Perl有更清晰的语法和对按键和菜单之类的GUI元素的良好支持外，我并没找到其它喜欢Python的理由。
我把书放进了书架，心想我以后或许会用Python做一些以GUI为主的小型项目，目的仅仅是为了确定我真的理解了这门语言。而那时我并不相信这门语言能对Perl构成什么实质的威胁。
许多其他的事情让我把那件事抛在了脑后达数个月之久。1997年剩下的时间对我而言具有重大意义，这一年我创作并发表了<<大教堂与市集〉〉。但我也抽时间写了一些Perl程序，其中两个也具有一定规模和复杂度。其中之一是keeper, 一个现在还被用来在Metalab的软件归档中对提交的注册进行辅助记录的软件。你在 http://metalab.unc.edu/pub/Linux/!INDEX.html 中看到的网页就是由它生成的。另一个，anthologize，则被用来为Linux文档工程的HOWTOs的Linux第六版自动生成PostScript。这两个程序都能在Metalab得到。
在写这些程序的过程中我渐渐对Perl产生了不满。更大规模的项目似乎把Perl的不足之处放大成了严重的、不断出现的问题。Perl的语法，在代码只有上百行时看起来只是有些古怪，而代码到了上千行以后，它就看起来就像充满荆棘、难以穿越的篱笆（充满困难无法理解的障碍）了。在代码少的时候，“用不止一种方法来完成一件事情”让语言很有特色、很有表达力，但是在一个很大的代码库中，很难维护代码风格的一致性。很多表达复杂度控制的语法特性（对象、作用域、use strict等）后来才加入到Perl中，但这些语法特性让人感觉很脆弱，有粗糙拼凑之嫌。
These problems combined to make large volumes of Perl code seem unreasonably difficult to read and grasp as a whole after only a few days’ absence. Also, I found I was spending more and more time wrestling with artifacts of the language rather than my application problems. And, most damning of all, the resulting code was ugly–this matters. Ugly programs are like ugly suspension bridges: they’re much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code.
With a baseline of two dozen languages under my belt, I could detect all the telltale signs of a language design that had been pushed to the edge of its functional envelope. By mid-1997, I was thinking “there has to be a better way” and began casting about for a more elegant scripting language.
One course I did not consider was going back to C as a default language. The days when it made sense to do your own memory management in a new program are long over, outside of a few specialty areas like kernel hacking, scientific computing and 3-D graphics–places where you absolutely must get maximum speed and tight control of memory usage, because you need to push the hardware as hard as possible.
For most other situations, accepting the debugging overhead of buffer overruns, pointer-aliasing problems, malloc/free memory leaks and all the other associated ills is just crazy on today’s machines. Far better to trade a few cycles and a few kilobytes of memory for the overhead of a scripting language’s memory manager and economize on far more valuable human time. Indeed, the advantages of this strategy are precisely what has driven the explosive growth of Perl since the mid-1990s.
I flirted with Tcl, only to discover quickly that it scales up even more poorly than Perl. Old LISPer that I am, I also looked at various current dialects of Lisp and Scheme–but, as is historically usual for Lisp, lots of clever design was rendered almost useless by scanty or nonexistent documentation, incomplete access to POSIX/UNIX facilities, and a small but nevertheless deeply fragmented user community. Perl’s popularity is not an accident; most of its competitors are either worse than Perl for large projects or somehow nowhere near as useful as their theoretically superior designs ought to make them.
My second look at Python was almost as accidental as my first. In October 1997, a series of questions on the fetchmail-friends mailing list made it clear that end users were having increasing trouble generating configuration files for my fetchmail utility. The file uses a simple, classically UNIX free-format syntax, but can become forbiddingly complicated when a user has POP3 and IMAP accounts at multiple sites. As an example, see Listing 1 for a somewhat simplified version of mine.
Listing 1. fetchmail Configuration File
set postmaster "esr"
set daemon 300
poll imap.ccil.org with proto IMAP and options no dns
aka snark.thyrsus.com locke.ccil.org ccil.org
user esr there is esr here options fetchall dropstatus warnings 3600
poll imap.netaxs.com with proto IMAP
user "esr" there is esr here options dropstatus warnings 3600
skip imap.21cn.com with proto IMAP
user esr here is tranxww there options fetchall
skip pop.tems.com with proto POP3:
user esr here is ed there options fetchall
skip mail.frequentis.com with proto IMAP:
user esr here is imaptest there with options fetchallI decided to attack the problem by writing an end-user-friendly configuration editor, fetchmailconf. The design objective of fetchmailconf was clear: to completely hide the control file syntax behind a fashionable, ergonomically correct GUI interface replete with selection buttons, slider bars and fill-out forms.
The thought of implementing this in Perl did not thrill me. I had seen GUI code in Perl, and it was a spiky mixture of Perl and Tcl that looked even uglier than my own pure-Perl code. It was at this point I remembered the bit I had set more than six months earlier. This could be an opportunity to get some hands-on experience with Python.
Of course, this brought me face to face once again with Python’s pons asinorum, the significance of whitespace. This time, however, I charged ahead and roughed out some code for a handful of sample GUI elements. Oddly enough, Python’s use of whitespace stopped feeling unnatural after about twenty minutes. I just indented code, pretty much as I would have done in a C program anyway, and it worked.
That was my first surprise. My second came a couple of hours into the project, when I noticed (allowing for pauses needed to look up new features in Programming Python) I was generating working code nearly as fast as I could type. When I realized this, I was quite startled. An important measure of effort in coding is the frequency with which you write something that doesn’t actually match your mental representation of the problem, and have to backtrack on realizing that what you just typed won’t actually tell the language to do what you’re thinking. An important measure of good language design is how rapidly the percentage of missteps of this kind falls as you gain experience with the language.
When you’re writing working code nearly as fast as you can type and your misstep rate is near zero, it generally means you’ve achieved mastery of the language. But that didn’t make sense, because it was still day one and I was regularly pausing to look up new language and library features!
This was my first clue that, in Python, I was actually dealing with an exceptionally good design. Most languages have so much friction and awkwardness built into their design that you learn most of their feature set long before your misstep rate drops anywhere near zero. Python was the first general-purpose language I’d ever used that reversed this process.
Not that it took me very long to learn the feature set. I wrote a working, usable fetchmailconf, with GUI, in six working days, of which perhaps the equivalent of two days were spent learning Python itself. This reflects another useful property of the language: it is compact–you can hold its entire feature set (and at least a concept index of its libraries) in your head. C is a famously compact language. Perl is notoriously not; one of the things the notion “There’s more than one way to do it!” costs Perl is the possibility of compactness.
But my most dramatic moment of discovery lay ahead. My design had a problem: I could easily generate configuration files from the user’s GUI actions, but editing them was a much harder problem. Or, rather, reading them into an editable form was a problem.
The parser for fetchmail’s configuration file syntax is rather elaborate. It’s actually written in YACC and Lex, two classic UNIX tools for generating language-parsing code in C. In order for fetchmailconf to be able to edit existing configuration files, I thought it would have to replicate that elaborate parser in Python. I was very reluctant to do this, partly because of the amount of work involved and partly because I wasn’t sure how to ascertain that two parsers in two different languages accept the same. The last thing I needed was the extra labor of keeping the two parsers in synchronization as the configuration language evolved!
This problem stumped me for a while. Then I had an inspiration: I’d let fetchmailconf use fetchmail’s own parser! I added a –configdump option to fetchmail that would parse .fetchmailrc and dump the result to standard output in the format of a Python initializer. For the file above, the result would look roughly like Listing 2 (to save space, some data not relevant to the example is omitted).
Listing 2. fetchmailrc
fetchmailrc = {
'poll_interval':300,
"logfile":None,
"postmaster":"esr",
'bouncemail':TRUE,
"properties":None,
'invisible':FALSE,
'syslog':FALSE,
# List of server entries begins here
'servers': [
# Entry for site `imap.ccil.org' begins:
{
"pollname":"imap.ccil.org",
'active':TRUE,
"via":None,
"protocol":"IMAP",
'port':0,
'timeout':300,
'dns':FALSE,
"aka":["snark.thyrsus.com", "locke.ccil.org", "ccil.org"],
'users': [
{
"remote":"esr",
"password":"Malvern",
'localnames':["esr"],
'fetchall':TRUE,
'keep':FALSE,
'flush':FALSE,
"mda":None,
'limit':0,
'warnings':3600,
}
, ]
}
,
# Entry for site `imap.netaxs.com' begins:
{
"pollname":"imap.netaxs.com",
'active':TRUE,
"via":None,
"protocol":"IMAP",
'port':0,
'timeout':300,
'dns':TRUE,
"aka":None,
'users': [
{
"remote":"esr",
"password":"d0wnthere",
'localnames':["esr"],
'fetchall':FALSE,
'keep':FALSE,
'flush':FALSE,
"mda":None,
'limit':0,
'warnings':3600,
}
, ]
}
,
# Entry for site `imap.21cn.com' begins:
{
"pollname":"imap.21cn.com",
'active':FALSE,
"via":None,
"protocol":"IMAP",
'port':0,
'timeout':300,
'dns':TRUE,
"aka":None,
'users': [
{
"remote":"tranxww",
"password":None,
'localnames':["esr"],
'fetchall':TRUE,
'keep':FALSE,
'flush':FALSE,
"mda":None,
'limit':0,
'warnings':3600,
}
, ]
}
,
# Entry for site `pop.tems.com' begins:
{
"pollname":"pop.tems.com",
'active':FALSE,
"via":None,
"protocol":"POP3",
'port':0,
'timeout':300,
'dns':TRUE,
'uidl':FALSE,
"aka":None,
'users': [
{
"remote":"ed",
"password":None,
'localnames':["esr"],
'fetchall':TRUE,
'keep':FALSE,
'flush':FALSE,
"mda":None,
'limit':0,
'warnings':3600,
}
, ]
}
,
# Entry for site `mail.frequentis.com' begins:
{
"pollname":"mail.frequentis.com",
'active':FALSE,
"via":None,
"protocol":"IMAP",
'port':0,
'timeout':300,
'dns':TRUE,
"aka":None,
'users': [
{
"remote":"imaptest",
"password":None,
'localnames':["esr"],
'fetchall':TRUE,
'keep':FALSE,
'flush':FALSE,
"mda":None,
'limit':0,
'warnings':3600,
}
, ]
}
]
}Python could then evaluate the fetchmail –configdump output and have the configuration available as the value of the variable “fetchmail”.
This wasn’t quite the last step in the dance. What I really wanted wasn’t just for fetchmailconf to have the existing configuration, but to turn it into a linked tree of live objects. There would be three kinds of objects in this tree: Configuration (the top-level object representing the entire configuration), Site (representing one of the sites to be polled) and User (representing user data attached to a site). The example file describes five site objects, each with one user object attached to it.
I had already designed and written the three object classes (that’s what took four days, most of it spent getting the layout of the widgets just right). Each had a method that caused it to pop up a GUI edit panel to modify its instance data. My last remaining problem was somehow to transform the dead data in this Python initializer into live objects.
I considered writing code that would explicitly know about the structure of all three classes and use that knowledge to grovel through the initializer creating matching objects, but rejected that idea because new class members were likely to be added over time as the configuration language grew new features. If I wrote the object-creation code in the obvious way, it would be fragile and tend to fall out of sync when either the class definitions or the initializer structure changed.
What I really wanted was code that would analyze the shape and members of the initializer, query the class definitions themselves about their members, and then adjust itself to impedance-match the two sets.
This kind of thing is called metaclass hacking and is generally considered fearsomely esoteric–deep black magic. Most object-oriented languages don’t support it at all; in those that do (Perl being one), it tends to be a complicated and fragile undertaking. I had been impressed by Python’s low coefficient of friction so far, but here was a real test. How hard would I have to wrestle with the language to get it to do this? I knew from previous experience that the bout was likely to be painful, even assuming I won, but I dived into the book and read up on Python’s metaclass facilities. The resulting function is shown in Listing 3, and the code that calls it is in Listing 4.
Listing 3. Metaclass Function
def copy_instance(toclass, fromdict):
# Initialize a class object of given type from a conformant dictionary.
class_sig = toclass.__dict__.keys(); class_sig.sort()
dict_keys = fromdict.keys(); dict_keys.sort()
common = intersect(class_sig, dict_keys)
if 'typemap' in class_sig:
class_sig.remove('typemap')
if tuple(class_sig) != tuple(dict_keys):
print "Conformability error"
# print "Class signature: " + `class_sig`
# print "Dictionary keys: " + `dict_keys`
print "Not matched in class signature: " + `setdiff(class_sig, common)`
print "Not matched in dictionary keys: " + `setdiff(dict_keys, common)`
sys.exit(1)
else:
for x in dict_keys:
setattr(toclass, x, fromdict[x])Listing 4. Code that Calls Metaclass Function
# The tricky part--initializing objects from the
# configuration global
# `Configuration' is the top level of the object
# tree we're going to mung
Configuration = Controls()
copy_instance(Configuration, configuration)
Configuration.servers = [];
for server in configuration[`servers']:
Newsite = Server()
copy_instance(Newsite, server)
Configuration.servers.append(Newsite)
Newsite.users = [];
for user in server['users']:
Newuser = User()
copy_instance(Newuser, user)
Newsite.users.append(Newuser)That doesn’t look too bad for deep black magic, does it? Thirty-two lines, counting comments. Just from knowing what I’ve said about the class structure, the calling code is even readable. But the size of this code isn’t the real shocker. Brace yourself: this code only took me about ninety minutes to write–and it worked correctly the first time I ran it.
To say I was astonished would have been positively wallowing in understatement. It’s remarkable enough when implementations of simple techniques work exactly as expected the first time; but my first metaclass hack in a new language, six days from a cold standing start? Even if we stipulate that I am a fairly talented hacker, this is an amazing testament to Python’s clarity and elegance of design.
There was simply no way I could have pulled off a coup like this in Perl, even with my vastly greater experience level in that language. It was at this point I realized I was probably leaving Perl behind.
This was my most dramatic Python moment. But, when all is said and done, it was just a clever hack. The long-term usefulness of a language comes not in its ability to support clever hacks, but from how well and how unobtrusively it supports the day-to-day work of programming. The day-to-day work of programming consists not of writing new programs, but mostly reading and modifying existing ones.
So the real punchline of the story is this: weeks and months after writing fetchmailconf, I could still read the fetchmailconf code and grok what it was doing without serious mental effort. And the true reason I no longer write Perl for anything but tiny projects is that was never true when I was writing large masses of Perl code. I fear the prospect of ever having to modify keeper or anthologize again–but fetchmailconf gives me no qualms at all.
Perl still has its uses. For tiny projects (100 lines or fewer) that involve a lot of text pattern matching, I am still more likely to tinker up a Perl-regexp-based solution than to reach for Python. For good recent examples of such things, see the timeseries and growthplot scripts in the fetchmail distribution. Actually, these are much like the things Perl did in its original role as a sort of combination awk/sed/grep/sh, before it had functions and direct access to the operating system API. For anything larger or more complex, I have come to prefer the subtle virtues of Python–and I think you will, too.
Resources