静态类型化使代码更快

Cython 是 Python 编译器。这意味着它可以编译正常 Python 代码无需改变 (除尚未支持的语言特征的一些明显异常外,见 Cython 局限性 )。不管怎样,对于性能关键代码,添加静态类型声明经常很有帮助,因为它们允许 Cython 跳出 Python 代码的动态性质并生成更简单且更快的 C 代码 - 有时快几个数量级。

然而,必须注意,类型声明使源代码更冗余,因此可读性较差。因此,不鼓励在没有很好原因的情况下使用它们 (譬如:若基准测试证明它们真的使代码在性能关键区间实质上更快)。通常,在正确位置上的一些类型有很长的路要走。

所有 C 类型均可用于类型声明:整数和浮点类型、复数、结构、Union 和指针类型。Cython 可以自动且正确地在类型赋值之间转换。这也包括 Python 的任意大小整数类型,其中转换 C 类型时的值溢出将引发 Python OverflowError 在运行时 (不管怎样,它不会在进行算术时校验溢出)。在这种情况下,生成的 C 代码将正确且安全地处理 C 类型平台从属大小。

类型凭借 cdef 关键字声明。

类型化变量

考虑以下纯 Python 代码:

def f(x):
    return x ** 2 - x
def integrate_f(a, b, N):
    s = 0
    dx = (b - a) / N
    for i in range(N):
        s += f(a + i * dx)
    return s * dx
					

简单从 Cython 编译它仅仅给出 35% 的加速。这比什么都没有更好,但添加一些静态类型会使差异更大。

带额外类型声明,这可能看起来像:

def f(double x):
    return x ** 2 - x
def integrate_f(double a, double b, int N):
    cdef int i
    cdef double s, dx
    s = 0
    dx = (b - a) / N
    for i in range(N):
        s += f(a + i * dx)
    return s * dx
					

由于迭代器变量 i 被类型化采用 C 语义,for 循环将被编译成纯 C 代码。类型化 a , s and dx 很重要,因为它们涉及 for 循环中的算术;类型化 b and N 使差异更少,但在这种情况下,保持一致并类型化整个函数并没有太多额外工作。

这产生 4 倍加速,相比纯 Python 版本。

类型化函数

Python function calls can be expensive – in Cython doubly so because one might need to convert to and from Python objects to do the call. In our example above, the argument is assumed to be a C double both inside f() and in the call to it, yet a Python float object must be constructed around the argument in order to pass it.

Therefore Cython provides a syntax for declaring a C-style function, the cdef keyword:

cdef double f(double x) except? -2:
    return x ** 2 - x
					

Some form of except-modifier should usually be added, otherwise Cython will not be able to propagate exceptions raised in the function (or a function it calls). The except? -2 means that an error will be checked for if -2 is returned (though the ? indicates that -2 may also be used as a valid return value). Alternatively, the slower except * is always safe. An except clause can be left out if the function returns a Python object or if it is guaranteed that an exception will not be raised within the function call.

A side-effect of cdef is that the function is no longer available from Python-space, as Python wouldn’t know how to call it. It is also no longer possible to change f() 在运行时。

使用 cpdef keyword instead of cdef , a Python wrapper is also created, so that the function is available both from Cython (fast, passing typed values directly) and from Python (wrapping values in Python objects). In fact, cpdef does not just provide a Python wrapper, it also installs logic to allow the method to be overridden by python methods, even when called from within cython. This does add a tiny overhead compared to cdef 方法。

加速:150 倍相比纯 Python。

确定在哪里添加类型

Because static typing is often the key to large speed gains, beginners often have a tendency to type everything in sight. This cuts down on both readability and flexibility, and can even slow things down (e.g. by adding unnecessary type checks, conversions, or slow buffer unpacking). On the other hand, it is easy to kill performance by forgetting to type a critical loop variable. Two essential tools to help with this task are profiling and annotation. Profiling should be the first step of any optimization effort, and can tell you where you are spending your time. Cython’s annotation can then tell you why your code is taking time.

使用 -a switch to the cython command line program (or following a link from the Sage notebook) results in an HTML report of Cython code interleaved with the generated C code. Lines are colored according to the level of “typedness” – white lines translate to pure C, while lines that require the Python C-API are yellow (darker as they translate to more C-API interaction). Lines that translate to C code have a plus ( + ) in front and can be clicked to show the generated code.

此报告非常宝贵,当为速度优化函数时,及为确定当要 释放 GIL :一般而言, nogil 块可能只包含白色代码。

../../_images/htmlreport.png

Note that Cython deduces the type of local variables based on their assignments (including as loop variable targets) which can also cut down on the need to explicitly specify types everywhere. For example, declaring dx to be of type double above is unnecessary, as is declaring the type of s in the last version (where the return type of f is known to be a C double.) A notable exception, however, is integer types used in arithmetic expressions , as Cython is unable to ensure that an overflow would not occur (and so falls back to object in case Python’s bignums are needed). To allow inference of C integer types, set the infer_types directive to True . This directive does a work similar to the auto keyword in C++ for the readers who are familiar with this language feature. It can be of great help to cut down on the need to type everything, but it also can lead to surprises. Especially if one isn’t familiar with arithmetic expressions with c types. A quick overview of those can be found here .

内容表

上一话题

构建 Cython 代码

下一话题

教程

本页