How Python was Shaped by leaky Internals, Armin Ronacher, Flask framework

Видео с мероприятий {speach! · 20K views · 2016 · 49:55

Open on YouTube

Python, Ruby & scripting languagesSoftware performance & optimizationProgramming language designtalk

Why Hacker News recommends it

It is HN's standard answer to why Python cannot easily get much faster, and is recommended by people who work on JITs and alternative runtimes.

What it is

Armin Ronacher, creator of Flask, explains how CPython's internals and C API leak into the language's semantics. Using examples like the complexity of the add operation, he shows why Python is hard to optimize or reimplement.

Recommended by 15 HN commenters across 16 different months, 2017 to 2025.

What Hacker News said

Strong rec ericfrederich · 2017-01 · on “Grumpy: Go running Python”

Python needs a new runtime. This talk shows how bad of shape it's really in. [this video] Basically, the language doesn't have a "spec" per-se. The language is whatever the defacto CPython implementation happens to do within it's giant eval loop. Another great talk about CPython internals: [LINK]

Strong rec eigenspace · 2020-02 · on “JuliaLang: The Ingredients for a Composable Programming Language”

As others have mentioned, Python does have JIT compilers. The problem is that havign a JIT doesn't solve the problem. PyPy is often a factor of 10 behind julia performance and projects like Numba, PyTorch (the PyTorch people had to build their own Python JIT compiler yikes!), etc. will always have a more restricted scope than a project like Julia because Python's very semantics make many optimizations impossible. Here's a great talk from the author of the famous Flask library in Python: [this video] where he discusses the fundamental problems with Python's semantics. If you fix these problems, you will end up changing Python so fundamentaly that you'll really have a new language. Generic CPython

Strong rec nerdponx · 2020-09 · on “Python's Innards: Introduction (2010)”

I highly recommend this talk by Armin Ronacher as an accompaniment (or prologue, or epilogue) to the article: "How Python was shaped by leaky internals", [this video]

Strong rec arc776 · 2020-10 · on “Attempts to make Python fast”

As the OP states: > mutable interpreter frames, global interpreter locks, shared global state, type slots On top of this, Python is extremely dynamic and nothing can be assured without running through the code. So this leads to needing JITs to improve performance which then give a slow start up time and increased complexity. Even with JIT, Python is just not fast thanks to the above issues and it's overall dynamism. It can be optimised and for sure there's some impressive attempts at doing so. However I don't think pure Python will ever be considered "fast" as these things necessarily get in the way. I highly recommend the two videos posted here that go into more detail as to why there are l

Strong rec haberman · 2021-09 · on “Writing Python Extensions in Assembly”

Python is difficult to optimize due to complex language semantics. This excellent talk describes how the semantics of the "add" operation are much more complex than generally described. a+b is not just a.__add__(b), you have to read 200+ lines of CPython interpreter code to fully resolve its semantics: [this video] Not saying it's impossible, but it's more of an uphill battle than optimizing other languages that have more regular semantics. Python is also constrained by its C API, which is a very low-level and exposes a lot of implementation details to C extensions. Changing the API could improve performance but it would also make Python incompatible with a large ecosystem of C extensions. I some

Strong rec haberman · 2021-11 · on “Pyjion – A Python JIT Compiler”

I think "a + b" is a good example. I really recommend the talk I linked ( [this video] ), it gets deep into this stuff. I don't think anyone benefits from "a + b" having so many special cases.

Strong rec fault1 · 2022-01 · on “Faster CPython (2021) [pdf]”

I thought this presentation by Armin Ronacher (the creator of Flask and Jinja) was very enlightening: How Python was Shaped by leaky Internals(2016): [this video] See also this HN post (Attempts to make Python fast): [LINK]

Recommends the_duke · 2017-01 · on “Grumpy: Go running Python”

Someone else already posted this link: [this video] It explains why CPython can't improve on many things.