AWS re:Invent 2016: Tuesday Night Live with James Hamilton
Why Hacker News recommends it
HN treats Hamilton's annual talks as the best window into how AWS regions and hardware are actually built.
What it is
James Hamilton's Tuesday Night Live keynote at AWS re:Invent 2016, touring AWS infrastructure at scale including networking, power and custom silicon. It also features NASA JPL's Tom Soderstrom.
Recommended by 5 HN commenters across 5 different months, 2016 to 2023.
What Hacker News said
Strong rec nodesocket · 2016-12 · on “Links to YouTube Recordings of AWS re:Invent Sessions”
The keynote by James Hamilton[1] is absolutely awesome. You can genuinely tell that James finds all this technology fascinating. He talks about fiber, networking, power, redundancy, scale all with utter enthusiasm. [1] [this video]
Recommends strong_silent_t · 2018-07 · on “Backblaze Durability Is Eleven 9s – And Why It Doesn’t Matter”
Others have answered, but I think the general principle for a Region is that between the AZs you have less than 1ms latency, physical separation between the data centers, but they may still be in the same floodplain, be able to be hit by the same hurricane, etc. More info here if you want to see how regions are structured at a high level of detail: [this video]
Recommends ignoramous · 2020-04 · on “Now Open – AWS Africa (Cape Town) Region”
James Hamilton [0][1] and others [2][3][4] have given pretty good overview over the years of the AWS infrastructure. [0] AWS Innovation at Scale (2014), [LINK] [1] Tuesday Night Live (2016), [this video] [2] Operation of the AWS Global Network (2019), [LINK] [3] Foundations of AWS Infrastructure (2018), [LINK] [4] Exploring the AWS Global Network (2018), [LINK]
Recommends EwanToo · 2021-07 · on “Amazon EC2-Classic Is Retiring”
Watch this video from 2016 by James Hamilton at AWS Re:Invent [this video] It'll tell you lots about how AWS is built
Recommends mackid · 2023-12 · on “The art of high performance computing”
Mark Russinovich gives a good talk most years on the internals of Azure and the systems that run it. [1] is an example. Look for talks from other years as well. Meta also publishes a number of papers/blogs/OSS projects on their engineering site [2] James Hamilton of AWS gives a talk most years on their infrastructure. Worth watching multiple years [3]. [1] [LINK] [2] [LINK] [3] [this video]
Cites douglasfshearer · 2016-12 · on “Liberouter Combo Cards – FPGA boards focused on network data processing”
Amazon announced they were doing network processing in hardware at Re:work last week [1]. Custom silicon, rather than the FPGA or ASICS they were using when they started the project in 2012. They tout not only the freed CPU resources, but lower latencies and increased security of isolating the networking from the CPUs running hosted VMs. [this video]
Cites rbranson · 2018-10 · on “Packets per Second Limitations in EC2”
James Hamilton talked about their commitment to 25GbE hardware at Reinvent in 2016. Fast forward to ~23m. [this video]