Ray Summit 2022 - Day 1 Keynote - Fireside Chat
Why Hacker News recommends it
Cited as a source for the fact that OpenAI uses Ray for training, and for Brockman's views on developer speed.
What it is
A Ray Summit 2022 keynote fireside chat between OpenAI's Greg Brockman and Anyscale's Robert Nishihara. They discuss how OpenAI's training infrastructure evolved and its use of Ray.
Recommended by 0 HN commenters across 0 different months, 2023 to 2023.
What Hacker News said
Cites robertnishihara · 2023-02 · on “How to Train Large Models on Many GPUs?”
I'm one of the Ray developers, thanks for the shoutout :) If you're curious about how Ray is used for LLMs, here are some interesting examples of LLM projects using Ray! - Alpa does training and serving with 175B parameter models [LINK] - GPT-J [LINK] - Another HN thread on training LLMs with Ray (on TPUs in this case) [LINK] - OpenAI fireside chat on the evolution of their infrastructure and usage of Ray for training [this video] - Cohere on their architecture for training LLMs [LINK] Some other thoughts 1. There is a lot more we want to do to make Ray better for working with large language models and for making training, serving, and batch inference work well out of the box. 2. The original pos
Cites sashank_1509 · 2023-06 · on “Speed matters: Why working quickly is more important than it seems”
Consistently I see this advice from everyone who I consider has achieved outside success in their field. Greg Brockman (OpenAI Cofounder) : Developer Speed is the most important thing I care about: ( [this video] ). I believe he said more about this topic but I can't find it now. Frank Slootman (Snowflake CEO): In his book Amp It Up, he stresses why pushing people to move fast is extremely important Elon Musk (Let's not get into a debate about Elon and just agree that he has helped setup 2 100bn+ companies) : "I believe in maniacal sense of urgency, if you can implement something right after the meeting, do that", upon his work philosophy when taking over twitter. In my personal experience one of
Cites waleedk · 2023-08 · on “Llama 2 is about as factually accurate as GPT-4 for summaries and is 30X cheaper”
[Author] OpenAI uses Ray, the open source version of Anyscale Platform, for training their models (video evidence: [this video] ). Ray ( [LINK] ) is available for anyone to use. You can also use Aviary ( [LINK] ) to serve any of those models yourself.