A Hackers' Guide to Language Models

What Hacker News said

Strong rec tarruda · 2023-09 · on “A Hackers' Guide to Language Models [video]”

Thanks a lot for this video, best LLM usage tutorial I've seen so far. At [this video] when talking about valid use cases for a local model vs GPT4 is: "You might want to create your own model that's particularly good at solving the kinds of problems that you need to solve using fine tuning, and these are all things that you absolutely can get better than GPT4 performance". In regards to this, there's an idea I've been thinking about for some time: Imagine a chatbot that is backed by multiple "small" models (such as 7B parameters), where each model is fine tuned for a specific task. Could such a system outperform GPT4? Here's a high level overview how I imagine this to work: - Context/prompt is s

Strong rec dojitza1 · 2023-11 · on “Effective GPT-4 Programming”

An interesting solution to the blind spot error (taken directly from Jeremy Howard's amazing guide to language models - [this video] ) is to erase the chat history and try again. Once GPT has made an error (or as the author of this article says, the early layers have irreversibly pruned some important data), it will very often start to be even more wrong.

Strong rec tarruda · 2023-11 · on “Generative AI for Beginners”

If you're looking for a practical guide on how to use LLMs, highly recommend "Hackers Guide to language models" by Jeremy Howard. 1.5h video packed with practical information: [this video]

Strong rec mangoman · 2024-02 · on “I'm an Old Fart and AI Makes Me Sad”

I understand the sadness around not understanding it, it's fucking hard. however, there are more and more resources online getting published for how to get started understanding it that help with understanding the math at an abstraction that helps with learning how to build with it. I would strongly strongly strongly recommend starting with karpathy's from 0 to hero neural networks youtube course - it starts with building a tensor library and back propagation, explaining it in a way that finally clicked for me [LINK] jeremy howard also has a fantastic video that is more around how to use LLMs and such called a hacker's guide to language models - [this video] as i've dove more and more into it, i

Recommends code_biologist · 2023-10 · on “Mistral 7B Paper on ArXiv”

Jeremy Howard talks about it in his recent video "A Hackers' Guide to Language Models": [this video] That link goes directly to the timestamp where he discusses fine tuning, but the whole talk is great. Punchline, check out Axolotl: [LINK]

Recommends lsedgwick · 2023-10 · on “Advent of Code 2023's new AI/LLM Policy”

You're right about the basic mistakes they can make - they can also excel at the same tasks if prompted differently. But I was making a slightly different point. The point is they can reason about things in a better-than-chance way (I mean, a much-better-than-chance way) when given problems not in the training set. Have you read the Codex paper? Seriously, go look, an LLM even from years ago (which is like decades ago in ML-time) is often able to write code to solve novel programming problems that were handwritten to not be in the training set! Also, the benchmarks against which GPT3, InstructGPT, and GPT4 were tested are illuminating about its problem solving ability on novel problems, thes

Recommends ghj · 2023-12 · on “A new old kind of R&D lab”

Will you still be working on educational content on the side? (e.g. updating fast.ai and/or making one off lectures like [this video] ) Either way thank you for all the amazing free content you've already put out and good luck on the new endeavor!

Recommends simonw · 2024-03 · on “GPT-4, without specialized training, beat a GPT-3.5 class model that cost $10M”

For "citation needed", read "please link me to a blog post showing how, don't just tell me to Google for one". The internet is full of blog posts about this. That doesn't mean they're actually good - I'd love to be pointed at one that has proven itself useful for someone (and definitely isn't just LLM blog-spam). I don't care if it's trivial to fine-tune and get crap results - I care about fine-tuning where the result was worth the effort. For the record, my favourite guide to fine-tuning is the section of this Jeremy Howard video that shows how to train a text-to-SQL model: [this video]