Machine Learning

timelapse 3

- was going through lilian weng policy grad blog
- covered till a3c, went through pg theorem proof & found another great blog
- next up is dpg, ddpg & d4pg
- haven't implemented them only theory as of now but will do it ofc
- & this timelapse looks good niceee :) https://t.co/hyt6yB4mQi

timelapse 2

i think session wise timelapses are good & then will merge one for the day.
still need to speed up things, anyways this one's better ig :) https://t.co/kj9zm8cjNU

ppo: proximal policy optimization is the math that allows the model to learn from the reward model without changing its personality too drastically. it keeps the updates small and stable so the model doesn't go off the rails. [56/100]

reward model: during rlhf, we actually train a second judge model that learns to predict what a human would like. this judge then watches the main model work and gives it a thumbs up or thumbs down millions of times. [55/100]

test timelapse ig, bruh there is hell lot of light source need to cover it or make it translucent :) https://t.co/r7N5dbkIdk

rlhf: reinforcement learning from human feedback is the secret sauce for alignment. we have humans rank different ai responses and then train the model to maximize the score given by those human preferences, making it safer. [54/100]

instruction tuning: this is a specific type of fine-tuning where we teach the model to follow specific commands, like summarize this or write a poem. it is what makes ai feel like a tool rather than just a generator. [53/100]

training loss curve: engineers spend their lives staring at a downward graph of loss. if the line stops going down, it means the model has stopped learning or has hit a bottleneck, and it is time to change the architecture or data. [51/100]

pre-training: this is the most expensive phase where the model reads the entire internet. it is where it learns the fundamental structure of human thought and language, before it is taught any specific tasks or rules. [50/100]

speculative decoding: we can speed up giant models by having a tiny assistant model guess the next five words and then asking the giant model to verify them all at once. if the small model was right, we save a lot of time and compute. [49/100]

flash attention: one of the biggest bottlenecks in ai is moving data around the gpu. flash attention is a brilliant mathematical re-write of the attention process that makes it fit into the fastest memory on the chip, speeding up training by several times. [43/100]

quantization: this is the art of squashing giant models down by using 4-bit or even 2-bit numbers. it is what allows you to run a massive, world-class ai on a simple laptop or phone that would otherwise be way too weak. [42/100]

floating point precision: ai models are usually built with very precise 16-bit or 32-bit numbers, but we are finding that we can often use much less precision without the model getting dumber, which saves massive amounts of power and memory. [41/100]

perplexity: this is a mathematical measurement of how confused a model is by a piece of text. a lower perplexity means the model found the text very predictable and easy to understand, which is a sign of a well-trained brain [37/100]

HeartMuLa shows why music generation is not just “text to audio”

- separate global structure from local detail
- compress audio into semantic tokens
- generate minutes of music autoregressively
- this design pattern will show up everywhere https://t.co/Emllhjxyas

adam optimizer: standard gradient descent is a bit clunky, so we use adam. it acts like a smarter version that remembers the momentum of past updates and adjusts the speed for each individual weight, making the whole training process much smoother [33/100]

AI Engineers: stop shipping notebook spaghetti into production!

Python Object-Oriented Programming (5th Edition) is a practical guide that teaches you how (and when) to apply OOP to build scalable, maintainable Python applications.

I reviewed this book because I keep reminding https://t.co/95SSTUs8QM

Build a Reasoning Model (From Scratch) https://t.co/RnxQ2z9VzH

I'm working with @RubenVeidt on my deep learning framework to support his image editor and more

because of that reason we are rewriting this framework to support future features and more features specific to images, videos etc... not just text and ofcourse we will use no

attention scores: the model calculates a score for every pair of words in a sequence to decide how much focus to put on each one it divides these scores by a constant number to keep the math stable so the training doesn't fail when the model gets deep [15/100]