3. Training & Optimization
Gradient Descent (மலை இறங்குதல் / Walking Down the Hill)
Let's walk down Ooty hills! (ஊட்டி மலையில இறங்குவோமா!)
Technical Meaning: சாய்வு இறக்கம் (Saaivu Irakkam) - An optimization algorithm to find the minimum of a function.
The Core Idea
Gradient Descent is the math algorithm that tells a neural network how to change its weights. Its entire goal is to minimize "loss" or "error"—finding the very bottom of a mathematical valley where the AI is making the fewest mistakes.
The Origin Story
If a neural network has millions of weights, trying every possible combination to find the best one would take thousands of years. We needed a shortcut. Gradient Descent uses calculus (specifically derivatives) to calculate the "slope" of the error at its current position, allowing the network to know which direction to step to reduce the error.
The Tamil Analogy
Imagine you are stranded at the top of the Ooty Hills (ஊட்டி மலை) at midnight. It is pitch dark, there is heavy fog (பனி மூட்டம்), and you have no flashlight. Your goal is to reach the valley at the bottom (where the error is zero).
How do you get down? You use your feet to feel the slope (க்ரேடியன்ட்) of the ground around you.
- You feel that the ground slopes downward to your left.
- You take a step (the learning rate) to the left.
- You stop, feel the slope again, and take another step.
If you take baby steps (low learning rate), you will reach the bottom safely but it will take all night. If you take giant leaps (high learning rate), you might accidentally jump over the valley and end up on the opposite mountain!
Try It Yourself
Adjust the learning rate (the size of your steps) and see if you can safely reach the bottom of the valley!
Concept 2 · Gradient Descent
Finding the lowest error
How does an AI know when it's right? It tries to minimize its 'error' (the height on the curve). The Learning Rate determines how big of a step it takes. Take a step down the mountain!
Good size steps. It should find the bottom quickly.
Current Error
Steps taken: 0