When a program reserves a fixed block of memory on the stack, like a 16-byte variable for a username, functions like strcpy in C copy user input without checking its size. Right next to this buffer sits the function's return address, which tells the CPU where to resume execution once the function finishes. If an attacker inputs 40 characters instead of 16, the excess text spills outside the buffer and overwrites that return address. By pointing the new address back to malicious machine instructions injected inside the input, the attacker can hijack the CPU to execute their own code.
To stop these exploits, modern systems rely on two essential defenses. First, compilers place a hidden random value called a stack canary directly between the buffer and the return address. If an overflow modifies this number, the program immediately crashes before any malicious code can run. Second, operating systems enforce NX or DEP, which marks stack memory as non-executable. Even if an attacker successfully changes the return pointer, the CPU simply refuses to run code residing inside a data buffer.
#CyberSecurity #SoftwareEngineering #TechExplained #Coding #CProgramming #SystemDesign #Interview #InterviewQuestion