I think all modern OS return the memory when the program exits.
Memory leaks are a massive problem in programs that are left running for long periods, as they slowly but inevitably drive the machine (or at least that program) to a crash.
Right, eventually you had to reboot the machines. Dos had this problem, but it had a ton of cool issues like allowing any program to edit any other programs(including the OS itself!) memory space! That will crash you on anything modern. Stuff from the mid 80s to early 90s. Around 1993 the internet was becoming mainstream and people were starting to think about hacks and security a bit, as well as stability and error recovery. To be fair, unix already discovered a lot of that stuff for corporate hardware, but it wasn't used on personal machines yet.
The crappy OSs that jonnin talks about only apply to desktop operating systems. OS's that do proper memory management and isolate memory in each process have been around since the 1970's or earlier, but they were in the "big iron" mainframe computers of the day.
Virtual memory management, which is the key to both concepts, takes up transistors and must be fast. Both things mean that it was expensive in the 70's and 80's so it wasn't available. I think the 386 was the first x86 processor with memory management that was capable of isolating a user program from the OS and other programs. The 386 was released in 1985 and it took at least 10 years for Microsloth to really start using its features.