When you connect a USB drive to a Computer, it gains the freedom to read/write data on the USB drive. The data which is being written on your USB drive from your Computer is not directly stored on the USB drive. But caching process occurs over here. During caching the data is not directly written on your USB drive. Instead it is stored on a temporary memory (RAM).
And if you are about to yank the USB drive directly without clicking the safely remove option. Then you may end up with a corrupted file on your USB drive. This is because the file was not completely transferred, instead was being cached to your computers memory. This is why users are recommended to click ‘Safely remove hardware icon’ on their windows computer.
However, Windows automatically disables the caching process on USB drives when read / write operations are in idle state. So you don’t have to click the safely remove option if you are not writing or reading anything on the USB drive.
The Safely remove option is there for extra security. To make sure that you wont lose any valuable data. Hence preventing you from destroying your own data.
Yanking out the USB drive directly has rarely caused any serious issues. But there are cases when the USB drive becomes corrupt and unusable. So its always recommended to click safely remove option. Its there for your own good.