It is normal if it just happens while you plug in and out. You know, most of recent electric appliance (for example TV, DVD player, etc.) has the remote function. That means, you can turn on an electric appliance with a remoter without pressing the power on button on the appliance. In another word, even if the electric appliance is power off, there is still electricity to support the remote function; otherwise the remote function does not work. So, for the electric appliance with the remote function, it is not really power off after you press the power off button. The microwave oven is same, because it has an LCD screen. But you cannot see the spark for a personnel computer, because it is not remote.
By the way, it is recommended to switch the electric power off when you leave home, in order to save the remote electricity consumption.