The main symbols date back to the 1960s, with the Pause symbol having reportedly been invented at Ampex during that decade for use on reel-to-reel audio recorder controls, due to the difficulty of translating the word "pause" into some languages used in foreign markets. The Pause symbol was designed as a combination of the existing square Stop symbol and the caesura, and was intended to evoke the concept of an interruption or "stutter stop". The right-pointing triangle was adopted to indicate the direction of tape movement during playback.

Source: Wikipedia