Ukemi is the one who receives, Torimi the one who takes.
Usually in Dojo outside of Japan you will hear the terms attacker and defender being used for Yakusoku gumite. Although it does not really matter, as the physical act is what ultimately counts, students can be put off track. Hence, if the student that is called defender considers himself to be defending misses the point during practice.
Torimi, who is usually called defender, is not defending but should be taking the initiative.
Tori means to take after all.
Ukemi is the one usually called attacker. Although Ukemi does attack, the name comes from uke, to receive. For example, Ukemi executes jodanzuki (attack). Torimi executes harai uke, pulls Ukemi off balance creating kuzushi (attack) and counter attacks immediately. Hence, Ukemi receives the counter attack.
Although you could call harai uke a defence, it should never be executed merely as defense. It should always be executed as offense setting up a counter attack.