To sign off, hold your non-dominant hand horizontally, with your palm down. Put your dominant hand on top of your other hand, and tilt it up so that it goes from being horizontal to vertical. Bend your dominant forearm at a right angle.
The sign is like your hands are a switch moving to the off position.
I like to teach on and off together as a pair of signs. I will demonstrate switching something on, for example a toy that makes sound, while making the sign - then demonstrate switching it off while making the sign. Your baby can join in the game making, the on sign and switching on the toy. Then making the off sign and switching off the toy.
Teach the off and the on signs together as a pair. Switch the radio on and off, using the contrast to teach the baby the sign.