Hold SHIFT + ERASE while switching on to wipe everything, factory sounds included.
This guide needs JavaScript for the playable device. It is unofficial and not affiliated with teenage engineering.