It depends upon your comfort level, and your rapport with the masseuse. Maybe for your first massage you will feel more relaxed, if you keep your undies on. Don't worry, the masseuse can work around them. After a few visits you may not mind being naked.
Enjoy your massage.