Challengee

A person who is challenged or issued a challenge by another person, especially in sports or competition.

From challenge plus the suffix -ee (from French -é, meaning 'one who receives the action'). This suffix creates the opposite role from -er, so a challenger challenges, while a challengee is challenged.