Other elected MPs came out to witness the swearing in