He's surprised that it happens so easily. He expected more of a fight, expected to be pushed off and denied, but the fact that it doesn't happen means something. He's just not quite sure what. Consciously, he's more disappointed that there's not a fight.

"Did we venture out more than once?"