There's an amused huff at the question. Wouldn't that be quite the holiday... "Curiosity."

Maybe he's just trying to suss out more of what Bucky has done in Zemo's absence. He shifts his weight a bit, looking Bucky over. "Did many know of your-- history there?"

If they did, it doesn't sound as if they judged him overly harshly.