「あっそ」と流すのもよし、本気にするのもよし。
You can let it slide with a "oh?" or you can take it seriously.
もちろん冗談と思って彼は「あっそ」と流していましたけどかなり嫌だったそうです。（ってゆうか冗談でもそんなこと言うな！）
Of course he thought it was a joke and let it slide with a "hm?" but he was seriously annoyed. (I mean, just don't say things like that even as a joke!)
