It's probably partly because it's so... different than what everyone else comes to this world remembering. It's only very recently that we've encountered travelers from other planes, besides the Shadow World.

I can imagine it's a little harder to adjust when there's only three of you from that entire world. Well, three of you around here at least.