The one that hit me the hardest was "Grave of the Fireflies." It was made sadder because the little girl reminded me of my younger sister (who is no longer around). My dad said that she reminded him of her too.
The big Fushigi Yuugi one hit me real hard too. It wasn't because of the surprise of it. It was because the way the scenes were handled. That was tough to take.
Other anime that come to mind that teared me up:
Kanon - Three of the girls stories I found to be very sad. The stories kept getting sadder as it went along. Ayu's story was extremely sad. Too bad the last episode backtracked on some of the stuff that happened earlier.
The end of Saber Marionette J.
Refrain Blue episode 3. That was a classic Japanese "cheering you up even when I'm gone" story.
Suika episode 2. This was another disappearing character story except that the character never returned. Sometimes the impact of the story is deadened because the character shows up somewhere in the future. In this case the character never came back so all those held back feelings could never be expressed to the character again. Ever.
Kare Kano episode 16 about Miyazawa's parents. Was just very sad to me. The impact of a goal that was never realized. At least that's how it felt initially when taken just on it's face.
Rurouni Kenshin First OVA. Tragic.
Kakyuusei TV episode 12 - This is the only episode I can list as being very sad and yet happy at the same time. "Tearful happy" for one character and "tearful sad" for the other.
Who's Left Behind - This was another World War II civilian story that was based off a true story. But it was in no way half as sad as "Grave of the Fireflies."
I know the Rurouni Kenshin TV series had a couple of moments got to me, but I can't remember exactly where they were.
