How Do I Soften My Aquarium Water?

Well I keep a small gl aquarium with a female guppy, and a little plastic container for six-day-old fry. Do fry need soft water? How do I soften my aquarium water? I know the driftwood/bogwood method, but since it turns my water a tea-color I want to know how to prevent that. Please no rude answers or answers related to using peat dust. Thanks!