Personally, I think everyone needs to cover the basics, then start moving on to specialized stuff.
Starting out, you really only need 4 guns to cover basically all your needs.
1, a .22 rifle. Most of them are very simple, very durable and extremely well proven. Ruger just came out with a takedown model of the 10/22, which is pretty enticing (to me)
2, a shotgun. 20 or 12 gauge will serve you very well here. Lots of options in the under $300 range. For home defense, I recommend a pump. Mossberg sells many model 500's with 2 barrels, one for home and one for field. That's a nice option.
3, a rifle. Usually a bolt action, pretty much anything will work. Savage makes some good quality rifles in just about any caliber you'd want for not a lot of $$$
4, a handgun. I can't recommend Glock highly enough for a first handgun. Dead simple, dead reliable. Just remember, a handgun is a tool to help you get to the real firepower... It's not magic.
Once you have the 4 bases covered, start looking at the neat, shiny toys.
Those are my thoughts on this. Carry on.