Many players underuse their keyboards while playing WoW. Using keybindings is a vital way to speed up your responsiveness and awareness to the game. It can be the difference between a 1900 and a 2100 arena rating, or the difference between 2000 and 2300 dps in a raid.
Clicking your action bars with your mouse is only okay in a very limited number of scenarios. Spells you would never use in combat or in a panic situation are spells that you don't have to keybind. Silly macros for things like /target Xaxziminrax /kiss are okay to click. Long-term buffs, rogue poisons, mage portals... these are things you would not use in an emergency, so it's okay to leave them as mouse-clicked.
Resurrection spells, food and water, and short-term buffs (Warrior shouts, shaman elemental shields, mage and warlock fire/frost/shadow wards, hunter aspects) are all things that you should keybind. All these spells are cast either while in combat, right before you enter combat, or right after you leave combat.
An entire well-written, with-pictures article has already been posted, and I won't reinvent the wheel. Take a look at TankSpot's General Keybinds for some detail on how and why to bind.
After you've read that over, there are a few more things to mention.
The numpad is a fabulous way to have all your buffs and non-combat spells in order. I use the big "zero" button to use my Mana Strudels because its easy to find and I can just start spamming it the moment I think I'm about to leave combat. I use 1-9 on the numpad for my buffs. With 3 different mage armors, 30 and a 60 minute intellect buffs, dampen/amplify magic, and two conjure food spells, I'm really glad to have such an out of the way yet easy to remember section of keys. You can even put your hearthstone over there! It's not that hard to remember, "Seven means Dalaran." And then you still have plus and minus, asterik and slash left over for whatever you desire.
I also use backslash. On all my keyboards, it's below Backspace and above Enter and it's really big so its easy to find. I have a non-emergency but vital spell bound there. Shattering Throw, Evocation, or Fire Elemental Totem are good choices because they will be used fairly infrequently, but are still important enough that you'll want to have quick access to them.
Remember: Take it slow at first. Don't overwhelm yourself! Frustration is not the way to go, and even if it takes you a whole month just to remember two spells on your keyboard... that's better than nothing, and will give you a definite edge against people who haven't any bound!