Restarting Router To Retrieve Connection?

I'm having an issue with my wireless router and I don't think it's normal. Basically, I would be on my computer or laptop, generally browsing the Internet, and all of a sudden the Internet will stop working. A small yellow exclamation mark appears beside the wireless icon in the system tray. In order to resolve this, I go downstairs and turn the router off and then turn it back on again. When the router loses the connection, the light on the router goes out (the one that says Internet), yet everything else is on as it should be. Restarting the router seems to fix this problem. I just can't understand why I have to do this at home when the wireless router at work never has an issue. It's literally on 24/7 and handles everything from laptops to iPhones and even iPads.

Is this normal for my router? I had this exact same issue with my previous one and so I ended up buying an N range router (the last one was G). The guide book says that you shouldn't ever turn it off as that could cause problems, but whenever it loses the Internet connection, I'm forced to restart it in order to get the connection back.