Wifi Stops Working When Wired Network Plugged In?

On the basement level, and on level 2, I have in wall junction boxes where the Cat5E Ethernet wires come together. There are 6 lines that go from the level 2 box to 6 wall jacks on levels 2 and 3. There are also 6 lines that go from the basement box to 6 wall jacks in the basement and on level 1. There is also a Cat5E line running between the two boxes.I have wired all these plugs on both ends with T568B standard, and in each box they go into a patch panel. I have used a cable tester to test and make sure all these plugs are properly wired. I have also tested individual computers hardwired linked through to my router (without going through switches) and confirmed that they work that way, so I'm pretty sure I've wired it correctly.

At the second floor box, I have an AT&T 2Wire wifi router and modem, with AT&T Uverse service. I also have a monoprice Ethernet switch. In the basement box, I have another monoprice Ethernet switch (link to the type of switches I have is below). The switches are connected together via the Cat5E that runs between the boxes, and the second floor switch is connected to a free port on the back of the AT&T 2Wire Uverse wifi router.These wall plugs are not all currently used. In fact the problem I describe below occurs even if nothing is plugged into the wall plates.My problem is: when I hook up everything, so my wifi router is wired to the first switch, which is in turn wired to the 6 wall plugs upstairs as well as to the switch in the basement, and that basement switch is in turn wired to the 6 wall plates downstairs... at that point when I plug it in, my wifi stops working. Basically, whenever my router is plugged into this hard wired network, the wifi stops working. When I unplug the network, the wifi comes back (I usually reset it, not certain if it would come back just from waiting).

Interestingly, it doesn't necessarily stop working if most or all of the plugs are removed so the network is not connected (for example, if I just go through one plug instead of all of them)Does a network of wired Ethernet, even if not plugged into anything, cause some sort of a load or drain on the router that could make it stop working?