Cisco Switching/Routing :: QoS / Queue Drops On 3750 Switches?
Apr 30, 2013
We currently have a site with a very simple topology that uses a 3750X switch stack for a collapsed core. Everyday, the users have a conference call and experience poor voice quality.Its not bad when users call from several conference phones, but when everyone calls in on individual phones, there is choppy and almost inaudible voice quality experienced. The voice traffic flow would be as follows: Phone <-> 3750 switch <-> Voice GW We have packet captures showing that RTP packet loss is occuring from the phone to the voice gateway, but none from the voice gateway to the phones. We also have drops in the output queues that match drops on the asics. I can reset the counters and they will be clear until the call, and then they increment significantly during the call. The voice gateway and phones are non-Cisco. The switch stack has 6 switches. We are trusting the DSCP settings on the phones. All the queue drops from the phones are usually in queues 0-3, but all drops on the voice gateway is in queue 0. Below are the QoS settings; they are mostly default and we have not changed any queuing, thresholds, or buffers. Should we specify larger buffers and threshold for a designated queue and send EF traffic to that queue?
MySwitch#sh mls qos
QoS is enabled
QoS ip packet dscp rewrite is disabled
trust state: trust dscp
I've been fighting what seems to be an increased number of outqueue drops on our core stack and edge switches for the last 3 or 4 weeks.(The core consists of a stack of 5 3750s in 32-gig stack mode. The wkgrp switches are 3560s. all are at 12.2.52) The wkgrp switches are directly connected to users. We use Nortel IP phones with the phone inline with the user PC. auto-neg to 100/full. [code] However I have tried turning off QOS on a couple of workgroup switches (no mls qos, but left individual port configurations the same) but am still seeing drops.Since I have disabled qos on the switches in question (no mls qos) (not the core tho) I am presuming these commands have no affect on the switch operation and therefore cannot be related to the problem. With QOS turned off one would presume that it is general congestion - especially at the user edge where busy PC issues might contribute. So I wanted to see if I could see any instances of packets in the output queues building up.
I wrote some scripts and macros that essentially did a snapshot of 'show int' every 20 seconds or so, and looked for instances of 'Queue: x/' where x was greater than zero.What I found after several days of watching the core stack, and a few of the workgroup switches that are most often displaying the behavior, was that I NEVER saw ANY packets in output queues. I often saw packets in Input queues for VLAN1, once in a great while I would see packets on input queues for fa or Gi interfaces, but NEVER on output queues. [ code] Additionally, when I look (via snmp) at interface utilization on interfaces showing queue drops (both core and wkgroup), they are occurring at ridiculously low utilization levels (as low as 4 to 8%). I've tried to look for microbursts between the core and a wkgroup switch where the core interface was experiencing drops, but haven't seen any (using observer suite). [code] While the queue-drop counts aren't critically high at this point, they are happening more frequently than in the past and I would like to understand what is going on... In most cases, no error counters are incrementing for these interfaces. Is there some mechanism besides congestion that could cause output queue drops?
I feel that 3560 and 3750 perform differently with the following two commands:
srr-queue bandwidth shape 5 0 0 0 srr-queue bandwidth limit 50 On 3750, the bandwidth for queue 1 is limited to 100mbps x 50% / 5 = 10mbps On 3560, the bandwidth for queue 1 is limited to the smaller value of BW / shape weight and BW x limit%.
Does it sound about right? is there a way to check for mls qos input queue drops? The show mls qos interface xxx stat only shows the output queue drops. Maybe for some reason the input queue never drops?
I have a 3750 as a core and have a series of HP Procurve switches that are daisy chained using one port. I have two vlans on the port now (6 &9) and everything works fine, all switches communicate and end devices on the switches are also talking. There is a requirement to add a device towards the end of the chain which requires it to connect using Vlan1. Once I add Vlan1 to the port onthe 3750 I lose connectivity to all the HP switches.
I can see drops on the 6509 Queue for interface gi1/6 , qos is disabled globaly with qos disabled all packets are in one Queue using best effort my question is if I can see drops using the sh queueing int Gi1/6 command why I am not seeing any drops when I run the Sh int (interface number ) command. [code]
we have two 6509E, as our core switches. Recently I noticed that on some connections I have a high output queue drop rate.
These 4 x 2 interfaces (gigabit) are connected to our blade encolure, consisting of 4 x WS-CBS3120X-S. The utilization of the links is really quite low, when I see the increase of the drops. (~=60Mbps). All the links are fiber (SFP) and the distance between the core switches and the enclosure is about 15-20m.
I am not aware of any service degradation on the part of the servers. No CRCs, collisions etc, on the interfaces, apart from the drops.
The line card is a WS-X6748-SFP, but other interfaces don't seem to be experiencing any problems.
We're having kind of a problem with our Catalyst 4507r switches. If we do a "show interface" command we're getting a lot of "Total output drops" on some of our interfaces. It seems to be most of the time on the same vlan.I was wondering if it has got something to do with QOS or queue selection As we don't have any QOS markings configured, is it possible that all traffic is using only one of the four tx queue's?
I am working on a QoS design which I hope to test at some point, but at this stage its from the books.My question is how to decide which queue and threshold to use for video traffic, then lower priority traffic.I understand the shaping and sharing commands, its the queuing and threshold bit I'm not clear on.The plan is to use the priority-queue for EF marked voice, this will be policed on ingress to provide an upper limit to EF traffic levels, then my second priority traffic will be video. Which queue will get serviced first once the priority queue is empty, and how do I decide which threshold to allocate my video traffic to? The document ion is not at all clear, I want to prioritse my traffic in the following order:
1 voice, use the priority queue 2 video, this to get serviced ahead of data, after voice. 3 interactive data 4 Bulk data 5 Best effort
So Q1 settings are ignored due to priority queue. Q2 gets 70%, Q3 25% etc.Is it as simple as putting video into Q2 T1, then interactive data into Q2 T2, will Q2T1 get a higher priority over Q2 T2 once the PQ is serviced?
I've been working on breaking down and understanding the default auto qos configuration on a Cisco 3750 in the hopes of putting together a QoS strategy that will fit our environment. I'm having some difficulty understanding how the "mls qos queue-set output" syntax works.
From another post, at [URL], the author offers the following example and explanation;
How come there is syntax stating "threshold 2" when in the succeeding part the 400 refers to thresshold 1 and threshold 2 again? The syntax 400 400 is, apparently, already referring to thresshold 1 and 2, no?
After opening up Solarwinds NPM, I noticed that a few of my interfaces had lots of discards (who knows how long it's been sets the counters were reset)
interface GigabitEthernet1/0/25description Etherchannel to MamaCassswitchport trunk encapsulation dot1qswitchport mode trunkswitchport nonegotiatepriority-queue outchannel-group 4 mode on
interface GigabitEthernet2/0/25description Etherchannel to MamaCassswitchport trunk encapsulation dot1qswitchport mode trunkswitchport nonegotiatepriority-queue outchannel-group 4 mode on
interface Port-channel4switchport trunk encapsulation dot1qswitchport mode trunkswitchport nonegotiate,It looks as if priority-queue was configured outbound on these interfaces, could this be the cause of the transmit discards which are now up to 79,835, I just reset the counters on the interfaces a little while ago.
I'm not the best in the world when it comes to QoS, we do have some VoIP phones, but they are only a specific network, and do not travel outside, since there are used mainly for VoIP training. I do know both interfaces are running the default of FIFO.
When configuring QoS on 3750s/3560s, we're mapping packets to particular interface output queues with commands such as: [code] The command to see what's actually being enqueued, dropped, etc. is: [code]
Note that these queues are numbered 0 - 3, and not 1 - 4. We've been assuming that the first queue number in the "mls qos" (i.e., 1) command maps to the first queue (i.e., 0) in the "show mls qos" command.
I am seeing Interface output drops that appera to be incorrect. When I do "Show Interface gi1/0/20", I will get interface output drops of "4294961382". But, when I do the same command again it shows "0" drops. Is this a reporting error? I am ruinning c3750-ipservicesk9-mz.122-58.SE.bin on a 3750 stack with 2 switches in the stack. [code]
We have the port on the switch set like this: switchport port-security maximum 25 switchport port-security switchport port-security aging time 2 switchport port-security violation restrict switchport port-security aging type inactivity
In case a device connected to the port is inactive for more than 2 minues ( aging time ) the first frame/packet the device generates arrives to the port on the switch, but the switch does not forward it to the appropriate port ( discards it or whatever ).
So far I tested on 1 30 WS-C3750E-24PD 15.0(2)SE2 C3750E-IPBASEK9-M 2 30 WS-C3750E-24PD 15.0(2)SE2 C3750E-IPBASEK9-M 3 52 WS-C3750G-48PS 15.0(2)SE2 C3750-IPBASEK9-M
When we remove port security from the port, it works perfectly fine, as expected.
It seems this is not HW or IOS version related. It seems it is not a stack synchronization issue, it does not matter if a device is connected to the first or other stack member. I tested on C3560 too, here there are no problems, so seems it is 3750 related.
If I have two stackable switches one switch uplinks to one 6509 core switch and the other connection uplinks to another 6509 core switch, and also the other stackable switch does not connect to the core switches. Because I am using hsrp and also we are not using vss then one uplink to the core is not in used only ones is so then how does creating an etherchannel between does two uplinks to both core switches benefit me in anyway such as more bandwith and using both uplinks at the same time or I'm I wrong?
If I have two stackable switches were only one stackable switch has two uplinks one uplink goes to one core 6509 switch and the other uplink goes to the other 6509 core switch can a Layer 3 etherchannel be used if each uplink go to a different core switch, by the way hsrp is running between both switches and also can you give an example how data will be routed from the stackable switch through the ethernetchannel to one of the core switch accross the WAN to another core switch?
I have a 3750X-24T in our production environment that is showing very high number of OQD's in the 'show int sum' output for 4 of the Gigabit interfaces; the interfaces are each in a seperate port channel and there are no OQD's for the relevant port channel and there are no output drops showing in the output for the 'sh int' command for each interface.
The following are the OQD's for the relenvant interfaces Gi1/1/1: 0 Gi1/1/2: 0 Gi1/1/3: 0 Gi1/1/4: 0 Gi2/1/1: 4252879251 Gi2/1/2: 4251090833 Gi2/1/3: 4251754140 Gi2/1/4: 4294942102 Po1: 0 Po2: 0 Po3: 0 P04: 0
Gi1/1/1 and Gi2/1/2 assigned to Po1, and so on. IOS version: C3750E-IPBASEK9-M 12.2(58)SE2
I am using a 3750 as a default gateway for multiple Vlans on a few 2960 switches. The trunk lines are configured and working and I have assigned ip addresses to each of the Vlan interfaces on the 3750. My issue is that I can only ping the ip address on the Vlan interface of the 3750 if I have a working computer plugged directly into the Vlan on the 3750. I only have 3 vlans on the 3750 that have hosts directly connected (vlans 2, 10 and 40) the other vlans ( 20 and 70) don't have any clients plugged into them on the 3750 but the hosts reside on 2 different 2960s that connect via trunk ports. How do I keep the vlan interface on the 3750 switch pingable when I don't have hosts directly connected in that vlan on the 3750? (yes, I have enabled ip routing on the 3750)
I have a simple design with 3750. I configured a route-map which define a next hop. I defined this route-map on a policy on a vlan interface.When I test some ping and a debug ip policy and it seems that my policy never match.Is there any mechanism that prevent the switch from using PBR? I think of CEF .
I have a simple design with 3750.I configured a route-map which define a next hop.I defined this route-map on a policy on a vlan interface.When I test some ping and a debug ip policy and it seems that my policy never match.Is there any mechanism that prevent the switch from using PBR?
Vlan interface would be dropping packets on the input queue? Refer to the drops/flushes below. This is from a 6500 with a Sup720, there are a number of vlans on it. This 6500 and it's HSRP partner are exhibiting the same symptoms on all the vlans I bothered to check. This particular vlan is quite lightly used, there are only about fifteen user PC's (each with 100 Mb interfaces) on it.
There is a bit of information on input queue drops on Cisco, but this is focused on physical interfaces where I can understand some packets being dropped. I would think that Vlan interfaces would have different issues.I note the "no buffer" errors as well, that also concerns me, especially as that counter is quite close to the "flushes".
Vlan123 is up, line protocol is up Hardware is EtherSVI, address is 00d0.04fd.6000 (bia 00d0.04fd.6000) Description: Vlan123 Internet address is 10.123.123.7/24 MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set Keepalive not supported ARP type: ARPA, ARP Timeout 04:00:00 [Code] .......
Switch A connects to Swich B and C using port channel. I am going to bring down one link on each.
switch A is server farm , switch B and C are core 01 and 02 . and all are 3750 switches.
1. what will be the impact on the network in terms of spanning tree recalculation 2. what duration are we talking about until the spanning tree convergence happens? 3.I plan to shut down the link from CLI to bring down the links
I have a small campus network using 3750 stackable switches and a 3725 router (see diagram below). Currently the 3725 router is handling inter-vlan routing for the campus and it looks like it's not able to handle the amount of traffic we're pushing. The router CPU sometimes hits above 90% due traffic load. What I would like to accomplish is move L3 process over to the 3750 MDF stack and the IDF1 stack. I am thinking creating SVI's on both MDF stack and IDF1 stack, run HSRP between the two stacks and may be do load balance traffic between the two stack as well.
Looking for feedback from other organizations that have large 3750 stacks. I've got one stack of (8) 3750's composed of (6) 3750G's and (2) plain 3750's. This particular stack is usually unresponsive to SNMP queries and often fails to write config when we make changes. After a couple tries it will finally go. Part of my probably here is likely the plain 3750's that always boot faster and come up as the master. I should manually set the master to one of the G's. What I'm wondering is who else has 7-9 3750's stacked and are they performing well for SNMP, telnet, etc? I've got another newer stack of 7 3750E's that I need to add one more switch to. Need to decide if I want extended downtime to break the stack up or just add the 3750X to make member 8 and hope it performs well. I have 50+ 3750 stacks working great on our campus.
I have a small network using a 24 port 3750 switch. I need to add five computers in another room and only have one Cat 6 cable running there and no room left on the 3750. I got a 3650 to put in the new room with the new computers. The problem is, whenever I plug the new switch into the 3750, it shuts down the port and gives me an err-disable. I can do a shut/no shut and re-enable the port. I searched the web and read about trunking and clusters. I'm not sure which, if either, is appropriate. I see various documentation that shows you can put one switch behind the other. But nothing tells me the configuration which will allow it.
If I have two stackable switches one that connects to the 6509 core switch and the other switch does not, do I have to uplink the switch to the other switch so that switch has route to the core switch or because the switches are daisy chain there is not need to cable one switch to to the other switch connected to core switch?
I am trying to do ios upgrade on 5 stacked 3750 switches. All the switches have different model number, so i am wondering which image file i should download. As far as i understand all the stacked switches should have the same IOS, i may be wrong. The switches have the following model numbers and SW images;
I have client who has two distant offices with 3750 L3 as core (do all vlan routing for local office) and multiple L2 access switches with multiple VLAN’s connected to it. First 3750 is hub also connected to internet, second 3750 is spoke and acting as a router on stick. We have eigrp configured on both side ISP provided client 100Mbps link as a trunk with two vlan; vlan10 for voice and vlan20 for data. We assigned two small subnets to these vlans 10.15.17.0/29 and 10.15.17.9/29. Hub addresses are 10.15.17.1 and 10.15.17.9 respectively. How to force voice over VLAN10 and data via vlan20, but still do some load balancing? How to setup default route on second (router on stick) switch?