Cisco Switching/Routing :: 3560X Crash And Reboot On IOS 15.0(2)SE
Jan 24, 2013
I have 3560X-24T-S switches with IP Services loaded at remote sites that I have been having troubles with. Originally, they had IOS 12.2(58)SE1 on them. I have up to four diverse paths via point-to-point microwave at the remote sites. The Microwave equipment is by Microwave Networks and is a Prodeus M series which Ethernet support. The original issue manifested itself as hardware loopback errors on some of the ports that were connected to the microwave links.
My experience in the past has been that when a hardware loopback error occurred it is usually a bad switch port. In this case however, if I disconnected all of the microwave Ethernet links and rebooted the switch and then connected the Ethernet connections to the microwave links again, everything works fine. No hardware loopback errors. That is, until the next switch reboot. Then the hardware loopback error would return. Interestingly, it would come back on different ports connected to the microwave links every time. So if a reboot was done without disconnecting the microwave Ethernet links the hardware loopback error would change from one microwave link to another after each reboot.
I then went through and read the lengthy release notes for IOS version 15.0(2)SE and found several fixes that I thought could fix my issue. So I downloaded it and updated a couple of the offending switches (not all of them were having this problem). After going through the second update required to resolve the 'open file error' that happens going between 12.2(58)SE1 and 15.0(2)SE the problem seemed to be resolved on the offending switches. So, I went ahead and updated the IOS on all of the switches with point-to-point microwave connections.
I now have one switch that was updated that is crashing and rebooting continuously when the Ethernet links for the point-to-point microwave are connected. Again, if I disconnect all the microwave links and reboot, it comes up fine ands stays fine when the microwave links are connected back up. It will work fine until the next reboot and then the crash and reboot loop starts over again. Below is a portion of the putty log when the crash occurs:
----------------------------------------------------------------------------------------------------------------------
previous memory block, bp = 0x59BF838,
memorypool type is Processor
data check, ptr = 0x59BF860
========= Dump bp = 0x59BF87C ======================
59BF77C: 0 0 0 FD0110DF AB1234CD FFFE0000 56 383FDB4
59BF79C: 212AC68 59BF838 59BF6F4 80000042 1 0 A504F53 543A2050
59BF7BC: 6F727441 53494320 506F7274 204C6F6F 70626163 6B205465 73747320 3A20456E
Our 6500 was reboot by self with crash info and i found it was happened by CPU HOG. The log is below for CPU HOG and you can see many times from attached crash info file.
%SYS-SP-3-CPUHOG: Task is running for (16000)m secs, more than (2000)m secs (15/12),process = RPC pagp_switch_mp2sp. [code]....
I think this process made high CPU & memory issue and then there were internal communication fail.. isn't it? So i need to know what PRC pagp_switch_mp2sp is to prevent next issue.
I have a WS-C3560X-24 and attached to that are some 9 acces switches, for some weeks now my 3560 reboots some time what couse that the other 9 switches are down for some minuts as well and i dont want this of course. the reboot happens at random times and some times one week not and then like yesterday afternoon it rebooted again.
when i check the Flash directory there is no crash file and when i look at the logging its clean and just shows the startup. it's not the powersuply it's redundend and more L3 switches are attached to this power source and they dont reboot.
L3_AIM#sh versionCisco IOS Software, C3560E Software (C3560E-UNIVERSALK9-M), Version 12.2(55)SE3, RELEASE SOFTWARE (fc1)Technical Support: [URL] Copyright (c) 1986-2011 by Cisco Systems,
My management has tasked me to give them a high level overview of the different switching we can choose for our new building.
This is what I know so far.4 Closets, each closet has 450 ports,One MDF room that is will contain one UCS Chassis and a Nimble iSCSI SAN.
I am working on the spreadsheet and it looks like this (Not totally filled):
2960s3560x3750x45064510Approx cost (Each, 48PORT, POE+, 10G uplink, Dual PS, IP BASE) 6K7K8K45K75KMax Capacity192432432192384Backplane speed206464520520ProLeast ExpensiveStackable to 9Stackable to 9ProDual PSDual PSDual PSDual PSDual PSProLayer 3 opt Layer 3 optDual SupsDual SupsConExpensiveExpensiveConNo Dual PSConLayer 2 OnlyCannot stack more than 4 For the MDF I would like to use 2 Nexus 5548's with FEX's, and the layer 3 daughter board. For the IDF's I was thinking of two 4010's.
i cant find any difference in these two devices when i am trying to compare throughput.I need upgrade our new POP and there will be around 4900 MAC adresses in VLAN 150 and 130 MAC adresses in vlan 200.Uplink is 1 gig routed internet connection and there is 14 downlinks to separate villages.i found a few differences for eg stack interface on 3750x but i dont need it.
I just upgraded several of my C2960S-48FPD-L from 12.2(55)SE to 12.2(58)SE and now almost every time I try to SSH to it the switch reboots and gives me these trace backs ...
07:58:52 CST Thu Apr 14 2011: Unexpected exception to CPU vector 200, PC = 1EADA28 -Trace back= 0x1EADA28z 0x1CDC324z 0x1EA5520z 0x1CDC35Cz 0x1CDD0E0z 0x1CDF0C4z 0x1CE06CCz 0x1CE07F8z 0x1D67348z 0x1D611BCz
Writing crash info to flash:/crashinfo_ext/crashinfo_ext_3
I unpacked a new 3560x and went to put our standard code version on it c3560e-ipbasek9-mz.122-53.SE2. Everything seemed to upgraded fine but it won't boot to a console prompt. It seems like it loads the IOS image fine but can't get passed the Front-end Microcode IMG MGR: Programming device 0. [code]
We have a small site that has an Avaya voice switch connected to a c3560x switch.The avaya tech told us to set this on our ports to which the phone and the desktop are connected.
What this crash means? This is a 6509 with a single Sup720 Cisco IOS Software, s72033_rp Software (s72033_rp-ADVIPSERVICESK9_WAN-M), Version 12.2(33)SXI, RELEASE SOFTWARE (fc2)System returned to ROM by s/w reset at 04:42:07 PST8PDT Mon Jan 2 2012 (SP by bus error at PC 0x40C6681C, address 0x424B).
We had an "Event" on our process network at the mill yesterday. I connected a new WS-C3560V2-48TS-S to our network and we lost communications with all of the other switches.
The core is a stack of 3 WS-C3750 switches, one is a -24TS-1U and the other two are -12S-S. These connect to 10 WS-C3560G-24TS and 5 Rockwell Stratix 8000 (IE3000) over fiber. I am planning on replaceing two 24 port switches with the new 48 port. I had the switch configured and running at my work bench. It was connected to the network with one sfp module and ran all weekend with no issues. Yesterday afternoon I took it to the network cabinet and installed it. I powered it up and connected two sfp modules to the fiber patch panel and made the connection at the core stack.
Everything looked OK. I had communication link lights working on everything. Within minutes, we lost communication with every switch connected to the core stack. I shut down the new 48 port switch and the network slowly came back up.
The new 48 port is configured with Flex Links for the fiber redundancy protocol and was connected to each of the 3750G-12S-S stack members.
I though it might have been a power issue but the stack is UPS protected and shows it has been up for over 10 weeks. I'm not even sure "Crash" is the best decription for what happened. The new switch has a high enough IP address that it would not take over as a IGMP Querier.
I have seen a similar post here from last year about a 10/half connection, but this is different. I have a provider using a 2950 switch (they left CDP on so we can see it). And we have to be set to 10/full to make this connection (as you know 10/half connected here would be or is horrible). I have had this connection running on my 2921 router w/o issue but when I connect it to my 3560X, then I get no link light (shows down/down, not connected).
What else to know about what has been tried: Connecting link to switch with switch set to auto/auto (sp/dup). Speed is 10 (as it should be) but of course w/o the other side pulsing the voltage to know what to auto to, it defaults to half so this is unacceptabel, but, I do get a link light and can pass traffic, it's just a horrible experience.
Also tried the same setting speed and duplex vs. auto disco speed and duplex on other ports of the switch, no difference, same results. Also different patch cables, same results.
My current work around is to connect the providers 2950 (10/full) to my 2921 router @ (10/full) with a BVI to another port on the router which continues on to my 3560X at 1G/full. And this works just fine. Here's more info from my 3560X and the providers 2950:
If the 3560 or 3750 "X" series support GRE.I am pretty certain the older 3750-E does not support GRE (both in hardware and software)Was hoping the new super duper X series do. If not, it could get expensive
I have a 3560X switch with interfaces 36-48 on the same LAN. All interfaces are switchports. Hosts on 38, 39 and 40 are multicast senders: all sending to the same single multicast address. Hosts on 36 and 37 are receivers, having joined that multicast group. I created an SVI for the LAN and put it in ip pim passive. (That is the only PIM mode allowed for an SVI with my IOS.) Show ip igmp snooping groups shows that 36 and 37 are the only interfaces in this group. I attach a laptop to interface 42 and Wireshark, and the laptop is receiving the multicast traffic. The laptop does not join the group. I expect it would not see the traffic.
last week we have changed the connection of small town from licensed radio to direct fiber with clasic 1gig sfp.After that we have connected the new fiber , we can see 5 min outages of customer internet connection. If we are conncted on the same subnet anywhere on network, we can ping the customer all the time. But if we try to ping the customers public IP it is each 5 min for 5 min unreachable. I have tryed to sho logg in all switches but there are no mesasages. I just see, when comunication have lost, the MAC and IP of customer will disapear from routers ARP tabbel. What is going on?When i enter sho mac address-table i can see customers MAC address in all switches. There are exactly 5 min outages. 5min it runs ok, 5 min outage and it repeats agan.
I looks like the swith blocks the ARP packets from customers to their default gateway.The routers are connected to catalyst 3560x lan base ios, This switch is connected to another 3560x lan base ios and it is connected with fiber to ME3400 where are customers connected.
It would be so easy but failed to upgrade 3560X switch to ip services licence is bought and paid for mail and log from the CLI and tftp server attached
Im going to be ordering a new 3560X L3 switch (ws-3560x-48pf-s). We currently have a 3560 V2 switch and I've read it supports stacking, but looking at the back I just see a console port. Is there an extra card I have to get for this 3560V2 to make it stakable with the 3560X?
I was viewing the 3560x switch datasheet and i need network module C3KX-NM-10G. I have dark fiber with over 12 km distance to i guess i will be needing SFP EX for single mode.
I currently use L3 switches as edge routers to my WAN. I want to use a pair of 3560x switches with IPbase to provide a failover path to my WAN using HSRP at one location but had some problems testing the configuration. My plan is use a virtual address on the LAN interface (VLANx which port gi0/1 accesses) and the WAN interface (VLANy which port gi0/24 accesses). I want switch 1 to be primary since it will have an IPS attached to it, and switch 2 will be backup and used only when switch 1 or the IPS requires maintenance. On both the LAN and WAN sides there is no advanced routing going on, the various hosts just depend on the availability of their respective default gateways, so HSRP should be sufficient to provide a failover in either direction.
In my testing I got 1 or the other link to fail over but not the entire switch. What should my config look like to achieve failover of the entire switch in the event 1 or the other interface goes down, and fail back when the primary links are again available?
I am stuck at a very early stage in the configuration of 3560x switch. It seemed very unusual and that is the reason, I configured basic config for line con 0 with login local. And then I had just put "enable secret xxxxxx."
Now I simply logged out and when i tried logging in, it is asking for a username. Now the thing is that I had not programmed any username. I tried using just "." followed by my secret password, but it did not work.
we just got brand new 3560X with LANBASE license installed. Wanted to confirm with a "show license"
To my surprice (output below) I hav IP Services evaluation. I tried to test with an "show ip route" and "ip route..." in configuration mode with no success. How do I use the trial license?
Switch#show license Index 1 Feature: ipservices Period left: 8 weeks 4 days License Type: Evaluation License State: Active, Not in Use, EULA not accepted License Priority: None [Code]...
if a connect to the switch via Internet Explorer or every other browser I can't enter configuration because I receive a screen as you can view in image below. And I can view those errors:
message: 're_preFlt_txt33' is not defined Linea: 31 Carattere: 1 Codice: 0 URI:
[URL]
If I connect in HTTPS with IE8 in compatibility mode I view the image as in attachment with_https.jpg.
why Cisco make 3560x switches with Lan Base(no routing functions)? What happen if I load an Image L3, previously downloaded? my device lost the warranty? or it doesn matter?
I have a cisco 2821 router in rommon and displaying the message '' softwre forced crash '' and '' checksum error'' .I tried to do rommon tftpdnld but as the image is self decompressing into the ram it again crashes with the same error although i have done it with various valid ios but in vain.
The switch 6500-E is frequently crashing whenever the pm scp process reach 100% .I got it under the command "remote command switch show proce cpu".How to solve the problem. I opened the TAC and the engineer says to upgrade the image12.2(18) sxf.Is there any other work around that would avoid to reimage the switch and reloading.
I got some unexpected System crash, and this happens to 2 different routers on the same network.. (We suffer a Crash and change the router, happens the same to the new router after some time, maybe 40 minutes!!)
Here is the last console report for the new router....
%Software-forced reload
Preparing to dump core... 4w1d: %SYS-2-WATCHDOG: Process aborted on watchdog timeout, process = IP NAT Age
About an hour ago I had the master switch on one of my 3750x (WS-C3750X-48PF-S) stacks crash. The only two items we've found that could have caused this issue are the roughly 1.3 million big buffer misses and several of the following in the syslog
SLT:WARN:No exporter configured for smartlog! I do not have smartlogging turned on, nor is there a netflow exporter configured
sh logging smartlogsmartlog is disabledsmartlog exporter:smartlog pkt length: 64 Total pkts processed: 0Total DHCP Snooping pkts processed: 0Total DAI pkts processed: 0Total IPSG pkts processed: 0Total ACL pkts processed: 0
I did not see any traffic spikes prior to the crash.
This stack has been stable since it's last IOS upgrade from 12.2(58)SE1 to 12.2(58)SE2 back in October 2011 so this has me a little worried.
I have a Cisco 3560X series 48 port PoE switch which im using to try to power a Axis Q6032E PoE+ PTZ camera that requires 25W of power. Although when I plug the camera into the switch it power as PoE Class 4 but will only assign the port 15.4W of power. I have tried going into the switch and manaually assigning the port to 30W of power but no luck. I have talked with the Axis vendor and they stated that Cisco does not turn on 802.3at protocol by default? This does not seem correct. I am running a single 1100W power supply as well. See output below. Gi0/43 is port in question.
Gi0/41 auto on 15.4 Ieee PD 4 30.0 Gi0/42 auto off 0.0 n/a n/a 30.0 Gi0/43 auto on 15.4 Ieee PD 4 30.0
I have a brand new 3650x switch that I was configuring via console and left it on overnight; this was friday night ( I forgot to keep the air conditioner on). When I returned Sunday afternoon and checked the switch I realized that the SYS and STAT lights on the front were not on, however the Leds on the PS module were still on. I powered it down and turn it on again but still there are no lights on the front. The PS OK and AC OK leds on the PS module are on but I noticed the leds on the two fan modules are not green or even amber; they are off.When I first turned on the switch I remembered distinctly that there were loud noice and if all fans were in overdrive mode for a few seconds then the went to normal. I'm not gettgin this loud noise phenomenom.
Is the CAB-C15-CBN - Cabinet Jumper Power Cord, 250 VAC 13A, C14-C15 Connector, supported on the Cisco Catalyst 3750X/3560X. I need to plug the switches into a UPS PDU, that requires a C14 plug.
Is it possible to to build a Layer 3 ether channel from two separate physical switches (layer3) that are trunked together?I know you can easily do this on a single switch and on stacked switches which I've done but in this case the customer have purchased two 3560X's which are not stackable yet want redundancy. The purpose of the etherchannel is to connect both switches to a private circuit provided by the hosted partner then route to the same setup in the DR location to different subnets.
I am trying to configure EIGRP on my ASA DMZ Interface - topology as follows: [code] The ASA is currently configured for EIGRP with the inside 3560x switch and passing routing updates properly.However, the ASA will not send/receive routing updates to/from the DMZ 3560x switch - the two devices do establish eigrp neighbor relationship. [code]
