Cisco Switching/Routing :: System Crash 2611 And 3640
Apr 8, 2013
I got some unexpected System crash, and this happens to 2 different routers on the same network.. (We suffer a Crash and change the router, happens the same to the new router after some time, maybe 40 minutes!!)
Here is the last console report for the new router....
%Software-forced reload
Preparing to dump core...
4w1d: %SYS-2-WATCHDOG: Process aborted on watchdog timeout, process = IP NAT Age
I'm using 3640 router running on c3640-js-mz.124-25d.bin IOS. I'm using NM-1A-OC3SML= (ATM OC3, long reach single mode) interface card. Now my pc is connected an fast ethernet interface of the router. Need to know the ATM configuration on this cisco 3640 router in order to achieve the ATM over ethernet traffic get success. As of now i've confugred as below but it's not working it seems.
interface ATM2/0 bandwidth 120000 ip address 10.2.2.1 255.255.255.0 no atm ilmi-keepalive pvc 0/36 protocol ip 10.2.2.10 broadcast cbr 70000 encapsulation aal5mux ip
let me know the correct encapsulation type for the connectivity.
The Cisco 3600 series (3640) is hanged during booting proccess, the IOS is decompressed, but after some text it hanges and I even cannot go to user exec. ROMmon and bypassing the config setting, but the result is still the same.
I noticed that my router 7200 was reboot for a bus error. The error occurred while the system was operational. After the restart the router works properly for more than 6 days.
This is the sh ver:
Cisco IOS Software, 7200 Software (C7200P-IPBASEK9-M), Version 15.0(1)M4, RELEASE SOFTWARE (fc1) Technical Support: [URL] Copyright (c) 1986-2010 by Cisco Systems, Inc. Compiled Thu 28-Oct-10 22:57 by prod_rel_team
[code].....
Seems that the error "System returned to ROM by bus error at PC 0x2E0BF30, address 0x0" should be a special type of bus error crash is when the crash is caused by a corrupted program counter (PC). The PC value is the location of the instruction which the processor was executing when the bus error occured. In this case, the PC has jumped to the address 0x0 (probably because of a null pointer), but this is not where the instruction is located.
Can I consider this as a processor memory parity errors and wait for a second parity error before I replace anything?
I am with Virgin media and have a home lab (2611 routers, 3550 switches, 2950s etc) which i usually just use for testing things on and practicing.
Now I tried to connect it to the internet and got some weird things happening.
I got NAT and DHCP client all setup correctly on the router (or so I think!) along with DNS settings but I still can't get Internet access.
My network is all setup right so that through using OSPF the router is the gateway (0.0.0.0 route) for all devices. Doing a traceroute confirms this.
Yet I never get any further than the router? I see IP nat translations ticking up as I send more requests but never get a webpage or a reply from any of the machines in my network.
HOWEVER... The router itself does get some replies from Google.co.uk and bbc.co.uk but doesn't seem to want to give anything else on the "inside" a response?
Here is a dump of config:
version 12.4 service timestamps debug datetime msec service timestamps log datetime msec
I got this 3640, trying to apply a service-policy (output and input), but seems like I do it something wrong...because he only apply the output policy... here the config, I already try to config the service police inside the fa0/0, but is not showed at all, he only show the output, its like I never apply that
We have a Cisco 3640 router running c3640-is-mz.123-3g.bin Switching ports are devided into several VLans. Each VLan has its own IP subnet. We can't ping IP address X in subnet A from subnet B unless we log into the router and ping it from there first. (and then the IP address will show up in "show ip arp" command. Then we can ping X in subnet A from subnet B, and browse web on device X from subnet B, etc. )
i am running c3640-is-mz.124-21.bin on a cisco router 3640. i am trying to create a monitor session in the CLI and everytime i type the command Router(config)#monitor session 1 interface ethernet2/1 % Invalid input detected at '^' marker. Router(config)#monitor session 1 interface ethernet2/1 ^% Invalid input detected at '^' marker. i get the error invalid input ?
So I have 2 routers (cisco 3640) that each go to their own ISP and then back to the same switch. I have setup ospf and glbp, and now have pretty good redundancy. If either internet connection or routers go down everything is still golden.
So I was thinking that if an interface went down then the router would not be load balanced with glbp which got me thinking whats the best way to get interface redundancy (and I was going to add a 2nd switch with the second interface).
1) Setup BVI on the 2 interfaces. 2) Setup a 2nd interfaces (on each router), I would have to split the subnet, for instance: [code]then the machines could be on the subnet 192.168.0. 0/23 and setup glbp for 1 ip across all 4 interfaces (I'm not even sure if you can do this but think it would work). 3) Is there a way to utilize etherchannel or anything like this
A negative to option 2 would be that if 1 of the interfaces went down, all the sudden 2/3 (or so) of your traffic would be going through 1 router.
How to configure my serial modules on my 2600 and 3640? Each router has the WIC-2T with two cables connecting them. I tried to ping one router from the other, but no luck. Serial IP address I just made up, this is my home lab, so if I messed those up some how,
Here is the running config for each rotuer:
2610: Current configuration : 1071 bytes ! version 12.3 [Code]...
I have two routers at our core data center, a 3845 and a 3640. These are configured with GLBP. There are 4 remote sites:
Site #1: One T1 link to the 3825 Site #2: One T1 link to the 3825, and One T1 link to the 3640 Site #3: One T1 link to the 3825, and One T1 link to the 3640 Site #4: One fractional T1 link to the 3825, and One T1 link to the 3640.
My question regards site #4. If i understand correctly, GLBP works on the premise of "host" balancing, and not true "load" balancing. The reason I ask is that the large majority of our WAN traffic is from our Exchange server to our remote sites. In the case of site #4, our exchange server is sending traffic on the fractional T1. Is there any way with GLBP to either split this traffic from a particular host across two links in a round-robin fashion, but leave other hosts to travel wherever the router sends them, or, to force at least our exchange server to use the full T1, rather than the fractional?
I've read up on the weighting mechanism, and it appears that tracking an interface has nothing to do with bandwidth use. If I understand correctly, if I were to track the Site #4 PPP to the 3640, and give a weight of 10 to glbp on there, it would really only take affect if the interface is down. It will have nothing to do with host AVF election.For the record, exchange traffic is constant to this site, so there is no chance for the host connection to reset and potentially elect to use the larger pipe. I would like to "tweak" this to make better use of available bandwidth.
I had been using my Linksys wireless-G PCI adapter WMP54Gv4 in a PC running XP (SP3) and an Airport Extreme wireless router for the past 4 or 5 years without any issues. Then recently, my wireless signal had become flaky on all client machines (on both PC and various Macs, which reported weak signal and kept dropping and searching for a new PPPoE from my DSL line). AT&T verified our line was fine and even sent a new modem but troubles persisted, so I bought a new Airport Extreme last week. This solved the network reliability issue on the Macs just fine.
However, the PC with the WMP54Gv4 adapter now crashes whenever the new Airport Extreme is turned on. There have been no hardware or software changes on the PC, yet it now reliably produces a BSOD stating "DRIVER_IRQL_NOT_LESS_OR_EQUAL". This crash also happens when booting in safe mode with networking. I can avoid the system crash when booting in safe mode without networking or booting normally without my Airport Extreme turned on, though I do see this error: "Linksys Wireless Network Monitor: Access violation at address 0040756D in module EMP54Gv4.exe"
So I cannot run any tests of the PCI card on the machine when the wireless network is on, since the operating system crashes.
The evidence thus points to an explanation where my wireless network from the new Airport Extreme is tripping up the operation of the linksys wireless card in a way that XP hates. I know the wireless network is OK, since it works fine on my Macs. I confirmed that my driver (v3.0.3.0) is still listed as current for this PCI card, so it's not a matter of having an out-of-date driver.
Here's another observation that may be relevant: When I first started using the new Airport Extreme with WPA2 protection, the PC could run without crashing, but it could not connect to the wireless network. I also had trouble connecting via my Macs. Then I switched the Airport Extreme to use WPA/WPA2 security, which is when the network-based PC crashing ensued (though the Macs can now connect just fine).
I just upgraded several of my C2960S-48FPD-L from 12.2(55)SE to 12.2(58)SE and now almost every time I try to SSH to it the switch reboots and gives me these trace backs ...
07:58:52 CST Thu Apr 14 2011: Unexpected exception to CPU vector 200, PC = 1EADA28 -Trace back= 0x1EADA28z 0x1CDC324z 0x1EA5520z 0x1CDC35Cz 0x1CDD0E0z 0x1CDF0C4z 0x1CE06CCz 0x1CE07F8z 0x1D67348z 0x1D611BCz
Writing crash info to flash:/crashinfo_ext/crashinfo_ext_3
I have 1 2611xm router and 1 2801 router. For my own lab purpose, i want to configure them back to back to support voice services. I don't know what configuration will be required at each end. in 2611xm, i have NM-2V and its also detecting the card, so i hope it will work ? also what commands i need to run on both ends .
What this crash means? This is a 6509 with a single Sup720 Cisco IOS Software, s72033_rp Software (s72033_rp-ADVIPSERVICESK9_WAN-M), Version 12.2(33)SXI, RELEASE SOFTWARE (fc2)System returned to ROM by s/w reset at 04:42:07 PST8PDT Mon Jan 2 2012 (SP by bus error at PC 0x40C6681C, address 0x424B).
I have 3560X-24T-S switches with IP Services loaded at remote sites that I have been having troubles with. Originally, they had IOS 12.2(58)SE1 on them. I have up to four diverse paths via point-to-point microwave at the remote sites. The Microwave equipment is by Microwave Networks and is a Prodeus M series which Ethernet support. The original issue manifested itself as hardware loopback errors on some of the ports that were connected to the microwave links.
My experience in the past has been that when a hardware loopback error occurred it is usually a bad switch port. In this case however, if I disconnected all of the microwave Ethernet links and rebooted the switch and then connected the Ethernet connections to the microwave links again, everything works fine. No hardware loopback errors. That is, until the next switch reboot. Then the hardware loopback error would return. Interestingly, it would come back on different ports connected to the microwave links every time. So if a reboot was done without disconnecting the microwave Ethernet links the hardware loopback error would change from one microwave link to another after each reboot.
I then went through and read the lengthy release notes for IOS version 15.0(2)SE and found several fixes that I thought could fix my issue. So I downloaded it and updated a couple of the offending switches (not all of them were having this problem). After going through the second update required to resolve the 'open file error' that happens going between 12.2(58)SE1 and 15.0(2)SE the problem seemed to be resolved on the offending switches. So, I went ahead and updated the IOS on all of the switches with point-to-point microwave connections.
I now have one switch that was updated that is crashing and rebooting continuously when the Ethernet links for the point-to-point microwave are connected. Again, if I disconnect all the microwave links and reboot, it comes up fine ands stays fine when the microwave links are connected back up. It will work fine until the next reboot and then the crash and reboot loop starts over again. Below is a portion of the putty log when the crash occurs:
---------------------------------------------------------------------------------------------------------------------- previous memory block, bp = 0x59BF838, memorypool type is Processor data check, ptr = 0x59BF860 ========= Dump bp = 0x59BF87C ====================== 59BF77C: 0 0 0 FD0110DF AB1234CD FFFE0000 56 383FDB4 59BF79C: 212AC68 59BF838 59BF6F4 80000042 1 0 A504F53 543A2050 59BF7BC: 6F727441 53494320 506F7274 204C6F6F 70626163 6B205465 73747320 3A20456E
We had an "Event" on our process network at the mill yesterday. I connected a new WS-C3560V2-48TS-S to our network and we lost communications with all of the other switches.
The core is a stack of 3 WS-C3750 switches, one is a -24TS-1U and the other two are -12S-S. These connect to 10 WS-C3560G-24TS and 5 Rockwell Stratix 8000 (IE3000) over fiber. I am planning on replaceing two 24 port switches with the new 48 port. I had the switch configured and running at my work bench. It was connected to the network with one sfp module and ran all weekend with no issues. Yesterday afternoon I took it to the network cabinet and installed it. I powered it up and connected two sfp modules to the fiber patch panel and made the connection at the core stack.
Everything looked OK. I had communication link lights working on everything. Within minutes, we lost communication with every switch connected to the core stack. I shut down the new 48 port switch and the network slowly came back up.
The new 48 port is configured with Flex Links for the fiber redundancy protocol and was connected to each of the 3750G-12S-S stack members.
I though it might have been a power issue but the stack is UPS protected and shows it has been up for over 10 weeks. I'm not even sure "Crash" is the best decription for what happened. The new switch has a high enough IP address that it would not take over as a IGMP Querier.
I have a cisco 2821 router in rommon and displaying the message '' softwre forced crash '' and '' checksum error'' .I tried to do rommon tftpdnld but as the image is self decompressing into the ram it again crashes with the same error although i have done it with various valid ios but in vain.
The switch 6500-E is frequently crashing whenever the pm scp process reach 100% .I got it under the command "remote command switch show proce cpu".How to solve the problem. I opened the TAC and the engineer says to upgrade the image12.2(18) sxf.Is there any other work around that would avoid to reimage the switch and reloading.
About an hour ago I had the master switch on one of my 3750x (WS-C3750X-48PF-S) stacks crash. The only two items we've found that could have caused this issue are the roughly 1.3 million big buffer misses and several of the following in the syslog
SLT:WARN:No exporter configured for smartlog! I do not have smartlogging turned on, nor is there a netflow exporter configured
sh logging smartlogsmartlog is disabledsmartlog exporter:smartlog pkt length: 64 Total pkts processed: 0Total DHCP Snooping pkts processed: 0Total DAI pkts processed: 0Total IPSG pkts processed: 0Total ACL pkts processed: 0
I did not see any traffic spikes prior to the crash.
This stack has been stable since it's last IOS upgrade from 12.2(58)SE1 to 12.2(58)SE2 back in October 2011 so this has me a little worried.
Our 6500 was reboot by self with crash info and i found it was happened by CPU HOG. The log is below for CPU HOG and you can see many times from attached crash info file.
%SYS-SP-3-CPUHOG: Task is running for (16000)m secs, more than (2000)m secs (15/12),process = RPC pagp_switch_mp2sp. [code]....
I think this process made high CPU & memory issue and then there were internal communication fail.. isn't it? So i need to know what PRC pagp_switch_mp2sp is to prevent next issue.
I am not able to find the exact bug for Cisco 3750E stack - Debug Exception (Could be NULL pointer dereference) Exception (0x2000) error. Closest i can find is CSCsa72400 which only affects ver 12.2(20)SE4.All the stacks (3 switches) are running 12.2(50)SE3, It appears that the switch 1 crashed and reloaded. My hunch is its software but i cant find any related bugs. It could be hardware issue as well ?
Problem I have encountered when upgrading a Cisco 6509 chassis with a new supervisor card from Sup2 to Sup 720b. The 6509 loads and then crashes completely, and when rebooted reloads into rommon.The same upgrade was performed on a similar switch with no apparent problems.
Got a problem with my 1760 router. Bought it from ebay and booted it on today and got this error,It has 180224K/16384K bytes of memory and 2 paritions of 32768K flash.I erased both partitions and put a different version of the IOS on (still 12.4) and there is no difference, still get the errors.These aren't on any of my other 1760 routers so I assume they are linked to the problem.
Our switch had a little crash-fest this morning at 2:30 AM. I did find a web page about diagnosing Software Forced Crash Exceptions, but it did not look like ours was one of the more easily-identifiable ones.
It may be worth noting that we've only used this switch for about a month, everything seemed fine until now. When we got the switch it did not have any GigE modules, and this week we put 2 into it and have been using them for 2 servers.
It looks like the switch was crashing repeatedly over a period of 20 minutes, and then it stopped and normalized. In the logs of the router that this switch uplinks into, we could see the ethernet port flapping during the time that the switch wasn't reachable.
Here's the Show Stack on the switch:
Sfld_3550# show stackMinimum process stacks:Free/Size Name4404/6000 vegas_flash init3352/6000 SaveCrashBuffer5716/6000 CDP BLOB8512/9000 IP Background5596/6000 vqpc_shim_create_addr_tbl5584/6000 SPAN Subsystem5552/6000 SASL MAIN4944/6000 vegas IPC process8704/9000 cdp init process5404/6000 RADIUS INITCONFIG4928/6000 Vegas CrashBuffer5664/6000 URPF stats2536/3000 Rom Random Update
I have a WS-C3560X-24 and attached to that are some 9 acces switches, for some weeks now my 3560 reboots some time what couse that the other 9 switches are down for some minuts as well and i dont want this of course. the reboot happens at random times and some times one week not and then like yesterday afternoon it rebooted again.
when i check the Flash directory there is no crash file and when i look at the logging its clean and just shows the startup. it's not the powersuply it's redundend and more L3 switches are attached to this power source and they dont reboot.
L3_AIM#sh versionCisco IOS Software, C3560E Software (C3560E-UNIVERSALK9-M), Version 12.2(55)SE3, RELEASE SOFTWARE (fc1)Technical Support: [URL] Copyright (c) 1986-2011 by Cisco Systems,
I have one Catalyst 6509E chassis and two SUP720. The bootup sequence on SUP 720 (standby hot) failed . Messages that appear on SUP 720, on the console port indicate o software crash. I don't have a flash card in SUP720.
This is the bootup process:
System Bootstrap, Version 8.5(3) Copyright (c) 1994-2008 by cisco Systems, Inc. Cat6k-Sup720/SP processor with 1048576 Kbytes of main memory
I have two routers I am trying to connect via the WIC-2T port. I can ping from router to router, but not from my PC (192.168.2.122) to the 3745 (10.0.1.3)..
3640: Current configuration : 1846 bytes ! version 12.2
I would like to know if Catalyst WS-C3750G-48TS-E recognizes and understand Cisco VSS ( Virtual Switching System) . Is there a List available which tells us which Old Catalyst Switches or current switches understand Cisco VSS?
I have many WiSM WLC's running 7.0.116.0. One WLC was rebooted few days ago but there was no crash file and nothing in logs say why this issue happened.There was a power problem at the same time the WLC rebooted (some switches and PE's was rebooted as well) but if it is a power issue why only one WLC inside the WiSM rebooted and the other WLC is still working fine with no reboot?I have 5 WiSM modules connected to the same 6500 box, only one WLC was rebooted which indicates a crash but no crash file registered for it.Is there anyway I can find the reason why that WLC was rebooted?