Cisco WAN :: TAC 900 - Crash Because Of Large Packet
Dec 8, 2010
We are working on taking a port off of a switch and connecting it to a mux (TAC 900). Port is trunked to another switch in another geographic location. From wht I am being told from the engineer who handles the transmission end of things he said once our segment is physically connected the mux will crash. He supplied the error code to the vendor and they said it is crashing because of a jumbo packet in excess of 1536 bytes. Soon it will be a moot point as the hardware will be, upgraded but we need tis running NOW.
Today, during a packet_trace to debug a routing problem, the active ASA
- thsasaprd02 - crashed suddenly.
I was able to copy-paste the console - including the command that triggered it - After the reboot I ran the command again, on the same ASA - after doing a manual failover - the command succeeded normally.
I have been searching the message boards and wasn't having much luck. I am running some monitoring sessions on my 6509 and on the VLAN I am monitoring, I am experiencing a really large packet loss. If we hook up a laptop to the destination port and run wireshark we are seeing between 80% and 90% packet loss. I dont see the packet loss on the show port command, but I do on the show int vlan command.
The config is as follows:
Session 2 --------- Type : Local Session Source VLANs : RX Only : 500
I was doing some reading on Egress vs Ingress and I am wondering if the Egress SPAN replication state could be causing the packet loss that we are seeing or does the ingress & learn command override that?
I have many WiSM WLC's running 18.104.22.168. One WLC was rebooted few days ago but there was no crash file and nothing in logs say why this issue happened.There was a power problem at the same time the WLC rebooted (some switches and PE's was rebooted as well) but if it is a power issue why only one WLC inside the WiSM rebooted and the other WLC is still working fine with no reboot?I have 5 WiSM modules connected to the same 6500 box, only one WLC was rebooted which indicates a crash but no crash file registered for it.Is there anyway I can find the reason why that WLC was rebooted?
I have two 7204VXR with NPE-G2 and 1Gb of ram. One router has 2 eBGP peers and the other has 3. The routers receive all internet routes from the 5 peers and send 2 internal routes. There is an iBGP peering between both routers. On all peers I have a route-map to send only our routes.
All was working fine since a couple of months when I suddenly saw an increase of memory on one of the router (router B), 1 hour later the memory was 100% and router crash and reboot. The other router (router A) with the same hardware capacity, same ram and same amount of routes was working good. After router B restart, I shut all eBGP peering on it, keeping only iBGP with router A, ram used was the same as router A (about 50% used) but CPU was about 30% used by process Router BGP whereas router A which has active traffic and active eBGP is only 20% and bgp process i almost 0%. Restarting peers one by one on router B cause the same issue, increase of memory then crash, even with only one peer.
What I suspected :
- A peer on router B but I can't isolate one because the problem appears with each taken one by one
- Not enough memory, but router A has the same number of routes and don't have any problem
- IOS version ? same on both 12.4.(15)T1
- Why process router BGP use 30% on router B when all eBGP peers are shut except iBGP and no traffic pass.
- A routing loop but I only send internal routes to peers and only have one iBGP session with no sync nor redistribution with an IGP
Of course I can't run any debu ip bgp on routers as the number of routes is very large (300K).
Just updated the fw to 22.214.171.124 and after a while, the device crashed wth the following message in the flash log.
21474836472012-Jun-17 11:46:21Emergency %SYSLOG-F-OSFATAL: FATAL ERROR: GOAH: ABORT DATA exception ***** FATAL ERROR ***** SW Version : 126.96.36.199 Version Date: 11-Jun-2012 Version Tim e: 17:35:31 Instruction 0x150348 Exception vector 0x10 Program state register 0x60000013 0x0015014c ***** END OF FATAL ERROR ***** Haven't seen the switch crashing before.
I have a rare case. switch 6500 make crash and go to rommon. when review the crash appear that the switch charge the configuration from nvram, but, at the ending there are a few line command, that make the switch go to crash.. then I have to booting from rommon and start again. I use the version 12.2.33.sxi9 and 12.2.sxi3 and the configregister is 0x2102
I think that the switch get the startup-config file from other file-system. *Sep 1 03:42:42.352 Inviern: %SYS-5-CONFIG_I: Configured from memory by console access-list 199 permit icmp host 10.10.10.10 host 188.8.131.52 crypto map NiStTeSt1 10 ipsec-manual match address 199 set peer 184.108.40.206 exit no access-list 199 no crypto map NiStTeSt1
*Sep 1 03:42:46.952 Inviern: %SYS-5-RESTART: System restarted.
Built in wifi causing blue screen of death. Disabled built in wifi and purchased USB N150 adaptor and blue screen of death has stopped. Download speed are 75% back to normal but surfing is extremely slow using chrome, IE, safari. Ethernet browsing is normal with no bugs or bluescreen.
I am in charge of taking care our small office network (about 25 PC's) and encountered a problem the other day. I got a call that nothing network related was working. I went to look at the switches/router and everything appeared fine. After some troubleshooting i found if i unplugged the switch and connected directly the the router everything worked prefect. Reconnected the switch, network goes down again. I assumed it was the switch at first but then did a walk around and a laptop was in a weird frozen state. I unhooked the network cable and everything started working away like it should.
Loaned old laptop when hard-drive went out of my own. I knew it would need some maintenance ie defrags, cleanups, updates etc. and have been trying to do so... in the process I am having a very difficult time as all of my browsers (that I thought were updated) are slow, hanging, crashing and causing my entire pc to freeze or hang.
I was uploading a big 2gb file to my FTP server, it was all going good and I managed to upload the whole thing hassle free. After I finish I log out, and return about 3 hours later and try to log back in, to no avail.WinSCP gives me the following error:
What this crash means? This is a 6509 with a single Sup720 Cisco IOS Software, s72033_rp Software (s72033_rp-ADVIPSERVICESK9_WAN-M), Version 12.2(33)SXI, RELEASE SOFTWARE (fc2)System returned to ROM by s/w reset at 04:42:07 PST8PDT Mon Jan 2 2012 (SP by bus error at PC 0x40C6681C, address 0x424B).
2147483647 2012-Jun-27 20:23:40 Emergency %OS-F-BUFFER: OSBUFG_buf_alloc: Buffer pool magic is invalid ***** FATAL ERROR ***** Reporting Task: GOAH. Software Version: 220.127.116.11 (date 11-Jun-2012 t ime 17:35:31) <snip> ***** END OF FATAL ERROR *****
The switch was running a relatively basic configuration and not under any appreciable load when the crash occurred. It had been running 18.104.22.168 for quite some time and had been restarted several times since the firmware upgrade. I have not been able to reproduce the issue on demand.
I have a WS6509-E which is running on below IOS had a software reset yesterday, however I am not able to identify the exact reason for this to happen.
1 day, 2 hours, 39 minutes Time since xxxxx switched to active is 1 day, 2 hours, 38 minutes System returned to ROM by error - a Software forced crash, PC 0x402D3920 at 13:14:51 AEDST Thu Oct 13 2011 (SP by power-on) System restarted at 13:27:28 AEDST Thu Oct 13 2011 System image file is "sup-bootflash:s72033-jk9s-mz.122-18.SXD7b.bin" 1 day, 2 hours, 39 minutes
Time since xxxxx switched to active is 1 day, 2 hours, 38 minutesSystem returned to ROM by error - a Software forced crash, PC 0x402D3920 at 13:14:51 AEDST Thu Oct 13 2011 (SP by power-on)System restarted at 13:27:28 AEDST Thu Oct 13 2011System image file is "sup-bootflash:s72033-jk9s-mz.122-18.SXD7b.bin"
I have tried to update firmware on a RV220W. Something stopped and now it won`t start. The power led blinks a few times and goes off, the LAN and WAN port wich is connected is lit. I have tried 10 seconds reset button but nothing happens.
I have 3560X-24T-S switches with IP Services loaded at remote sites that I have been having troubles with. Originally, they had IOS 12.2(58)SE1 on them. I have up to four diverse paths via point-to-point microwave at the remote sites. The Microwave equipment is by Microwave Networks and is a Prodeus M series which Ethernet support. The original issue manifested itself as hardware loopback errors on some of the ports that were connected to the microwave links.
My experience in the past has been that when a hardware loopback error occurred it is usually a bad switch port. In this case however, if I disconnected all of the microwave Ethernet links and rebooted the switch and then connected the Ethernet connections to the microwave links again, everything works fine. No hardware loopback errors. That is, until the next switch reboot. Then the hardware loopback error would return. Interestingly, it would come back on different ports connected to the microwave links every time. So if a reboot was done without disconnecting the microwave Ethernet links the hardware loopback error would change from one microwave link to another after each reboot.
I then went through and read the lengthy release notes for IOS version 15.0(2)SE and found several fixes that I thought could fix my issue. So I downloaded it and updated a couple of the offending switches (not all of them were having this problem). After going through the second update required to resolve the 'open file error' that happens going between 12.2(58)SE1 and 15.0(2)SE the problem seemed to be resolved on the offending switches. So, I went ahead and updated the IOS on all of the switches with point-to-point microwave connections.
I now have one switch that was updated that is crashing and rebooting continuously when the Ethernet links for the point-to-point microwave are connected. Again, if I disconnect all the microwave links and reboot, it comes up fine ands stays fine when the microwave links are connected back up. It will work fine until the next reboot and then the crash and reboot loop starts over again. Below is a portion of the putty log when the crash occurs:
---------------------------------------------------------------------------------------------------------------------- previous memory block, bp = 0x59BF838, memorypool type is Processor data check, ptr = 0x59BF860 ========= Dump bp = 0x59BF87C ====================== 59BF77C: 0 0 0 FD0110DF AB1234CD FFFE0000 56 383FDB4 59BF79C: 212AC68 59BF838 59BF6F4 80000042 1 0 A504F53 543A2050 59BF7BC: 6F727441 53494320 506F7274 204C6F6F 70626163 6B205465 73747320 3A20456E
I noticed that my router 7200 was reboot for a bus error. The error occurred while the system was operational. After the restart the router works properly for more than 6 days.
This is the sh ver:
Cisco IOS Software, 7200 Software (C7200P-IPBASEK9-M), Version 15.0(1)M4, RELEASE SOFTWARE (fc1) Technical Support: [URL] Copyright (c) 1986-2010 by Cisco Systems, Inc. Compiled Thu 28-Oct-10 22:57 by prod_rel_team
Seems that the error "System returned to ROM by bus error at PC 0x2E0BF30, address 0x0" should be a special type of bus error crash is when the crash is caused by a corrupted program counter (PC). The PC value is the location of the instruction which the processor was executing when the bus error occured. In this case, the PC has jumped to the address 0x0 (probably because of a null pointer), but this is not where the instruction is located.
Can I consider this as a processor memory parity errors and wait for a second parity error before I replace anything?
Any online 2504 controller crash after upgrade to the point where even console port does not put anything out? Perfectly working 2504 running 7.0.116 ran upgrade through web GUI to 7.2. GUI showed perfect transfer, save etc... Asked for save before reboot. Select yes. Controller reboot and now it sits with following status lights:Power-greenSYS - amber blinkingALM - amber solidNothing out of console.
I found a new bug in cisco IOS 15.1(4)M3 when running EEM script with syslog event detector.If system logging performed using the "logging discriminator" and run concurrently EEM script with syslog event detector, then Cisco router crash and goes to reboot.
We had an "Event" on our process network at the mill yesterday. I connected a new WS-C3560V2-48TS-S to our network and we lost communications with all of the other switches.
The core is a stack of 3 WS-C3750 switches, one is a -24TS-1U and the other two are -12S-S. These connect to 10 WS-C3560G-24TS and 5 Rockwell Stratix 8000 (IE3000) over fiber. I am planning on replaceing two 24 port switches with the new 48 port. I had the switch configured and running at my work bench. It was connected to the network with one sfp module and ran all weekend with no issues. Yesterday afternoon I took it to the network cabinet and installed it. I powered it up and connected two sfp modules to the fiber patch panel and made the connection at the core stack.
Everything looked OK. I had communication link lights working on everything. Within minutes, we lost communication with every switch connected to the core stack. I shut down the new 48 port switch and the network slowly came back up.
The new 48 port is configured with Flex Links for the fiber redundancy protocol and was connected to each of the 3750G-12S-S stack members.
I though it might have been a power issue but the stack is UPS protected and shows it has been up for over 10 weeks. I'm not even sure "Crash" is the best decription for what happened. The new switch has a high enough IP address that it would not take over as a IGMP Querier.
I am dealing with three office networks, all three have the same conflict. The error messages I get all say there are two devices assigning ip addresses. As soon as I get the computers communicating with each other, two seem to crash overnight, I suspect due to the conflict. Here is the rundown on the networks. All are windows networks - the problem I am having is file sharing mostly and printing.
Network A: server OS is XP Pro - SP3, four workstations, XP Pro-SP3, Windstream DSL and Netgear Router. I have already bridged the router on this one, and it did not correct the conflict.
Network B: Server OS - Windows7 SP1, three workstations all XP Pro SP3, ISP is Clear (formerly clearwire). Router is the old wired Linksys befsr41 v4.
Network C: server OS - XP Pro SP2, ISP - Clear, three workstations - two are XP ProSP3 one is XP Pro SP2. I have had less trouble with this network, but I suspect the conflict exists here as well.
I have tried to get a static IP address from Windstream where I have had my worst problems, but cannot seem to get one. I suspect we have the least expensive service option possible, and are possibly using a service designed for home use as opposed to business, as someone told me recently that getting a static IP address can be expensive. Is this true? Currently all networks have DHCP enabled, and the routers are set up to use DHCP. My thoughts were to disable DHCP and assign ip addresses manually.
About a week ago, our network started to experience complete failure.
Router: Netgear FVS336.xxx Switch: Dlink DES-1024D (we have 6 of these switches in our building) Wifi Routers: Dlink DIR-601, Linksys wrt54g, wrt54gv2 Computersservers: 40x Copiersfax: 5x Wireless clients: (iphones, laptops) at least 30
All has been working together in harmony for many years.Fast forward to last week when my hell began with intermittent crashes.Not sure what's causing the problem.When it happens, everybody loses their internet, the ability to communicate over the network; ping tests won't respond to anything (including the router) that exists on the lan, even with a static IP against other statics.In visually inspection of the switches and router, they seem to be happily blinking away with no apparent issue (visually).The only way I've been able to fix the problem is by power cycling the router.About 1 second after doing that, we're back in action for 2-6 hours (very unpredictable).I never had to power cycle any other devices)
Steps I've taken in troubleshootingeliminating potential culprits:
Replaced the router with a spare (new) FVS318G
Replaced the power supply for the router (suspected because problem remained after router swap)
Replaced cable going from router to lan switch
Removed linksys wifi router from network (the router had needed power cycling weekly and was old. So I unplugged it)
Removed Dlink wifi router from network (I hate dlink routers, so removed it).I'm at a loss.
i upgraded my computer from win 7 ultimate 32 bit to 64 and when i shut down my computer and power up the local area connecion dont recognize the ethernet card my ethernet card is 82578 dc gigabit not onboard card and i installed all the drivers to the ethernet card could it be my motherboard ? dp55wb?
I just recently had my hard drive replaced by the manufacturer of my computer (Lenovo) because it started clicking. When I got it back it started making my wireless router/DSL modem reset all the time. I didn't have this issue before. I'm running Windows 7 Home 64 Bit. While the computer was off being repaired I was using a Win XP machine and had it connected through Ethernet cable. I did not have any issues with the router/modem while the XP machine was connected. I tried connecting the Win 7 computer up through ethernet cable (hard wired) and it still crashes the router/modem. I also connected the XP machine back up (hard wired) to the router and had no problems. Is it possible that the wrong drivers were installed by the lenovo techs? I have also noticed that my touchpad does not work the same as it did before too. And is it possible that my laptop is making the router crash out due to wrong configurations?