Mike's PBX Cookbook

BCM 400 Trouble Shooting Guide

(or cursor keys)
Customers hear cracking noises
Potential issue: BCM 4.0 static problems.

Action: Determine where customer is reporting static, and isolate to specific equipment.

Use the following steps to narrow scope of problem (Note: Static typically occurs in analogue sections):

Start with a digital set if possible, go off hook, listen for steady dial tone, type number to break dial tone, listen for quiet.

If static isolated to digital set:

  1. Try another hand set
  2. Replace curled cord
  3. Replace the set

If static isolated to GATM port:

  1. Check demarcation point with Butt-in Set to verify line quality
  2. Frog lines with another GATM port to isolate wiring or port

If static isolated to IP set:

  1. Try another hand set
  2. Replace curled cord
  3. Review alarms for packet loss
  4. Replace the set

If static isolated to T1/PRI:

  1. Verify settings such as clock source, build out, protocol
  2. Determine if there are DTI alarms in logs
BCM 4.0 new unit with flashing status, MSC, WAN, MODEM, LAN1, and LAN2 LEDs
Potential issues: BCM 4.0 No bootup / long boot up

Action:

  1. If it is a new unit, then ensure the installer gives it at least 60 mins to boot
    If the BCM is performing FSCK (file system check), it should come up within 1hr to 1hr15min

  2. If unit is not up after 60mins, connect to the console port with a hyperterminal and a null modem serial cable (9600,8,n,1). Hit enter to get a login prompt (nnadmin, PlsChgMe!) otherwise the boot up sequence can be observed.

  3. If unable to login on serial port, remove VGA & keyboard access panel, and attach a monitor to view POST and Linux boot.
    If POST completes correctly, the last line displayed will be "Uncompressing Linux OK, booting the kernel.".

  4. If you do NOT see "Uncompressing Linux OK, booting the kernel.", the likely cause would be a "bad/corrupted HD".
    Confirm error messages from POST, replace or re-image the HD.

  5. If POST (power-on-self-test) has NOT completed correctly, open up the BCM to ensure all connections are intact.
Customers can't access BCM to do any programming
Potential issues: EM unable to connect to BCM

Action:

  1. Check LED status normal (ie. LED 1, 3-10 All Green). (LED 2: Drive LED could be dark if no HD activity).

  2. Ensure installer has access to the LAN port (ie. able to ping the port).

  3. Check to ensure the installer has the correct version of EM on PC.
    If in doubt, download the current version from the Web page of the BCM.

  4. Ensure the Date and Time are set correctly in BCM

  5. Try Telset programming. If Telset programming works, then it confirms the problem resides with EM.

  6. If nothing works, then collect logs and escalate. Could be other issues such as RPM corruption
BCM 4.0 can't patch or backup
Potential issues: BCM configuration or performance problems

Action:

  1. Review alarms and status LEDs for errors (ie. LED 3-9 should be Green). Take action on any errors.

  2. If alarms report "/var/log partition full", check if BCM is patch current, if yes then escalate.

  3. Verify where patch is located (e.g. from USB, server). Try another PC to see if the operator can backup or patch.

  4. Check to ensure the installer has the correct version of EM on PC

  5. Ensure the PC is set up correctly to communicate with the BCM

  6. Ensure the Date and Time are set correctly in BCM

  7. Check physical connections (e.g. LAN cables)
Customers hear echo on the phone
Potential issues: BCM poor echo cancellation, poor line quality or incorrect configuration

Action:

  1. Ensure the unit has the latest core patch and dependencies

  2. Check analog, GATM at correct firmware/version, DIP switches are properly set Bus Resource switches set properly: mode/country all off. GATM comes with auto line tuning functionalities (autoswitch between 600 & 900 ohms)

  3. Check digital, verify PRI/T1 settings for clock source, interface build out, protocol

If echo is occurring on GATM:

If echo is occurring on PRI/T1 with IP sets:

If echo is occurring on PRI/T1 with digital sets:

IP sets unable to find the server, "restarting DHCP"
Potential issues: Can't make calls on IP sets

Action: Check for Configuration and network problems

  1. Connect a laptop to that LAN drop to determine if there is connectivity
  2. Check with customer network architect/support to confirm VLAN setup
  3. Check if there are any errors while IP phone boots

If BCM CANNOT be pinged:

If BCM CAN be pinged:

Can't program BCM or IP sets can't connect
Potential issues: BCM 4.0 LAN port failure

Action:

  1. Check LED status normal (ie. All Green)
  2. Check with customer network architect/support to confirm VLAN setup

If link status LEDs are good and IP network valid:

If connected to customer's network:

If any digital sets present, telset admin may be used to:

If BCM cannot be pinged, use console port to:

Error Status (Red LED) on RAID HD controller
Potential issue: RAID controller or HD has failed. (85% of RAID drive subsystem failures are caused by a HD failure).

Action: Check Raid LEDs status: Normal = All 3 LEDs Green); Drive fail = Drive and Status LED red.

If Secondary drive failed:

  1. Connect secondary drive directly to motherboard to confirm drive failure
    Connecting the failed drive directly to the motherboard eliminates the RAID controller as a failure point

  2. Replace secondary HD if HD still unresponsive
  3. Replace RAID controller if secondary HD works directly with motherboard

If Primary drive failed:

  1. Move secondary to primary to confirm RAID controller good.
    Moving the secondary drive to the primary eliminates the RAID controller and prepares for remirror operation

  2. Replace primary HD if secondary works in primary position
  3. Replace RAID controller if neither drive recognized on primary RAID channel

Note: Mirroring takes hours (e.g. over 3hrs).

If after replacing the hard-drive you're still seeing errors, check the HD size. Off-the-shelf HDs may not not be compatible with BCM. Always buy the correct replacment HD.

BCM restarting, loss of IP sets & voicemail
Potential issue #1: Resource conflicts?
  1. Use BCM Monitor to confirm that there are enough resources available. Signalling channels, PEC resources.
    In a 6/2(5/3) Split, there are only 59(91) signalling channels available for VoIP trunks, Call Pilot, and IP sets.

  2. Confirm that no ports are in "disabled by user" state (resources->telephony->bus->port details)
  3. Confirm that call center CDNs are properly configured (Digital Station emulation, port unequippped)

Potential issue #2: Defective BFT (Base Function Tray)?

  1. Confirm that there are no leaking or damaged capacitors evident on motherboard
  2. Confirm that motherboard does not contain light green capacitors
  3. Check the Power Supply LED
  4. Ensure DC voltages are correct (+/-12V; +/-5V; 3.3V)

Potential issue #3: Power source problem?

  1. Ensure the power source (e.g. from the wall outlet) is a dedicated circuit and it is NOT shared with other appliances BCM draws a significant amount of current especially with an Expansion Module

  2. If customer is using an UPS or other power conditioning equipment, bypass those devices and connect directly to the wall outlet. This will help to isolate the problem to the power source or BCM
Loss of telephony programming
Possible Cause: Defective MSC (Media Services Card)

Action:

  1. Confirm if telephony data back at factory (3 digit, 221 starting DN)
    BCM will default back to 3 digit, 221 starting DN if customer data is corrupt

  2. Change number of digits and/or starting DN, then remove power for 15 minutes and confirm configuration valid

  3. Replace the MSC if it is unable to retain data
Cannot apply keycodes
Potential issue: Get invalid keycode when trying to apply new keycode.

Action:

  1. Verify date, time and time zone are set correctly
  2. Verify no conflicts in IP subsystem
  3. Confirm System ID (Via EM) matches the one on the keycode file
  4. Ensure the sequence number of the keycode is greater than the one currently in the system
  5. Verify keycode file against KRS. If everything matches up, try to re-apply the keycode file.
  6. If re-apply fails, try enter keycode via Telset

After trying these steps and you still can't apply keycodes, escalate to KRS

BCM doesn't come up after patch
Potential issue: Patch Stuck

Action:

  1. Allow the system sufficient time to boot (eg. 60mins). Core patches need to be transferred to the MSC (Media Services Card), and they can take up to 45mins to install

  2. If unable to recover based on info in alarms, escalate problem
BCM 4.0 No LEDs
Potential issue: Is there power to the unit?

Action:

  1. Check whether the fans are working
  2. If fans not working, check electrical wall outlet
  3. Check the power cord condition
  4. If power source is verified, replace the power supply or replace the unit
BCM 400 system not coming up
Can't access telephony resources from EM, calls going into wrong Vmail, spontaneous restart, perhaps following a change from 2/6 to 3/5 split

Potential issue #1: Resource conflicts

Action:

  1. Confirm that no media bay modules are configured for bus 07
  2. Confirm dip switch settings on all MBMs looking for conflicts
  3. Confirm all CDNs are valid.
    Port use must be Digital Station Emulation and port must be unequipped (no phone attached)

Potential issue #2: Disabled Resources

Action:

  1. Perform a proper shutdown of BCM
  2. Disconnect power, then remove all media bay modules, disconnect expansion chassis if present
  3. Reconnect power and startup BCM
  4. Set all buses to DSM type and review status of all ports
  5. Any port with status of "Disabled by user" needs to be enabled
  6. Set ports back to original config on all ports enabled
  7. Perform a proper shutdown and disconnect power
  8. Install all media bay modules
  9. Connect power and restart BCM
  10. Escalate problem if procedure fails
Media Bay Module related issues like echo or other unexpected behavior
Potential issue: Media Bay Module firmware download failure. Note: Only GATM2 or newer version supports FW download.

Action:

  1. Check alarm 31 or 79 to determine which module is having issues
  2. Confirm FW version, under B1 channel
  3. Check dip switch settings. Mode/country switches all off.
  4. If it is an older MBM, replace MBM
BCM Password Reset
FEATURE 9*8

The User ID is the default Nortel telset one of ID = 738662 (setnna), and password 266344 (config).

Just click on the NEXT button 3 times to see System Restart appear.

Click OK and Yes to reboot the BCM.