Telecommunications Tech Blog June 2014
Can You Troubleshoot Phone Problems Remotely,
or Do You Have to Send out a Technician?
By Mike Sandman •
Last month I got a call from a
technician at a federal government agency that has modems all over the country
that talk to his main computer on the east coast. He had a problem connecting or
staying connected to some of them.
The technician wanted to buy an
"amplifier" to put on the modems at the main computer. He figured that if he
made the sound louder at the main location all the modems would work better. Uh,
I guess he's used to troubleshooting
computer problems remotely. Troubleshooting problems on analog phone lines with
analog phone equipment (like a modem) is totally different than troubleshooting
a digital problem.
When I told him that he's going to
have to pick just one site with a problem, figure out what's wrong, fix that one
and move on to the next one, he said "We can't do that."
He said the modems are spread out all
over the place and he can't send technicians out to fix the problems. He wanted
to do one thing at the main location to fix all his problems.
I got a call about the same time from
a radio station who said their on-air callers sound distorted so they wanted to
make the lines quieter (attenuate them) to prevent the distortion. I suppose
that's an interesting idea but without checking the phone lines themselves with
a butt-set or checking voltage, current and circuit loss, which he didn't do,
that's a heck of an assumption and would probably leave the guy spending money
on stuff he doesn't need and chasing his tail for quite a while. He wanted a
magic gizmo that would solve his problems.
The previous Sunday evening I had watched a
segment on "60 Minutes" about US nuclear missile sites (ICBMs). In an interview
a soldier at the site mentioned that they had problems with their phones. In a
later interview Major General Jack Weinstein said their telephones were analog.
That it would be years before they replaced them with digital phones (for billions
Well, all phones are analog to some
extent because our ears and mouths are analog. Although someone might be able to
figure out how to put a USB connector in our necks some day, I don't think it's
coming anytime soon. Most phones even use our analog eyes to see flashing lights
Analog phone equipment and POTS lines
have been repaired every day for over a century. It's not nuclear science and
almost anybody can fix phone stuff with a little experience.
Then I realized that the missile
silos are all located in hard to get to parts of the country. Even though broken
phone stuff is easy to fix, the logistics of getting someone who understands a
little about phones out to those sites isn't easy.
But there is absolutely no choice.
Phones don't fix themselves. You can't remotely fix wires and replace worn parts
(like a handset, cord or hookswitch). Same problem the guy from the other
government agency has. They don't have the manpower to send out to the remote
sites, and the guys who they do have are likely to be computer guys who don't
understand phone stuff.
The worst part for both of these
government agencies is that they've already waited until there are lots of
problems. If something is maintained in a reasonable matter it's just a service
call now and then. When someone lets the problems build-up with any kind of
gizmo, even a car, fixing all the problems is overwhelming and a new device may
be in order.
If there's no new device available,
the technicians tasked with maintaining the equipment are overwhelmed and
stressed. Stressing out employees is a good way of losing them. When you lose a
good employee, you lose all the knowledge they have of your particular products
The military has a bigger problem in
that all of their workers (soldiers) are contract workers. At the end of their
contract (tour of duty) they move on for greener pastures. Along with all the
knowledge of the equipment they've been working on and the specific sites
they've been maintaining.
In businesses, the bosses often tell
the IT guy that he has to fix the phones from now on (to save money on the
expensive telecommunications vendor they've been using). The boss figures phones
have wires just like computers, and the IT guy is sharp. Any maintenance
contracts get cancelled. That often doesn't work out too well since the IT guy
doesn't know anything about phones and he doesn't want to look bad to his boss
by calling in the old (expensive) telecom vendor for help. The real loser in the
end is the boss and the company.
I've always wondered how many bosses
tell the IT guy to fix their $60,000 Mercedes? After all, it's got wires.
Digital stuff is 1s and 0s. Usually
+5VDC or 0VDC. On or off. Put the ones and zeros in one end and they come out
the other end rearranged in the way the computer in-between wants to arrange
Analog stuff is totally different. A
modem or fax is just like our voice. One end makes beeps and boops at particular
frequencies and the other ends hears them and replies with similar sounds.
If every fax or modem was connected to exactly the same central office, with
exactly the same number of feet between the central office and fax or modem, the
fixes would be pretty much the same for every case of trouble.
• There are several makes and models
of central offices, with different line cards. The different line cards look
different electrically to the telephone equipment at the subscriber end - and
they can sometimes be optioned for more or less loop current (including
mistakenly optioned for one or the other).
• There is a different amount of wire
between every subscriber and every central office, and the wire could be bigger
• There could be load coils either
properly or improperly placed on the wires (load coils are inductors used to
raise the volume on a long phone line, but it often kills data being send on the
• There may be pair gain equipment
between the central office and the neighborhood the fax or modem is in. Like a
T1 or fiber. The line card that faces the subscriber is different for every
brand/model of pair gain out there, and can sometimes be optioned to look
• The phone equipment's impedance may
not match the impedance of the CO or pair gain line card, causing really strange
audio problems (and faxes and modems send audio beeps and boops to the
other end). The impedance can sometimes be optioned (including mistakenly
If you figure in all of the above
variables, you can see that no two phone lines and no two makes or models
of phone equipment looks alike.
The design of the telephone equipment
attached to the line may be good or bad. Almost every piece of phone equipment
uses a different circuit and different components facing the phone line.
Sometimes the engineer designing the
equipment chose the right components that would work in almost every condition
(which was the case in the old days), but more often these days the cheapest
possible components were used making it more likely the equipment won't work
right on any particular phone line.
It takes a technician with a spare
piece of phone equipment (for troubleshooting), a meter and butt-set to diagnose
why a piece of phone equipment/phone line combination isn't working right. Can
it be done remotely? No.
Here's what a flow chart would look
like for troubleshooting modem problems at multiple sites (you can see or
download the pdf version at:
The technique would be the same for
different types of trouble calls. Strip the problem down to the smallest
component, make sure it works (or doesn't work), then put stuff back one at a
time until the problem returns. The last thing you connected when the problem
returned is probably the culprit.
You can also do it backwards by
disconnecting one thing at a time from the end until the problem clears, but I
don't usually do that.
You need spares! If
you don't have spares to check to see if some gizmo is bad you're really going
to bang your head against the wall. Anything electronic can break, and does.
Replacing a device temporarily to see if the problem clears is often the only
way to find the problem. One problem with having spares is when the previous
tech using them didn't mark them as bad or questionable. Now you're going to
bang your head against the wall anyway!
I personally keep a roll of gray tape
with me for the bottoms of phones, and some orange tags for cards and phone line
demarcs, to make note of my troubleshooting (both for myself and the next guy on
the service call). I mark what I've done or what I suspect with a fine point
Almost every time I guess at a
problem and skip proper troubleshooting I waste more time than I could have
gained. But, there are a couple of cases I can think of that are so common that I'd
start troubleshooting in the middle. Even then sometimes I get surprised and chase my
The most common call I get where I
assume the problem and the fix based on symptoms, without taking any
measurements, is when a fire alarm panel throws a "communications failure"
In the old days companies would put
dedicated lines on their fire alarm. The guys who designed the alarm panels
didn't think companies would share these lines, that were supposed to save lives
and property, with a phone system. Today, it seems like no company will put in
dedicated phone lines for an alarm system so I point callers to our Fire
Alarm Tech Bulletin where the fix works 100% of the time:
The bottom line is that 99% of the time you can't
troubleshoot or repair phone problems remotely...
can do fewer service calls!
If equipment is installed so it can be unplugged and a spare can be plugged in
by the end user, and the premise has spares on-hand or you can ship a spare
there, you can save a lot of time and money. Your average customer may not go
along with that, but if they're in a remote area or are otherwise hard to get a
technician to, it may be a good way to handle service calls.
Handsets and cords are already
modular on most phones and can be swapped with another phone to see if they are
bad or intermittent, or replaced with a spare.
In the case of the missile bases they
could make sure the wall phones are modular, show someone who works there how to
swap a phone or other type of device with a spare, and just ship stuff up and
One of the problems with our missile
bases is that the phones were likely installed before phones were modular. Even
if installed later there may be special types of handsets or phones that aren't
modular. In that case someone at the site should be able to unscrew the phone
from the wall and splice wires with a spare with UY connectors (or just convert
stuff to modular so the workers at the site can swap stuff out themselves).
In the old days the #1 service call
an Interconnect phone man would go on is a dead CO line. And the problem was
almost always with the phone company.
When I realized what it was costing
us I started making all lines modular before they went into the system on new
installations with loop start lines, and put a numbered cord from the system for
each line into the numbered mod jack. I put a cheap ugly wall phone (so it
doesn't get stolen) by the mod jacks so the customer could unplug the system,
put the cord from the wall phone into the jack and test the lines themselves.
The modular cords were long enough so
the customer could even swap a line above or below if there was an intermittent
problem. So if they had a problem on CO 3 they would swap the cords between the
jacks marked CO 2 and CO 3 to see if the intermittent problem showed up on CO 2.
Unfortunately my experience is that
only 10% of the customers want to save the service call and troubleshoot CO line
problems themselves. So there are still a lot of service calls for lines that
are dead from the phone company. Maybe you'll have better luck in your neck of
the woods, or with specific customers?