Sherlock Holmes: "Once you've eliminated the impossible, whatever remains, however improbable, must be the truth."

In the Trenches with LAROKE

Konsultant's Log, Cyberdate 12.30.1998 (Old Blue survives his sea trials)

SITREP: Well, it's Christmas eve, and we're only putting in half a day here at the architectural firm where I work. A good time to compose the first draft of a long overdue log entry for In the Trenches, the last for this year.

This time we will finish up Old Blue's transition from Web/file server to firewall/proxy server and mail server. Old Blue has been undergoing testing and fine tuning for several months now, his "sea trials".

He's been running "like a top" except for an alarming behavior upon startup that has me perplexed. Upon booting into Windows, the machine goes into a thick molasses slow-motion state where the cursor freezes and even keyboard commands are impossibly slow, It seems some process is stuck in a loop that eats all of Old Blue's CPU clock cycles.

This episode is about all my efforts to remedy this supposed glitch. These herculean efforts are all for naught and in the end I discover that there is not a problem after all. Roll up your sleeves and we'll get started.

TACAMO:

Part 1: Scrambling around in search of a solution

6/5/98 Old Blue just began to choke - He entered an extremely "slow-motion" state, as if all his clock cycles were being eaten up in some infinite loop. I performed a warm restart, but the problem remained. Next, I tried a cold-boot recycle. That appeared to fix the condition.

6/6/98 Replaced Old Blue's 2-4x32 8MB 70ns non-parity SIMMs with 2-8x32 32MB 70ns non-parity SIMMs to bring him up to 64MB total RAM (his maximum). Old Blue restarted without problems for a change and noticed the additional memory. After entering the BIOS setup program and saving to record the memory change, the PC911 "watchdog" noticed the BIOS change and asked for instructions - it was told to accept the change and record a copy of the BIOS. Old Blue continued to bootup into Windows 95 without problems (PC911 was a wonderfull DOS utility that it's publisher, Cybermedia, no longer supports).

6/9/98 Updated WinGate's Registration to Professional edition 5-user license (does not expire). Downloaded and installed WinGate 2.1d version (replaced 2.1b trial evaluation version).

6/10/98 WinGate Gatekeeper locked-up with a BSOD (blue screen of death) unexpectedly . . . I terminated the application from the death screen, then tried to restart GateKeeper. Old Blue responded by locking up solid with bizarre screen images . . . needed a cold reboot. Started Gatekeeper again and tried to adjust "Cache" size setting from 100MB to 50MB (buggy behavior has been reported in the WinGate support forum for sizes above 50MB). Then I tried to purge the cache before exiting the GateKeeper settings dialog - Old Blue entered his extremely "slow-motion" state first observed on 6/5/98. Had to shutdown and cold reboot again. This time when Windows started, the WinGate engine was shutdown in order to delete the cache files manually (9,694 files, 60.1MB). The WinGate engine was then restarted and the Cache size adjusted from within Gatekeeper.

6/12/98 Old Blue's WinGate environment has been stable for two days. I downloaded and installed a trial version of MDaemon 2.73 mail server software and configured it to work with WinGate. Several other PC's on the company network were configured with Windows Exchange and MS Internet Mail clients. Tests were run continually with e-mail and big file attachments, both to the Internet and back, and through the LAN. Old Blue performed superbly. He did not choke once while dealing with Internet Web and FTP access sessions through WinGate, E-mail sessions though MDaemon (and WinGate) while simultaneously performing a network backup using Cheyenne Backup!

6/18/98 Recycled the entire network to hopefully purge the gremlins from the computers "4-Bits" and "Shamrock I". This scattergun approach seemed to work for Shamrock but 4-Bits is still malfunctioning. In addition, Old Blue would not shutdown properly and also suffered the "slow-motion" malady on startup, but was OK after that.

6/22/98 WinGate GateKeeper caused a "Blue screen of death" cascade. MDaemon was the next application to lockup, then the Windows Explorer shell and finally the WinGate engine. Old Blue was cold-booted twice, both times with the "slow-motion" problem. In both cases the start button and task bar never appeared and MDaemon was the only program that started. A third cold-boot was tried into Windows "safe-mode" this time - HIMEM.SYS reported "HIMEM.SYS has detected Unreliable XMS memory at 03F80030 XMS Driver not installed" and aborted the process (I was unceremoniously sent to the DOS prompt instead of safe mode). a CTRL-ALT-DEL boot was attempted at the command prompt after the abort and a normal Windows 95 startup was achieved (without slow-motion effect).

6/27/98 During network backup, establishing the drive mapping to Drive C: of system 16 "Pentagon" caused the DUN dialog for our ISP to appear. The MS Networking workgroup was changed to "VPA" and Old Blue was restarted. I got a "Disk Drive Not Ready: Retry, Abort, Fail" error during restart and after retrying one time, Old Blue was shut down for a cold boot. The cold boot process completed normally.

7/18/98 The Cheyenne Backup caused a lockup. Old Blue was cold-booted many times, with the "slow-motion" problem. In both cases the Windows 95 start button and task bar never appeared and MDaemon was the only program that started. A third cold-boot was tried into Windows "safe-mode" . . . Again, HIMEM.SYS reported "HIMEM.SYS has detected Unreliable XMS memory at 03F80030 XMS Driver not installed" and aborted the process. ~!@#$% This effect was achieved time after time. I could not get Old Blue to reboot normally into Windows 95.

"PC Clinic" by Data Depot which I had purchased from my rep, Basil, at Quarterdeck was still laying around unopened in the Krash Lab. With some difficulty I managed to install it on Old Blue (the PC Clinic Installation floppy was bad - I ran it thru SCANDISK on 4-Bits and had to "truncate" the CLINIC.HLP file before I could copy it to Old Blue's hard drive. That done, I started running memory tests . . . went thru the 1st iteration without comment. At the bottom of the screen was a note that performance could be increased by turning off the cache. I stopped the process and turned off the external cache in the BIOS setup. I restarted into Windows just for giggles to see if this had any effect . . . no joy.

Back to PC Clinic. Ran the XMS memory test - memory was reported OK. Turned-off Fast CPU start in BIOS setup and tried again . . . no luck.

Next, I dug Old Blue out of his corner of the Krash Lab and replaced the 2-8x32 32MB 70ns non-parity SIMMs with the original 2-4x32 8MB 70ns non-parity SIMMs. Put Old Blue back in his corner and after resetting memory in BIOS tried to boot into safe mode again - this time it worked but I got a "corrupt registry, restore from backup" message. ~!@#$%^! Curses! I accepted and when Windows was finished restoring, let Old Blue reboot. Still caught in the impossible slow mode. I looked back thru Old Blue's maintenance log entries at this point and discovered the first time this trouble manifested was shortly after installing Quarterdeck's RealHelp.

I painfully negotiated the Windows 95 menu system on Old Blue in his crippled slow-motion state using keyboard commands until I got to the Install/Remove Programs applet in the Control Panel. I slowly uninstalled Realhelp. Eureka! As soon as Realhelp was gone, Old Blue's interface sped up to normal and this was before rebooting Old Blue again which I did for good measure. Wingate and MDaemon were back! Dare I try Cheyenne Backup? I dare. Cheyenne Backup was fine . . . In fact, the operation that blew up Old Blue in the first place worked now!

Quarterdeck RealHelp is a diagnostic utility. One of its functions is to keep the machine stable and here it is, the cause of the instability!! The ~!@#$%^ computer industry is right where the American automotive industry was in the early seventies. Bloated, broken products served up by arrogant, damn the public, management. We all watched the Japanese and the Germans take the market away from the American automobile companies. I hate to see the same thing happen to the small computer industry, but it is ripe for someone to come from behind and serve the customers. OK! I feel better now. Down off the soapbox to ponder my next move. I'll wait til tomorrow to restore the memory SIMM's.

7/20/98 The 32Mb SIMM's were reinstalled in Old Blue. After restoring Old Blue's BIOS setup to 64Mb memory I tried booting into safe mode . . . HIMEM.SYS still reported "HIMEM.SYS has detected Unreliable XMS memory at 03F80030 XMS Driver not installed" and aborted the process again. Old Blue will startup normally into Windows and runs Wingate and MDaemon with no discernable defects, and PC Clinic reports that XMS memory is OK. So why the heck is HIMEM.SYS choking in safe mode?

7/26/98 The slow-motion problem is back, so it's probably not RealHelp after all. ~!@#$% Darn! Thought I had this one solved. The Cheyenne Backup has malfunctioned several times this weekend. Could that be part of the problem? I've also had problems connecting to the Internet today. The connection problems have been sporadic. Is it WinGate? Is it the SupraExpress modem? Is it the ISP? Is it Old Blue? Is it all of the above? I think I'm going to carry through my original plan of moving the tape drive and Cheyenne Backup to Johnny Mnemonic to try to isolate Old Blue's problem . . . I want to get beyond this disturbing behavior.

8/5/98 The slow-motion malfunction DITTO. Old Blue was backing up Johnny Mnemonic. This problem seems more prevalent during backups . . . or maybe they trigger it. The next step in tracking this bug will be to move the tape drive and Cheyenne Backup to Johnny Mnemonic to see if that makes any kind of difference.

8/8/98 Prepared to move DataStor tape drive and SCSI adapter from Old Blue to Johnny Mnemonic. Dug up the DataStor documentation and Cheyenne Backup installation CD. Checked the SCSI jumper settings in Old Blue's Device Manager. They were set at the default values of IRQ 12 and I/O Address of 140h. Pulled up Johnny's Device Manager to see if these resources were available. A good omen . . . They were. Removed the SCSI adapter and found a back cover plate for the vacated slot. Fired up Old Blue with his cover off to determine if all the cooling fans were still working . . . They were (two fans on a fan card and the power supply fan).

During bootup, Old Blue reported a corrupted CMOS checksum. I reset the clock in BIOS setup, then continued the bootup. The PC911 watch dog noticed the CMOS changes and offered to restore from floppy disk. I accepted but the restore failed because the floppy drive was not working anymore. The bootup continued into Windows in the now all too familiar slow-motion mode. When the bootup was finally complete Old Blue went back to normal speed and both WinGate and MDameon appeared to be working properly. The "Removable" drive icon was back in the "My computer" Window, however. I decided to let sleeping dogs lie for the moment and get on with installing the tape drive on Johnny. I will fix the floppy problem and restore the CMOS from PC911 later. I will also get a new CMOS battery.

Part 2: Solution found - Do nothing!

8/11/98 Just read from WinGate maillist that "frozen mouse slow-motion startup" is caused by WinGate purging its cache and that normal desktop operation will return (sometimes 20 to 30 minutes) - will have to check this next by turning cache off for awhile.

8/30/98 During normal maintenance, I deleted all the files in the WinGate Cache (except CACHE.IDX) then logged on to GateKeeper and rebuilt the cache index. This action caused the same slow-motion mouse activity but this time I waited it out and sure nuff, mouse movement returned to normal after a few minutes. Old Blue was rebooted several times during the course of checking emergency bootup disks and he started normally every time without the slow motion problems.

They have a "wish list" in the WinGate Forums for future features. I wish for a "Progress Thermometer" to inform users while WinGate is slowing down the computer interface during the cache purge and indexing operations, or at least documentation of the phenomena in the help file. It would have saved me a lot of anxiety and misdirected maneuvers to correct the situation.

CM:

8/30/98 Again, during the first reboots, the PC911 watch dog noticed the CMOS changes and offered to restore from floppy disk. I accepted but PC911 still could not read from the floppy drive. After two attempts, I selected the skip option and PC911 trundled on to report the operation a success . . . Wot the heck? When Old Blue finished booting into Windows, the floppy drive was back in operation. I'm not sure what happened here, but I'm not going to look a gift horse in the mouth and worry about it anymore.

10/8/98 Am having trouble with the browsers on the various PC's not getting updated versions of envoy files. These files are being cached somewhere, either in the WinGate proxy cache or the browser cache on the PC or both. In the GateKeeper interface for WinGate I found I had a cache filter wrong - the filter was set up not to cache anything from a "client" IP address of 192.168.0.21 (Johnny Mnemonic) where it should have been set up not to cache anything from a "server" IP of 192.168.0.21. After correcting and saving the filter, I shut down the WinGate engine, deleted all files in the WinGate Cache except "CACHE.IDX" (the index). After restarting the WinGate engine, I entered GateKeeper again and rebuilt the Cache Index. I purged the browser cache on HAL, set preferences to check for a new file every visit, and tested. Preliminary results indicate this may have solved the problem.

MISREP: Whew! At the date of this writing, Old Blue is working like a Timex watch (John Cameron Swayze's "Takes a licking and keeps on ticking!"). I have not gotten around to replacing the CMOS battery and this neglect may eventually kick me in the . . . Uh, seat of my pants. Since Old Blue is a 24/7 (24 hrs a day/7 days a week) server and receives his power via an APC UPS, he is rarely on battery power.

This is the end of 1998. A lot was accomplished this year at my day job. We added/upgraded two network servers "Old Blue" and "Johnny Mnemonic". Three new workstations were added "Kato", "Cygnus" and "Trailer Trash". One workstation "Pentagon" was upgraded. The first 10BaseT Ethernet hubs "Deep Space Nine" and "Babylon 5" were added to our original 10Base-2 coax segment "The Stillwell Burma Road".

Initial planning for 1999 has started. The "Millennium Bug" or Y2K issues will demand most of my attention this year. With that in mind the last Windows 3.x machine, "Merlin", on the network will be retired. He will be replaced by "P2" after I build a new machine for Christine, P2's current user. Our last MS-DOS machine "Kenny A" will also be retired. His duties as plotter server will be taken over by a dedicated device, an ignoble end for a 286/12MHz old timer that has been chugging along without complaint for twelve years. A new 10BaseT Ethernet hub, yet to be named will be added in the Krash Lab area to make temporary connections more convenient in the "service area" and to shorten the length of the existing 10Base-2 coax segment. Finally, a top-of-the-line workstation may be constructed for the firm's president.

That takes care of hardware. My biggest headache on the software front will be our custom programmed database applications which are not Y2K compliant. I could curse the original programmer of these dBASE IV and FoxPro v2.6 applications but I won't. Why? Because the programmer was me (We have met the enemy and he is us).

I'll get started on some of these weighty issues with the next installment. Until then, happy holidays, everyone!

LAROKE Microcomputer Consultants
155 East Boca Raton Road
Boca Raton, Florida 33432
(561)368-0659 (Tel & Fax)

Issued Wednesday December 30, 1998

copyright © 1996-1998 LAROKE Microcomputer Consultants all rights reserved

Search www.laroke.com the Web for