Friday, October 25, 2024

Lessons Learned - What I Learned on My Last Real Job (the Technical)

Lessons Learned

Episode 02: What I Learned on My Last Real Job (the Technical Stuff)


Read Episode 1 here: https://blog.smallbizthoughts.com/2024/10/lessons-learned-new-blog-series-episode.html



It is unfortunate that we are often drawn into the daily drama of our lives and only appreciate the "lessons learned" some years later. Reflection is a beautiful thing. I am grateful to have been in the right place at the right time with my last real job. 

The Internet and small office computing were taking off. I was hired to manage the "big iron," as mainframe and mini computers were often called way back when. Shortly after I arrived, the person in charge of all the other technology in the office (local area network, desktop computing, and even the security system) quit. Without batting an eye, my boss handed me all of his responsibilities. I remember thinking at the time, "Well, this guarantees a full MBO payout and a raise in the new year." 

(MBO stands for Management by Objective, a good idea that is normally poorly implemented and forces middle managers into despising their jobs. We'll hear more about this in the next episode.)


Luckily for me, I had managed people and complex systems before. And this move made it easier for me to manage both sides of the glass wall between the "big" computer systems and office desktops. One of my team was a Cobol programmer who dealt strictly with the HP 3000 systems. I hired an additional programmer who could work on smaller and Microsoft-centric systems. We gathered data from many sources, and it was easier to deal with much of it in C+ than in Cobol. This also set the stage for a major development project.

The HP 3000 family. Image source:
https://www.hewlettpackardhistory.com/item/all-in-the-family/

In 1993, the Internet was not open to the public. I wrote an essay explaining why we, as a private enterprise, should have access to this government-funded resource. Part of the deal was to provide access to information that would not otherwise be accessible to the public. And so, we had one of the first HP-MPE/x systems connected to the Internet. Subscribers no longer had to rely on modem access because we had a blazing 56K fractional T-1 line!

One interesting piece of this puzzle is that I learned some cool technology, including the details of T-1 communications, data packets, DSU/CSU, and Router setups. I noticed that pretty much everyone used Cisco routers and filed that information away for investment purposes. 

I also saw that all of our clients were going through a technology turmoil. They either had no networks or networks that could not route traffic to the Internet (and therefore to our services). I learned many different systems for fixing and upgrading Novell and NetBEUI systems so that TCP/IP played a role. I also learned the various ways to get them connected to the Internet.

In 1994, the Internet was opened to the public and I was beginning to see that the next five years would be dominated by getting millions of un-connected people and offices connected. Like everyone else, I saw the boom coming. I can't say I saw the bust.

As far as daily operations were concerned, I was lucky to have read The Emyth Revisited by Michael Gerber. I did not own the business, but documenting every procedure in excruciating detail became my standard operating procedure. I had been thrown into a bit of a sink-or-swim situation. And when I took over all technology in the corporation, that was even more sink-or-swim. I vowed to pass on a better system.

Most of my staff were college students who worked primarily during the legislative sessions in Sacramento. So, creating top-notch documentation became a high priority. We wanted to save time and money in the long run by making sure that all new employees received real training, with detailed checklists for every procedure. 

In preparation for expanding our operations into a new state, I elevated two staff members to be the Morning Supervisor and the Evening Supervisor. That freed me up to work normal hours and have a life. My staff operated from 7:00 AM until midnight, and I refused to make people work unreasonably long shifts as I believed it to be an unsustainable practice. The two shift managers did not know it at the time, but they were being groomed to take over an entire new office in another state. In the meantime, they learned processes, procedures, and some management skills.

One huge crisis occurred under my tenure. We had a hard drive failure on the HP 3000. This was almost inevitable as the drives were big, more mechanical than electrical, and old. I managed a backup system that consisted of a 57-tape reel-to-reel tape backup every night. 

When the hard drive failed, I learned the most important lesson of backups: You cannot rely on the "most recent" backup! You need multiple restore points, and good media.

The most recent backup was incomplete because it was under way during the failure. Because we sent tapes offsite every day, I had to request the backup tapes from the previous night. I loaded as much as I could from them, and then loaded any newer information I could from the newer, incomplete backup. Overall, we lost a little bit of data, but our team re-entered the day's data in short order.

Luckily, we had a great backup system in place, and it worked. It was a lot more labor than it should have been, and we immediately made plans for a better, newer, faster system. But having lived through a backup that cost the company a HUGE amount of money due to downtime, I learned to make good backups an absolute requirement going forward.

My greatest opportunity and adventure in this job was to design and manage the creation of a whole new legislative tracking system to replace the MPE/x and Cobol system. The new system would be based on Windows NT Server and SQL. Both of these were pretty new. SQL was designed for the OS/2 operating system and was updated to work on Windows NT Server when it was released in 1993. We hired a team run by a former Microsoft employee to design and build the new system. They were based in Seattle. Our first new site would be based in Olympia, Washington to track the Washington Legislature.

This was not a coincidence. I was from Washington State and had run politics-focused educational programs in the state. I had also worked on a gubernatorial campaign in college, and a key staff member for that was now a highly placed lobbyist. He gave us an introduction to the people we needed to know.

Designing the massive database, the overall project, all the inputs, and all the reports was a massive undertaking. The project started at $250,000 and went from there. By the time it was complete, we had offices in Washington State and we'd sold a license to the Chamber of Commerce in another state. The system worked, although I spent a year learning all the hard lessons of project management and programming on the fly.

In January, 1995, I made a trip to Missouri that is very reflective of the technology of the era. It also represented the end of the era. I met the principle programmer there, and we spent four days building the system, downloading drivers by dial-up bulletin boards, and bringing it to life.

It was also the last place I ever worked without an Internet connection, except businesses where I installed the Internet connection. Once again, it was clear that the world needed people who understood the old ways and could bring businesses into the new always-on connectivity of the Internet.

We had built a great, modern "client-server" system. Today it would be considered a hosted service or SaaS. Terminology comes and goes. Technology always moves forward.

By the time I finished that trip, I knew I was going to quit my job. I needed to put some things in place before I departed. But - technology aside - it was time to go.

I'll tell you that story in the next episode.

-----

How to follow the "Lessons Learned" blog series: 

1. Follow this tag: "Lessons Learned" - the link is https://blog.smallbizthoughts.com/search/label/Lessons%20Learned

2. Be sure to subscribe to this blog. Over on the right-side column, you'll see the Subscribe by Email link. Just remember to white list the first email you get. And if you filter, put them all into one folder so that you can find new installments.

3. Subscribe to my weekly Small Biz Thoughts newsletter. Go to https://smallbizthoughts.com/newsletter/. That will get you some news, some commentary, and links to this and all the content I create.

:-)


No comments:

Post a Comment

Feedback Welcome

Please note, however, that spam will be deleted, as will abusive posts.

Disagreements welcome!