Full Text Search: The Key to Better Natural Language Queries for NoSQL in Node.js
In this article, I have jotted down some observations I have made over the years that I'd like to share with you. Perhaps if you have flipped the bozo bit on someone, after reading this you will either feel vindicated in doing so or give that person a second chance.
Software Development Truisms
I am a freelance software architect. I work for a lot of different companies on both very short and sometimes very long projects on a contract basis. Often, I am asked to manage teams as an outsider, occasionally making hiring and firing decisions (which is tough as a contractor). Either way, I have witnessed a lot of good and some not so good events, have had both good and bad experiences, and seen some of the same bozo behavior in a lot of different places. Maybe you will recognize some of your experiences in these collected truisms.
Change is constant but people will seldom thank you even for good changes. Consider prefix notations. Prefix notations were adopted for weakly typed languages such as C and older versions of VB. But, weakly typed languages are ending up on the scrap heap because they are, well, weak. Hence, even Microsoft is not encouraging the use of prefix notations anymore. Are they still being used? Yes. Will the person next to you thank you for telling him or her about the history of prefix notations? No. In fact, they are likely to look at your constructive comments or historical perspective as a criticism. And, it is. Flip. Worse still, if the person next to you uses goofy prefix notations that you think make the code an eyesore, they are likely to cast disparaging comments about your lack of ability or professionalism.
The squeaky wheel gets the grease. Translation: Often, decisions are made to shut up the shrillest or most passionate person in the room. If you are bored and want to have some fun, make a decision that is really goofy and then see if you can sell it even in the face of its obvious lack of merit simply by being really outspoken and passionate. Odds are that if you are the last one talking, you win. Passion often overrules reason because people are emotional in nature more than they are rational. Have you ever heard the saying emotion sells? It is true. If it weren't true, arguments—like tax cuts—are only for the rich and Dick Cheney's friends at Halliburton are getting a sweetheart deal would seem silly to everyone. However, a rational person might concede that if one pays more taxes, one should get a bigger refund. Forget Halliburton. I am not even going there because it is too emotional.
When someone says a software project is going to be late, they are never lying. Delivering bad news is a risky proposition because messengers get shot all the time. (Socrates said none love the bearer of ill news. And, didn't they make him drink hemlock?) Software projects are late more than they are early by a huge margin. There is no way a Vegas odds maker would ever bet on an early project, yet project managers ignore news about late projects all the time. Flip.
Reading is Fun-da-mental
I have my theory about the phrase reading is fundamental because a lot of bad software is being written by people who don't read. It can't be an accident that the root words fun and mental are so strongly associated with reading. Be very wary of programmers who don't read. (Besides the fact that I don't get royalties for my books from these people,) non-readers ideas are based on their own gut instinct. Some very smart people have solved a lot of programs each of us will face and have taken the time to write them down, but some people are so smart that they can do it better the first time out. Flip.
Too many people believe too much of what they read. Simply because something appears on the front page of a newspaper doesn't make it true. The same is true of books. We authors are diligent, the editors are careful, but we make mistakes and give bad advice all of the time. Worse still, if the information is dated, it may be anachronistically wrong. For example, if you are reading a 1970s book on C Programming and it uses a prefix notation, it doesn't make it gospel. For this reason, it is a good idea to consider several sources before believing what you read. If anything you read in a book sounds wrong or fishy, it probably is. Read another source that contradicts the first and use your own judgment or gather your own empirical data, then decide. And if some bozo comes along and doesn't agree with you, consider the fact that they may have read something you haven't or that their evidence—which contradicted yours—was equally convincing to them.
Authors aren't smarter then you are; they just read more. A funny thing happened to me. A couple of people claimed to have read something by a particular author, and claimed to have adopted it as gospel. The funny part is that their adopted apostle is just one guy, and the other funny part is that they only adopted about half of the suggestions as gospel and made up the other half. As I said, anything goes. Flip.
A manager is not an architect, designer, or programmer and shouldn't do any of those things. Yet, how come managers are in technical meetings making decisions all the time? Frederick Brooks in his years-earned wisdom is exactly right; programming teams are like surgical teams. The architect is the surgeon, relative to the patient (the software) and what happens in surgery is up to the architect. Hospital administrators (unless practicing surgeons) are not permitted to meddle around in the operating room. A manager's job is to hire the right people, make sure the pencils are sharpened, the bills get paid, and pay attention when someone says the project is going to be late.
Beware of the manager that says I am not technical anymore. When a manager says that I am not technical anymore, and I stay out of the technical decisions, you should understand this to mean that the person probably used to consider themselves to be very technical. This person is going to ask a lot of questions, ask you to spend a lot of time explaining things, and then make technical decisions. (The project is going to be late. Flip.)
There are a lot of bad managers making technical decisions about things of which they know little. The few good managers have bad bosses. Enough said.
Project managers hire experts and then ignore them all the time. There are actually two phenomena here. The first is that employees are often ignored and consultants brought in and paid more. The second is that these so-called expert consultants are brought in for their advice and ignored, and the manager ends up deciding. In the case of the employee, why are employees hired by these managers who think so little of their ability, and why, when a second hiring takes place, are yet more highly compensated people hired and ignored? The answer is that managers often think they are managers because they are the smartest person, and they wanted to make the decision in the first place. The truth seems to be that managers are often least suited for making technical decisions and were promoted to manager because they were lousy programmers or they didn't offend anyone for the longest period of time. Flip.
Messengers get shot all the time. Purportedly, Bill Gates expects bad news to travel faster than good news. This is a good idea because bad news, like garbage, starts to smell very quickly. Unfortunately, especially in large corporations, messengers get shot so often that the best way to ensure a short career is to carry the baton of bad news too often. It is important to remember that it doesn't matter whose fault it is; messengers get shot. The reason we shoot messengers is because people are basically emotional and bad news is a drag. As emotional animals, we do all kinds of things that are bad for us because of the impact they have on our emotions: drink too much, take drugs, pierce our tongues, shoot messengers, gamble, blame wealthy people for our problems, love people who abuse us, and the list goes on.
A leader has to lead even if no one follows; an effective leader is actually capable of persuading someone to follow (even if the direction is wrong). Any time one mentions Adolf Hitler great personal risk is taken, but Adolf Hitler was an extremely effective leader but a tragic example of one that went in the wrong direction. According to his physician's journal, Der Fuhrer was also addicted to methamphetamine and described his oratorical tirades as a sexual experience. Emotion explains why mass murderers such as Hitler, Stalin, and Pot got away with it. These men appealed to some emotional aspect of their respective audiences (Hitler appealed to impoverished post-war Germans, Stalin by continuing to blame the Romanovs for Russian poverty, and poverty undoubtedly helped Pol Pot along his merry way.) I am suspicious of leaders who lead to someone else's detriment. If someone says hey we have to stop this madman because he is deleting files on the weekend, I say give that leader a listen. If Martin says hey we have to do something about Ralph because Ralph doesn't use the same number of white spaces as I do, I flip the bozo bit on Martin. Martin's priorities are not even in the ballpark.
Every one-on-one meeting after lunch on Friday is bad news for you. Projects are cancelled, people get fired, and everything else that a manager has been convinced that you are doing wrong will be related to you on a Friday. In fact, Friday meetings are so notoriously bad that I am only working half days on Friday from now on. That way, I won't be there to give or receive bad news and Friday can go back to being a joyous occasion.
Every programmer secretly knows that he or she is the best programmer. For this reason, every idea you or I have is most often inferior, occasionally as good, but almost never better than the idea of the programmer we are talking to. Hence, there is a lot of bozo bit flipping. To counter this tendency, I try to use a couple of simple factors when evaluating an idea. Has someone smarter or more successful than I am supported this idea? Is it simple or elegant? Does it make sense? What have the experts written about it lately? Finally, if the idea comes from a bozo, then.... Well, you understand.
Programmers hate to read anybody else's code because it is always crap. If someone volunteers to review your code, there is malice afoot. No one is going to review your code because they like you, like your code, or want to tell you what a good job you are doing. People volunteer to review code because you are their bozo, the bit has been flipped, and they want to tell you why they are the better programmer. Constructive code reviews seldom happen because programmers hate to read someone else's code, and everyone already knows that only bad news is likely to be delivered in a code review.
There really are programmers who are ten times faster than everybody else, but who are they? And, do they get paid ten times as much?
Emerson wrote, "Every man I meet is in some way my superior." Programmers don't read Emerson. The current adage is that all men and women are created equal. This is actually part of a very old proverb, but in our age of political correctness the rest of the proverb has been lopped off. The original proverb is that all men are created equal but by practice grow apart. This means that you and I were equal to Bill Gates and then life happened and now God goes to Gates for a loan. Emerson more than likely knew the original proverb and still believed that each man had at least one quality that was superior to that quality in him. And, yes, you and I are probably superior to even Bill Gates in some way. Unfortunately, if we flip the bozo bit, we are unlikely to ever discover those superior qualities and our lives are diminished.
Programmers are emotionally attached to their code, but don't want you reminding them of it. Pardon me, ladies, but programming is as close to childbirth as a man is likely ever to get (except for passing gallstones). We love our code like a woman loves her own children. This is why we find negligent mothers, and those that besmirch our code so loathsome; it seems unnatural and offensive. Can you imagine if mothers indicated that other people's children were brats as often as programmers think someone else's code is crap? I guarantee you that if each mother had to raise someone else's children as often as one programmer has to read and maintain the code of another, that there would be a whole lot less understanding between mothers. George Carlin wrote a book called Brain Droppings. Lines of code clearly and intimately show how we think. If one is a pedant, it will show up in your code. Blocked comments and in .NET the #region directive will be everywhere. If one is a slob, commented and useless code will be left lying about like dirty underwear and empty beer cans. Lines of code are our brain droppings. The metaphor is accurate. Don't read my crap and I won't read yours.
Many programmers are intellectual bullies and egomaniacs. It is commonly believed that we programmers are mostly geeks that were picked on in grade school and are now paying the rest of the world back for their transgressions. To some extent, I have witnessed and occasionally practiced this bullying behavior, although I am trying to grow out of it. Oddly enough, programmers seem to practice geek-on-geek violence by disparaging each other's code as crap.
Everyone talks about constructive tension, but they still don't want you to disagree with them. Constructive tension is when you and I disagree in a civil manner—like how Congress is supposed to act—and this tension yields the best alternative. Unfortunately, because each programmer is the smartest person in the room arguments ensue, and constructive tension is out the window.
The loss of civility results in destructive tension, name calling, back stabbing, bozo-bit-flipping, and character assassination. As with people, all ideas are not equal. Over time, some ideas are demonstrated to be better than others. This is why I like Brooks' idea of a surgical team. In the operating room, the surgeon leads and everyone else follows. On a software project, this means the technical team lead or the architect has the final say. Consensus is for politicians and car builders, and often results in ungainly compromises and mediocrity. Consider abortion laws: No one is happy with compromise made. Another example is exhibited in some cars that don't make any sense, like that mini-van, truck, SUV sort of car that looks like an ungainly turtle; a committee designed that and every constituency got something in. I will bet that the original Mustang, Corvette, and Thunderbird were all designed by one guy.
The best meals, software, and cars are designed by one guy, and he is usually a tyrant. Single-minded purpose, vision, and great ideas are almost always owned by one person. Greatness requires a tyrant. Ray Kroc made McDonald's franchisees pick up trash within a two-block radius of their stores. A bit of a tyrant. However, McDonald's is one of the biggest success stories of all time. Ray Kroc didn't invent McDonald's, but it was Ray Kroc's vision and tyrannical hold—ingeniously enforced by a creative lease devised by Harry Sonneborne—that permitted Kroc to exercise his tyranny.
The benevolent dictatorship will always yield a better product than a committee. Unfortunately, only one person at a time can be dictator, and in our modern world this seems very un-PC.
Here is a summary of the truisms:
- It is still the Wild, Wild West out here and anything goes.
- When someone says the schedule is going to be missed, they are never lying.
- Change is a constant, but people will seldom thank you for changing their code.
- A lot of bad software is being written by people who don't read.
- People believe too much of what they read.
- Authors are not smarter than the rest of us; they just read more.
- Managers should not make technical decisions, but do.
- If a manager says I am not technical, be prepared to spend a lot of time explaining things to them so they can make decisions they shouldn't be making.
- Managers hire experts and ignore them all the time.
- Messengers get shot more often than not.
- Leaders have to lead; sometimes you will look behind you and find that someone is actually following.
- You are the best programmer.
- Programmers hate to read another programmer's code; if they volunteer to review your code, it is not to do you a favor.
- There really are programmers ten times faster than everyone else.
- Every man is in some way my superior, as long as he doesn't keep reminding me.
- Programmers are emotionally attached to their code, but never say this out loud.
- Many programmers are intellectual bullies and egomaniacs.
- Everyone talks about constructive tension but doesn't want you disagreeing with them.
- Benevolent dictators build the best software.
- Decisions are, more often than not, emotional.
- The mean time between the time you start speaking and someone flipping the bozo bit on you is ten seconds.
Software Survival Rules
To survive the hostile environment in which we work, I have compiled a few simple rules here. (I only mention a few because I have to save some for the made-for-TV movie.) You can avoid having someone flip the bozo bit on you by following a few simple rules:
- Don't say anything.
- Begin your statements with yeah, no a lot so that it sounds like you are being agreeable even when you disagree.
- Deliver the no at the end of the sentence. This is a technique that Jerry Hirschberg says the Japanese practice to great effect.
- Act like you are thinking about someone's bad idea before telling him or her it is crap. The crappier the idea, the longer you should wait before responding; hopefully, they will discover the inferior quality of the idea before you have to say anything.
- If you need a friend, get a dog.
Do I follow any of this gutless advice? Absolutely not! I can safely say that I don't listen enough, say no right off the bat, call crap crap, express exactly just how bad I think an idea is, and own a dog but try to make friends as best I can. That's why I am encouraging you not to do any of those things. That's why I freelance. Open, frank honesty and having an opinion will result in your being asked to drink hemlock. Don't do it.
On Being Savvy
Software development is a political enterprise. Writing code is easy. Building consensus is hard because there are a lot of bad ideas that are loved by their authors.
A friend of mine kept reminding me to be savvy, but savvy sounds too much like trying to survive politically. In reality, what I prefer to do is work a lot of places, stealing the best ideas from each place, and hanging out long enough to do the fun part, leaving the routine tasks and committees for someone else. It is a bit selfish, but I recognized a long time ago that I just wasn't savvy enough to climb through the shark-infested political waters of a big corporation and consequently would never thrive. And, if I did get comfortable, I figured I'd get down-sized or right-sized just about the time I was too old, too tired, or too lazy to be marketable in this rapidly changing technological environment.
The Perfect Software Company
There is a perfect software company. It is where I work. The coffee is excellent. The chairs are comfortable, the computers are wicked fast, and we take a lot of video game breaks. The humidor is well-stocked with Cohibas, there is a killer library, and naps are encouraged. The furnishings are handsome, and the décor is pleasant. There are no cubes. The Managers sign checks and buy whatever software, computers, books, gadgets, and video games are desired. Schedules are not set until requirements are defined, software release dates are only announced after the features are done and rock solid. There are no suits. No ties. No cubes, and no timesheets. The work hours are very flexible but long, but I show up because I have more fun there than anywhere else.
Many times, decisions seem to be made by consensus and offenses. By this, I mean people agree to things they don't believe in to avoid conflict and take offense over the smallest details (for example, someone tells them their code is crap). To me, politicking is the hardest part of software development. Comparatively, the technology is easy.
Many best practices are documented by people who went before us. These best practices are ignored more often than not because there is no standards body. It is like I imagine the Wild, Wild, West used to be, anything goes. Ultimately, software engineers will have to be licensed, but I hope to be retired before that happens.
About the Author
Paul Kimmel has written several books on Visual Basic, including the recently released Visual Basic .NET Power Coding from Addison Wesley. Pick up your copy at www.amazon.com. Paul is also available for short- and long-term consulting, public speaking, and .NET training. You may contact him at email@example.com.
# # #