Technical Debt

"Myrtle the Turtle told me it would take him 3 months to make my app, but Speedy Slim told me he could do it in 3 weeks? WTF?" my friend complains to me. I wince. This is not going to be easy to explain.

As with most things in life, both Speedy and Turtle are right but for different reasons. At the root of it is the Technical Debt that my friend is about to signup for.

Technical Debt is the "cost" of scaling, debugging and enhancing your application over its lifetime. Minimizing these cost up-front takes time. Sometimes a lot of time. Myrtle the Turtle is all about minimizing these costs.

An analogy that most laymen will understand

Over the entire life of your application there is a cost and the very first item that effects that cost is Speedy and Turtle. You can get this application built on cash, and pay upfront, and pay less. Or you can gt this application on credit and pay less up-front. The choice is really yours to make.

With a cash buy, you are paying every thing upfront. It's going to "hurt" more but once you get over that first build its relatively down hill from there, you will find that scaling, bugs and enhancements have a minimal cost.

With credit, your paying a little up-front but it hurts less...initially. As time passes on you'll be making continue payments to the debt of this purchase. You'll find that scaling, bugs and enhancements will cost you a fair amount of money. Inevidably, you'll pay more this way then you would from a build by Turtle. This is because of the interest on your technical debt.

What's the cost of using Speedy and receiving a "quicker" delivery?

• Speedy is going to deliver exactly what you specify
  
• Enhancements are going to take longer because Speedy will have to "re-work" a lot of code.
• When more users start to use the app or more data gets into the system, the app will start to slow down and eventually will die
• Speedy's going to get tired of working on the bad code in the app.
• When you ask a "new" programmer to pickup the code up, you'll find that they tend to have to re-write the entire thing from scratch.

All of these are reasons that are not apparent in the initiall 3 to 6 months of the applications lifetime. They only become apparent down the road.

I want to point out:

1. Even the most well built and thoughout applicaiton build can fall to the points above.
2. There are developers out there who take 6 months but still deliver the equivalent of Speedy's work.
3. It's not an easy thing to build a scalable, bug free, flexible application.
4. Scaling, Bugs and Enhancements are NEVER free in any app. It's about minimizing the cost on these items.
   

When do I personally take Speedy up on his offer?

I'll take Speedy over Turtle when I know that what I want is a proto-type or proof of concept. I'll take Speedy when I "have" to have something done by a given date, and there is hard dollars tied to that date. I'll take Speedy when I don't know exactly what I want built yet, but I have a vague idea.

Everytime I take Speedy, I'll write down Turtle's name because I'm going to need him to build the real deal when the short need is done.

Are you a Speedy Slim or a Turtle Tim? Can a developer be both "on demand"? Which developer bills higher?

More about Jarvis the Chat Bot

I mentioned that I created a chat bot named Jarvis. Since then I've been asked a few times what goes into creating Jarvis. I thought I'd take the time and explain what is going on behind the scenes.

First off Jarvis sits on top of database with the following tables.

• word - A library of known words he found online
• keywords - collections of words that Jarvis thought went together
• responses - Response that can be given to a user

There are other tables, but that is the core ones.

Now I loaded Jarvis with the library that ELIZA the original chat bot used. That system had a number of keywords and responses to those key words. For example: If the user had typed "Hi", there were 3 possible responses. "How are you?", "Hello", "Welcome".

Jarvis would randomly pick one of those.

I also hand entered in some other "choice" keywords and phrases.

The next step was to have him learn from input. So when ever Jarvis puts out a response that includes a question (question mark), he would wait for the response and tag it as a possible "answer" to that question. That way when another user asked the question of Jarvis this was in his possible responses.

So for example, Jarvis might start off with a memory like the following:

Key: How are you?
Possible Responses: "Good", "I'm sad" or "I'm happy".

Let's say he had the following conversation with user 1.

Jarvis: Hi
User 1: Hello Jarvis
Jarvis: How are you?
User 1: I'm good, I had bacon this morning.

Now Jarvis' memory looks like the following:

Key: How are you?
Possible Responses: "Good", "I'm sad" or "I'm happy" or "I'm good, I had bacon this morning"

So when user 3 comes along the conversation looks like this...

Jarvis: Hello
User 3: Hi Jarvis, How are you?
Jarvis: I'm good, I had bacon this morning
User 3: That sounds good
Jarvis: how are you?
User 3: Hungry

Jarvis used the first users answers and he also learned a new answer to the question "How are you?"

Key: How are you?
Possible Responses: "Good", "I'm sad" or "I'm happy" or "I'm good, I had bacon this morning", "Hungry"

Curiosity
At first I built him so that any response that was a question was given priority. But that lead to Jarvis asking to many questions. So Jarvis now has a curiosity factor. The more curious he is, the more likey that he will give a response that is in the form of a question. This sponsors more responses for him.

You can increase or decrease curiosity by typing "moreCurious" or "lessCurious" into the text box.

Peeking into Jarvis Brain
There is a checkbox "Peek into Brain". Clicking this will let you see into Jarvis process. It shows you the following:

• Words gathered from your input
• Last 5 words learned (popularity is in parens)
• The Keywords or phrases that were triggered based on your words
• Selected/Best Keyword is highlights in bold
  
• The Responses triggered from the best keyword
• The last 5 learned keyword phrases
  
• The last 5 learned responses

Scoreing
Jarvis tracks each chat session and gives it a score. Keep in mind each possible response is scored. Anytime Jarvis uses an response with a score higher than 0, he adds one to the session score.

This should lead to interesting sessions having a higher score, but I have not thoroughly tested that.

Jarvis can be found at http://www.mobeamer.com/lab/jarvis

Randomness

There are two places where Jarvis uses randomness.

First: Sometimes after finding all the keyphrases from the users input. Multiple keypharases have the same score. In this case, Jarvis will randomly select one of them.

Second: Sometimes after finding all responses from a keyphrase, mulitple responses have the same score. In this case, Jarvis will randomly select one.

The random function uses php's in-built rand() function. Which is not 100% random but close enough for me.

Jarvis - A chatterbot

If you didn't know AI (artificial intelligence) has become a bit of a hobby of mine. In an effort to explore the domain, I thought that I would throw together a little chat bot.

He's based off of the original chat bot, ELIZA.

I have/had a lot of thoughts of the direction I could take this in, but I wanted to get a version 1.0 all wrapped up and released before taking on other challenges.

You can chat with Jarvis at: www.mobeamer.com/lab/jarvis/

Some fun facts:

• Jarvis was named after the AI in Iron Man
• My 11 year old son, taught Jarvis a few key sayings
• Sometimes Jarvis is incredibly stupid
• Jarvis has a fully built out relationship model for words, key phrases and responses.
• Jarvis can learn from chatting with other users

Some thoughts on what to implement next:

• Allow Jarvis to "read" or learn from websites (Wikipedia)
• Allow Jarvis to formulate higher "concepts" where responses are all connected via a single word or key phrase.
  
• Allows users to chat with other users and Jarvis at the same time and ask them to spot Jarvis in the crowd
• Allow Jarvis to get happier the longer someone chats with him
• Add some type of visual representation of Jarvis
  

Charlotte BDPA - 2011 Kickoff

We had the Charlotte BDPA 2011 kickoff meeting last night. I think it went very well. A little recap of what I remember off the top of my head.:

• Diondre, with IBN Consulting, is interested in teaching folks a number of IT topics, his business has an office downtown and has been training folks for a few months now. Definitely want to keep in touch with this guy.

• Sheila (spl?) - The Urban League has a new program, they are giving CISCO certifications to 11th and 12th graders. The programs are held at Phillip O'Berry. Sounds like there are some nice perks with the program as well. (Laptop, software and such). She needs IT professionals to come by and speak about the career. I told her to count me in. If you know of anyone or are interested, reach out to me.

• 2 BDPA members recently moved to new roles within thier current companies. You know I love to hear news like this.

• The Charlotte BDPA 2011 calendar is full of education and training opportunities for adult professionals. A bit of a change in focus for our chapter but a refreshing one.

• Charles - With APEX has a few development, PM and BA job opportunities for anyone who is looking.
  

All in all, it was a good meeting and better networking. I'm looking forward to our next meeting in Feb.

Hope you are starting the year off right.