Random Musings on AI

I've been reading more on AI (again). It left me pondering...

Einstein had a great quote: 

"Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction."

I agree. Most likely the solution to General Artificial Intelligence is simple and elegant. The current models seem ... complex.

Ray Kurzweil talks about how our intelligence is built from layers of simple mechanics. Each part of the layer works identically, but it gathers info from the prior layer. I like this idea, it sounds "simple". The difficulty is determining what the "rules" are for the smallest element in the layer.

Monica Anderson talks about a non-model based approach. Specifically one in which "intuition" plays the largest role in "getting to" intelligence. She is definitely on to something. Her ideas that the world as we see it is not built on specific models (forumula's) is right on target.

My primary concern with this field is that there is no clear agreed upon definition of when "intelligence" has been reached. Like wise there is no test for intelligence, at least for a digital avatar. We have the Turing Test but that comes up short on a few fronts, in my opinion.

Recall that the goal of General or Strong Artificial Intelligence is to create a machine that can successfully perform any intellectual task that a human being can.

In simple words, it has to be able to learn anything.

So we need simple rules that can enable a system to learn anything.

From this thought my mind jumps through the following sequence of thoughts:

• Humans have many senses, AI would need to have many types of input.
• Wait, an infant, in his mom's belly has very few senses. 
• Rather, he has all senses, but the input coming in is very small.
• AI with multiple inputs should be able to "learn" something" with very few inputs
• As we grow new senses are introduced one at a time. We grow fingers before ears, ears before eyes, and so on.
• Perhaps the method by which the AI can learn, needs to be adaptable to many different types of inputs
• So all inputs regardless of source must resolve to the same "signals"
• I think all the input needs to be in a binary state
• Or all inputs need to be on a sigmoid curve (value between 0 and 1

Can we build rules based on the last three statements? We can and have. This would be the neuron inputs that have been long discussed.

Now I know that I have Sigmoid coming into my input layer...what rules can I build around this?

Another interesting item, at a certain point in our learning, we do not need a lot of exposure to new input to "learn" a new concept. For example, the effort to learn how to add is significantly harder than the effort to learn how to subtract, once we have learned how to add.

Put another way, when placed in a new environment, but the environment is "similar" to something we know, we learn quickly. The more the environment is different the slower we learn.

What this means is that once our AI has learned a specific subject area, learning in that area should be quick. For example, a maze solving AI would solve mazes quickly with few "learning" cycles, but that same AI placed in a Turing test would take "longer" to learn how to interact within that system.

Notes on AI

I was watching/exploring an online AI course over at https://class.coursera.org/neuralnets-2012-00

Here are some notes that I don't want to forget:

Types of Neurons (Single points of contacts in a 'brain')

Linear Neuron - Collects all input values and applies a weight and that is it's output.  (aka Linear Filter)

• y = b + sum(xi + wi)

Binary Threshold Neuron - Collects all input values and when sum hits a "threshold" it outputs a value of 1

•  if b + sum(xi + wi) > 0 output 1 else output 0

Rectified Linear Neuron - Collects all input values and when sum hits a "threshold" it outputs a value progressive value (dependant on input)

• In simpler terms, it works like a Binary Threshold but outputs like a Linear Neuron

Sigmoid Neuron - "Smoothes" the output to something between 0 and 1

• Most commonly used
• Ouput = 1 / (1+ e^-(b + sum(xi + wi)) 
• Leads to smooth derivitives

 Stochastic Binary Neuron - Take a Sigmoid Neuron and randomize if it actually fires are not.

• Follow up: Not sure what randomization is based on...
• Follow up: Why is this usefull
• Poisson Rate for Spikes (huh??)

Perception Based Architecture - Will always find a solution within its Test Cases IF a solution exists within Test Space. Often times a solution does not, due to what measurs(features) are chosen.

• If you choose the right measures(features) then this is a great learning model
• Choosing features is the hardest part though!
• Once features are "chosen" you have limited your learning process
• Do not use this learning model for "multi-layer" networks, it doesn't work

Important Questions to Ask of Your System:

• Will it eventually get to a correct answer?
• How quickly will this happen? (How many evolutions/learnings/weight adjustments)

Bouncing Box:

Some of my students asked me to show them some "game programming" code.

I started off by showing them a simple canvas program, that uses HTML5 and the canvas tag to create a bouncing box.

Take a look at the following code.

<html> <body> <canvas id="mapCanvas" width="300" height="300" style="position:absolute; top:110px; left:0px; width:300px; height:300px; border:5px solid black;"></canvas> <script> var x = 0; var y = 0; var xVel = 3; var yVel = 0; function animate() { var canvas = document.getElementById("mapCanvas"); var context = canvas.getContext("2d"); context.clearRect(0, 0, canvas.width, canvas.height); context.fillStyle = "blue"; context.fillRect(x, 10, 25, 25); x = x + xVel; if(x > 300) { xVel = -3; } if(x < 0) { xVel = 3; } requestAnimFrame(function(){ animate(); }); } window.requestAnimFrame = (function(callback){ return window.requestAnimationFrame || window.webkitRequestAnimationFrame || window.mozRequestAnimationFrame || window.oRequestAnimationFrame || window.msRequestAnimationFrame || function(callback){ window.setTimeout(callback, 1000 / 60); }; })(); animate(); </script> </body> </html>

About BDPA

For those of you interested in knowing more about the BDPA organization, check out this great T.V. interview with its founder.

http://vimeo.com/25390274

Popular Technology TV | 2011 Pilot Episode from bdpatoday on Vimeo.

FYI: The National Convention is coming in August.

---

Free Code

Just a quick post..I'm slowly posting my playground work to my "lab" site.

www.mobeamer.com/lab

Consider it all open source, please provide a link back to this blog or mobeamer.com if you use in a production site.

Free Training - Sign Me Up!

BDPA Charlotte had their monthly meeting in a new location last night. We were at Metrotek which is located across from Microsoft. It was one of the best meetings I've ever attended with our group.

A fellow named Larry Edwards gave a presentation on Project Management and Six Sigma. How they are alike, how they are different and how they be leveraged for one another. I thought it was very informative and a good introduction into those two disciplines.

A lot of great folks turned out, we had folks from Wells, Bank of America, AT&T and a few other companies as well.

And then there was the great give-away. Metrotek gave everyone in attendance access to their Microsoft Project Certification Course free of charge. They are also going to do a drawing for all those that attended and give away additional training. If that is not some great ROI on an hour spent mingling with IT folks, I'm not sure what you are looking for.

For those of you who do not know, Charlotte is currently giving away $2,000,000 in training. If you are NOT employed, you need to contact Metrotek today and ask about their program. The training can be done virtually or self paced. The opportunity is knocking at your door.

Oh, and there was free food. (Quizno's...I love a good sandwich).

If you are in Charlotte, look us up, and join us, http://www.bdpacharlotte.org.

Markus Beamer

Building a Pipeline of American Talent

I was sitting with some folks last week, after watching Wayne Hicks recieve recognition at the Microsoft Charlotte campus. We had a very interesting conversation which struck a cord with me. We were talking about the need to create a pipeline, a steady stream, of high quality minority talent. Specifically for companies looking to hire this talent.

Now, this idea is not all mine, so I certainly won't take credit for it, but I thought I'd discuss it a bit.

The realization was that, maybe, just maybe, the BDPA SITES program is allready setup to deliver just such a pipeline.

Think about it from a hiring companies perspective. They need to find top talent for junior level technical positions. Typically they would service this need by going to College Events and other well known HR practices.

But how do they find those really talented kids. The ones that have a 4.0 but also have real world experience. Those talented "Lebrauns", "Kobes" and "Shaqs" of the IT world.

I know that we (BDPA) know who those students are, I personally can tick off 4 in the last two years. It's those students who have gone through our Youth Technology Camp then gone on to our "High School Computer Academy" and then followed up on our collegiate program the "IT Showcase". It's students who have had in-depth conversations and mentorship by real working IT Professionals, year after year.

So when you find one of these students, you know you have found an individual who:

• Excels at technology
• Understands technology
• Is able to develop and deliver real world solutions
• Is capable of functioning appropriately in a company environment
• Has excelled academically and professionally
  

Put simply, these companies would find that they have a student who is ready to work and able to learn how to perform the job quicker and easier than your normal college graduate.

A reliable, flexible and productive employee is a very solid ROI (Return on Investment). One might even say it is the most import measure for an HR department. BDPA has the ability to offer those candidates on a platter.

As a hiring manager, you want to get your new higher to produce as soon as possible and you want your new hire to be flexible to the nuances of your business. BDPA students deliver on both fronts.

So what does BDPA ask in return, just a few things:

First: Cash. It costs to get the right equipment, to market to the right youths, to give the youths experiences that will prepare them for the work force. It simply takes money.

Second: Internships. You can only get experience by working. I've found that most of our students stand out as interns. Even when our high-school students are interning with college students. We encourage companies to setup high-school and college internships, both paid and unpaid, where our students can learn and contribute.

Third: On the job Advancement. Once on the job, evaluate theirperformance, advance them as needed. They will shine, but you must give them opportunity to do so. Doing so will allow them to find other good employees to join your company.

All in all, it was an interesting conversation. It definitely needs some polish. I'd like to put a deck together illustrating this concept in simpler (fewer) words.

But sometimes it's more important to simple capture the thought.

Please keep in mind, these are my personal thoughts and opinions and not necessarily the opinions of BDPA at a national level.

Thoughs? Comments?

Technical Debt

"Myrtle the Turtle told me it would take him 3 months to make my app, but Speedy Slim told me he could do it in 3 weeks? WTF?" my friend complains to me. I wince. This is not going to be easy to explain.

As with most things in life, both Speedy and Turtle are right but for different reasons. At the root of it is the Technical Debt that my friend is about to signup for.

Technical Debt is the "cost" of scaling, debugging and enhancing your application over its lifetime. Minimizing these cost up-front takes time. Sometimes a lot of time. Myrtle the Turtle is all about minimizing these costs.

An analogy that most laymen will understand

Over the entire life of your application there is a cost and the very first item that effects that cost is Speedy and Turtle. You can get this application built on cash, and pay upfront, and pay less. Or you can gt this application on credit and pay less up-front. The choice is really yours to make.

With a cash buy, you are paying every thing upfront. It's going to "hurt" more but once you get over that first build its relatively down hill from there, you will find that scaling, bugs and enhancements have a minimal cost.

With credit, your paying a little up-front but it hurts less...initially. As time passes on you'll be making continue payments to the debt of this purchase. You'll find that scaling, bugs and enhancements will cost you a fair amount of money. Inevidably, you'll pay more this way then you would from a build by Turtle. This is because of the interest on your technical debt.

What's the cost of using Speedy and receiving a "quicker" delivery?

• Speedy is going to deliver exactly what you specify
  
• Enhancements are going to take longer because Speedy will have to "re-work" a lot of code.
• When more users start to use the app or more data gets into the system, the app will start to slow down and eventually will die
• Speedy's going to get tired of working on the bad code in the app.
• When you ask a "new" programmer to pickup the code up, you'll find that they tend to have to re-write the entire thing from scratch.

All of these are reasons that are not apparent in the initiall 3 to 6 months of the applications lifetime. They only become apparent down the road.

I want to point out:

1. Even the most well built and thoughout applicaiton build can fall to the points above.
2. There are developers out there who take 6 months but still deliver the equivalent of Speedy's work.
3. It's not an easy thing to build a scalable, bug free, flexible application.
4. Scaling, Bugs and Enhancements are NEVER free in any app. It's about minimizing the cost on these items.
   

When do I personally take Speedy up on his offer?

I'll take Speedy over Turtle when I know that what I want is a proto-type or proof of concept. I'll take Speedy when I "have" to have something done by a given date, and there is hard dollars tied to that date. I'll take Speedy when I don't know exactly what I want built yet, but I have a vague idea.

Everytime I take Speedy, I'll write down Turtle's name because I'm going to need him to build the real deal when the short need is done.

Are you a Speedy Slim or a Turtle Tim? Can a developer be both "on demand"? Which developer bills higher?

More about Jarvis the Chat Bot

I mentioned that I created a chat bot named Jarvis. Since then I've been asked a few times what goes into creating Jarvis. I thought I'd take the time and explain what is going on behind the scenes.

First off Jarvis sits on top of database with the following tables.

• word - A library of known words he found online
• keywords - collections of words that Jarvis thought went together
• responses - Response that can be given to a user

There are other tables, but that is the core ones.

Now I loaded Jarvis with the library that ELIZA the original chat bot used. That system had a number of keywords and responses to those key words. For example: If the user had typed "Hi", there were 3 possible responses. "How are you?", "Hello", "Welcome".

Jarvis would randomly pick one of those.

I also hand entered in some other "choice" keywords and phrases.

The next step was to have him learn from input. So when ever Jarvis puts out a response that includes a question (question mark), he would wait for the response and tag it as a possible "answer" to that question. That way when another user asked the question of Jarvis this was in his possible responses.

So for example, Jarvis might start off with a memory like the following:

Key: How are you?
Possible Responses: "Good", "I'm sad" or "I'm happy".

Let's say he had the following conversation with user 1.

Jarvis: Hi
User 1: Hello Jarvis
Jarvis: How are you?
User 1: I'm good, I had bacon this morning.

Now Jarvis' memory looks like the following:

Key: How are you?
Possible Responses: "Good", "I'm sad" or "I'm happy" or "I'm good, I had bacon this morning"

So when user 3 comes along the conversation looks like this...

Jarvis: Hello
User 3: Hi Jarvis, How are you?
Jarvis: I'm good, I had bacon this morning
User 3: That sounds good
Jarvis: how are you?
User 3: Hungry

Jarvis used the first users answers and he also learned a new answer to the question "How are you?"

Key: How are you?
Possible Responses: "Good", "I'm sad" or "I'm happy" or "I'm good, I had bacon this morning", "Hungry"

Curiosity
At first I built him so that any response that was a question was given priority. But that lead to Jarvis asking to many questions. So Jarvis now has a curiosity factor. The more curious he is, the more likey that he will give a response that is in the form of a question. This sponsors more responses for him.

You can increase or decrease curiosity by typing "moreCurious" or "lessCurious" into the text box.

Peeking into Jarvis Brain
There is a checkbox "Peek into Brain". Clicking this will let you see into Jarvis process. It shows you the following:

• Words gathered from your input
• Last 5 words learned (popularity is in parens)
• The Keywords or phrases that were triggered based on your words
• Selected/Best Keyword is highlights in bold
  
• The Responses triggered from the best keyword
• The last 5 learned keyword phrases
  
• The last 5 learned responses

Scoreing
Jarvis tracks each chat session and gives it a score. Keep in mind each possible response is scored. Anytime Jarvis uses an response with a score higher than 0, he adds one to the session score.

This should lead to interesting sessions having a higher score, but I have not thoroughly tested that.

Jarvis can be found at http://www.mobeamer.com/lab/jarvis

Randomness

There are two places where Jarvis uses randomness.

First: Sometimes after finding all the keyphrases from the users input. Multiple keypharases have the same score. In this case, Jarvis will randomly select one of them.

Second: Sometimes after finding all responses from a keyphrase, mulitple responses have the same score. In this case, Jarvis will randomly select one.

The random function uses php's in-built rand() function. Which is not 100% random but close enough for me.

Jarvis - A chatterbot

If you didn't know AI (artificial intelligence) has become a bit of a hobby of mine. In an effort to explore the domain, I thought that I would throw together a little chat bot.

He's based off of the original chat bot, ELIZA.

I have/had a lot of thoughts of the direction I could take this in, but I wanted to get a version 1.0 all wrapped up and released before taking on other challenges.

You can chat with Jarvis at: www.mobeamer.com/lab/jarvis/

Some fun facts:

• Jarvis was named after the AI in Iron Man
• My 11 year old son, taught Jarvis a few key sayings
• Sometimes Jarvis is incredibly stupid
• Jarvis has a fully built out relationship model for words, key phrases and responses.
• Jarvis can learn from chatting with other users

Some thoughts on what to implement next:

• Allow Jarvis to "read" or learn from websites (Wikipedia)
• Allow Jarvis to formulate higher "concepts" where responses are all connected via a single word or key phrase.
  
• Allows users to chat with other users and Jarvis at the same time and ask them to spot Jarvis in the crowd
• Allow Jarvis to get happier the longer someone chats with him
• Add some type of visual representation of Jarvis