Why is AI so difficult in 2015?

For my 16 year old son Alex, who asked about bots that are able to think,

This is based on a scientific publication "Symbol Emergence in Robotics: A Survey". This article deals with the ability for a computer to think, act and communicate as a human. We shall call this a bot going forward.

Since a bot that I can't speak to is a dumb bot, this bot has to have some understanding of language. 

They propose that the bot has some sort of internal "symbol" recognition system.

What's a symbol, you ask? A symbol is a word that represents something (a single thing) in the real world. IE: Dog, Cat, People, Sherry, Jump, Run, Etc...

We humans are very good at stringing these symbols together to mean something larger. This is called emergence. Simple put emergence is when simple rules when followed create a idea/action/object that is larger than itself. Be careful as this is VERY different from traditional logical programming. But we can dive into that at a different time.

So, if humans are good at stringing symbols together, our intelligent bot better be too. However there are three almost universally accepted challenges with this system.

• Some symbols just don't mean anything. This is called grounded symbol problem.
• A symbol means something but it can't be explained by a collection of simpler symbols. This is called the dynamic symbol problem.
• A symbol itself and the way people "interpret" the symbol are different. Usually due to some social aspect. This is called the social symbol problem.

Looking at it a little deeper...

Grounded Symbol Problem

"Feeling", that is a word or a symbol. But what is it's connection to the real world. There is nothing you or I can touch/see/taste/hear that defines the word "feeling". It is an internal symbol that is exclusive to the human brain.

Now, this leads down a very slippery path of what is the internal representation of the word feelings in our brain. Unfortunately, that is too large a subject for me to cover. (Also, I don't think I could put it in layman's terms.)

However, just note that there are words and symbols that have this issue. And symbols that have this issue are extremely difficult to fit into our intelligent bot's brain. (more reading: https://en.wikipedia.org/wiki/Symbol_grounding_problem)

Dynamic Symbol Problem

Let's do a simple walk through of the word "Dog", it's symbols might be "animal, mammal, canine". We step deeper, what's an "mammal", "organism", what's an "organism", "cells". What is a cell?

See how we just kept drilling down but we never actually arrived a meaning for the word "Dog".

This is the issue with dynamic symbols.

FYI: If you google Dynamic Symbol Problem you won't find much as I made it up. It's a subset of the Grounded Symbol Problem.

Social Symbol Problem

"Screaming"

The girl was screaming in terror.

The car was screaming down the track.

See how the word screaming means two different things depending on the social context?

This is a difficult to capture in a symbol collection system.

Ok, so now you know what the issues are how do we proceed to solve?

First, it's been proposed that we separate our definition of symbols into two families.

c-symbols, are symbols that are related to computer science (Think: Things with very exact definition)

m-symbols, are symbols that have meanings (Think: Feelings, senses and ideas)

Now, forget about c-symbols, I think us programmers have that covered.

m-symbols are the hard part. Let's say we look at these symbols and break that into 3 parts

The Sign - This is the word. That actual physical characters of the symbol. The symbol Dog is made up of 3 characters. D, O, and G. That is it's sign.

The Object - All symbols have to refer to something. Dog refers to a four legged animal, at a basic level we connect this symbol with our mental image of a dog.

The Interpretant - Now this is where things get odd. Ummm, I have to use some science lingo. Human have 5 senses (see/touch/hear/feel/taste). ESP might be a sixth but I digress. When we use one of our senses we call that an input to the brain. So humans have 5 inputs. With this we are aware of our environment. Ok, lecture done.

Back to the Interpretant, this is when the remembered input is reconciled with the sign. In other words we connect the sign with some remembered sense we got from our environment. So the three letter word(symbol) dog, triggers us to remember a bark, growl, petting, dog walk, etc that makes us "know" what a dog is.

This translation is called Interpretant (in this paper). I hope I got that right...I'm not sure that I did.

Interesting Distraction: This is why reading a story can be so powerful to us. As we read these senses are triggered creating an internal feeling of what is going on. The more closely we can relate our internal translations to the words we read the more powerful the story becomes. I think this is why we love some books and just can’t seem to get into others. Some stories just don’t trigger those internal translations.

Back to the matter at hand, we each have a different set of senses that we recall this leads each of us to have a different view of what we hear/read/see. I think the technical term for this is Umwelt.

Reality encapsulate all of our translations of every symbol we hear or express. That is a deep thought that should be pondered…deeply.

So let’s bring it all together, we have these symbols, which are made up of signs, objects and interpretations. There are rules for how these symbols can be shared with others, this is called a language. This language emerges from the simple actions of combining symbols together. This is called the Emergent Symbol System.

The How

The first thing the bot must do is collect a set of inputs that it can identify as a “thing”. It might be as simple as an image of a black square versus a white square. The inputs of the two differ enough for it to categorize these two objects as two different ‘things’.

Notice, it has not assigned a symbol to these two things, rather it has an internal mechanism to say this black square falls  into this category and this white square falls into this other category.

This is a complex example though as the input to a bot is all digital, so it really is just categorizing its digital input (senses).

We have solved this problem already there are many algorithms (programs) that are able to classify input. Search for pattern recognition for examples.

Let’s call this categorization of the inputs, patterns.

The next step is to connect these patterns with a sign. Remember a sign is the physical representation of a symbol. “Black Square” = pattern 1, “White Square” = pattern 2.

There have been a few attempts to create these connections with written words, LCore by Iwahashi is an example. However I don’t think that anyone has successfully conquered this problem.

If we solve this problem, the next issue is how to create a collection of symbols that have some type of actual meaning to another person. The technical word for this is “Double Articulation”.

From a high level perspective this sums up the difficultly of creating a bot that can think/act/communicate as a human. That being said there are a few other considerations, I’ll just list them to be brief.

Time – The ability to establish time sequence events and how they relate to our symbols and actions. This has been explored but is presenting issues in the field of A.I.

Mutual Belief System – There is a set of ideas that we all share. Typically from a common experience that we all have. “This coffee is cold”, we can easily determine that this means our coffee is not hot enough. But humans will also perceive that this means we will be getting/needing a different cup of coffee. Because that is what we all do when we drink cold coffee. This is a shared belief or implied understanding.

Active Learning and Active Perception – As children we explore. We actively seek new inputs (senses). We do this as children but our need to do this lessons as we grow older. Programmatically finding this balance is challenging.

Syntax – There is a “proper” way to group symbols (words) so that they make sense. While the bot may understand that the words “dog bites man” are the correct symbols. It may put together the sentence “man bites dog”. A totally different meaning. This is syntax, this is difficult to solve.

New Words learned:

Semiotics - The study of symbols and how they play together.

Symbol Emergence in Robotics (SER) - The official term for all the above, this is a research topic which seems to be spinning off in Japan.

Epigenetic Robotics - Robotic developement which models how children learn

Umwelt - German for "Self Centered World", a particular organisms view of the world.

Emergent System – A complex system

For the Scientific Community

This is my layman interpretation of the following paper:

Symbol Emergence in Robotics: A Survey

Advanced Robotics Vol. 00, No. 00, January 2015, 1–27

Authors: Tadahiro Taniguchi, Takayuki Nagai, Tomoaki Nakamura, Naoto Iwahashi , Tetsuya Ogata, and Hideki Asoh

Link: http://arxiv.org/pdf/1509.08973.pdf

Link: http://arxiv.org/format/1509.08973

FYI: This took a lot longer to write than I thought. I wish more people would write a laymans explanation of their scientific publications

Startup Weekend - Transformations in Business

I participated in the Charlotte Weekend Startup Event this past weekend. I had a great time, met some awesome people and learned a good deal about starting a business. I suggest it for anyone with a business idea.

That being said, I wanted to talk about Transformations. I was on a team of 7 folks and we spent 3 days working through a business idea. I won't go into the details of the idea as what I want to talk about is the transformations that I saw took place.

I think everyone could learn from the changes that I saw in my co-founders.

Flexibility
Shaun was the originator of the idea. He had been thinking of this business for quite a number of years and had some very strong ideas on what the business was about. While I think his vision is great, the scale of it was daunting. To be honest, I watched as for 54 hours 5 strangers poked and prodded at his "baby", criticizing and making "suggestions". He took it like a trooper!

The transformation came about half way through, he pulled the team together and said

"Look, I want this to be OUR business, we all have good ideas and I think the business will be better due to our collaboration". (I paraphrase)

From my stand-point this was an amazing way to engage your team and give them power to run with your idea.

Adaptation
Cris was our finance guy, I am not a finance guy. My eyes glaze over when you start talking assets, debits, bips and percentage points. (Not really, but you get the idea).

Cris could TALK the numbers! Within one hour of business planning, he was asking and pointing out concerns and benefits from a strictly finance perspective.

However, we were too early in the process, the value proposition hadn't been flushed out so conversations around revenue were pre-mature.

He must have realized that at some point. Because I noticed that suddenly he wasn't talking "numbers" but was talking general financing approaches and tactics.

I knew a transformation had taken place when I heard him mutter to himself "That's too much detail, that's too technical, how do I say it easier".

This was a transformation that I think we could learn from...bring our knowledge to bear at an appropriate level for the task at hand.

Filling the Gap
Dan was our "intern". He's a student at Davidson college and I think he came just out of pure interested and thought he probably wasn't going to have much to contribute, considering he was in a room of seasoned professionals.

He had come to learn as much as he could from the process and soak it in. (I apologize Dan, if that was not your first intentions, however that was my perception.)

At some point in the game he realized that he could contribute. He could organize the presentation.

See we had a lot of great ideas, our plans were coming together, but none us where slide show jockeys, or even that creative.

Dan saw a gap and filled it.

What was special was the smoothness with which this happened, we all just kinda turned around and there he was with the framework for a smooth "show", he just need the details.

I think his transformation was all about stepping up and filling the gap. There always is one, there is always "something" that needs to be done. You may not be the best for the job, but step up and do the best YOU can at it.

Learning to Sacrifice
Rama (short for a long Indian name that I can't remember, I apologize Rama)., he was our initial developer. Rama's transformation was "willing sacrifice".

It's an odd transformation to think about. A few hours into the project we realized we wanted some "demo" product to show, a simple website.

Rama was all over it. He jumped on Wix and began banging out some code.

Unfortunately, later in the game we decided to go with a mobile app. If I was in Rama's shoes, I'd have been a little pissed. All that hard work for nothing. However he took it like a champ.

He understood, and made some pieces of his work could be used in the mobile app, and he showed no remorse about us throwing his hard work away.

Sometimes in business, you start down one path and you find you have to back-track and take another. That's the way it is, and because of that hard work will be wasted. The best attitude to take is to salvage as much as you can and move forward.

Becoming the Glue
Maria was our glue. She held the team together. Typically you might call this a project manager. I don't think she knew that was going to be her role when she joined the team.

Her's was not a transformation but rather a catalyst for the transformations in the rest of the team.

I watched as she bounced from teammate to teammate and opened lines of communication, pointed out where we were not aligned and when an important concept was being missed.

All transformations need a catalyst to spark change, in your business you will have to be that spark or find people to give your business that spark. It is hard to see the transformation that need to be made from the inside of our problems. 

Transforming into a Leader
Carlos was our leader. He would NOT like me saying this. Other team members may lay claim to the title. But I watched the transformation. Carlos has a started a business, he had run large times, he was knowledgeable about management. Carlos was NOT there to manage. I think he joined the team because for once he wanted to sit back and let someone else run the show and just "help out" someone.

For the first 75% of our trip, we didn't have a navigator.

The business idea was too large

We were all trying to be "nice".

We were trying to incorporate all ideas and push nothing off the table.

It was GREAT...but we were getting no-where fast.

All this time Carlos sits quietly, nods his head, smiles, works a bit hear or there and generally goes with the flow.

12 hours to presentation time and we have nothing to show...we all go outside to talk about whey we were not getting anywhere. There's a 20 min debate...in circles. Frustration is starting to show.

Then Carlos looks up from his laptop and speaks.

"Frankly, I think we lack from leadership. We need some clear direction, even if it is wrong. We need to have some decisions made". (I paraphrase).

It was just a sentence or two... not more then 20 words. Said into the exact moment of silence when we all were thinking it.

Problem clearly stated, solution proposed and feedback requested. All said in a graceful but commanding way.

"Frankly", I was impressed.

For the reminder of our planning, I think he offered a maximum of 3 full suggestions. He didn't even go on stage with us, or come up after the presentation, or even mingle after the show. But he offered guidance and a beacon for a very large idea.

As for myself...

I'm not sure how I contributed or how I transformed. BUT I did learn a ton. And for me that was why I was there.


I thank all my teammates. I learned from all of you. I enjoyed my time with you. And find inspiration in what all of you did. Thank you.

To My Future Self

I was listening to The Tim Ferris Show, a set of podcast interviews with interesting folks. He was talking with Peter Diamandis,  the guy that ran the first Xprize.

He made three very interesting statement.And I paraphrase...

First: Think about what you enjoyed when you were a kid. What did you love to do and were passionate about it. Find a way to do that today. This will be your passion.

Second: What can you build that will impact a billion people?

Third: What can you do that will be remembered 200 years from now? 

The focus is to think BIG. Not incremental, not next new thing, but different...dramatically different.

You know, I love what I do. I'm very good at what I do. But I feel like my talent is wasted. I can be building something that will change the world. I have it in me.

My most compelling excuse...I get paid to well to stop doing what I do now and start doing what I was meant to do.

I would say it is sad, but I in actuality it's just reality.

But I want to say to my future self, think on this, perhaps there is a way that you can start small on this world changing idea.

Internet Background Noise, WTF?

Stumbled across and thought it was interesting.

From Wikipedia: https://en.wikipedia.org/wiki/Internet_background_noise

Internet background noise (IBN, also known as Internet background radiation) consists of data packets on the Internet which are addressed to IP addresses or ports where there is no network device set up to receive them. These packets often contain unsolicited commercial or network control messages, or are the result of port scans and worm activities. The Conficker worm in particular is responsible for a large amount of background noise generated by viruses looking for new victims. In addition to malicious activities, misconfigured hardware and leaks from private networks are also sources of background noise.^[1] For example, some DSL modems have a hard-coded IP address to look up the correct time.

As of November 2010, it is estimated that 5.5 gigabits of background noise is generated every second.^[2] It is also thought that a modem user loses about 20 bits per second of their bandwidth to unsolicited traffic.^[3] Over the past decade, the amount of background noise for a section of the IPv4 address block that contains 17 million address, has increased from 1 to 50 Mbit/s. The newer IPv6 protocol, which has a much larger address space, will make it more difficult for viruses to scan ports and also limit the impact of misconfigured equipment.^[2]

Internet background noise has been used to detect significant changes in Internet traffic and connectivity during the 2011 political unrest from IP address blocks that were geolocated to Libya.^[4]

Backscatter is a term coined by Vern Paxson to describe Internet background noise resulting from a DDoS attack using multiple spoofed addresses.^[5] This backscatter noise is used by network telescopes to indirectly observe large scale attacks in real time.