Saturday, May 21, 2011

Thoughts on early computing history, addendum

1928 - The Entscheidungsproblem decision problem was proposed by David Hilbert
1936 - Church publishes "An Unsolvable Problem of Elementary Number Theory", Church's Thesis [1]. It is a paper on untyped lambda calculus. American Journal of Mathematics, Volume 58, No. 2. (Apr., 1936)
1936 - Alan Turning publishes a paper on an abstract machine , On Computable Numbers, with an Application to the Entscheidungsproblem' Proceedings of the London Mathematical Society, Series 2, 42 (1936-37). He proposed the concept of the stored-program.
1937+ - John von Neumann gains knowledge from Alan Turing's papers but Turing was not directly related to the development of ENIAC.
1943 - 1946 - Creation of ENIAC (Electronic Numerical Integrator And Computer)
1944 - John von Neumann became involved with ENIAC
1949-1960 - Early stored computers were created, some of the based on von Neumann architecture.
1952 - The IBM 701 was announced. The 701 was a stored-program computer.
1954 - IBM 704 introduced
1954 - FORTRAN and LISP developed for the IBM 704
1970 - Forth developed. Forth inspired the Java virtual machine.
1973 - C programming language appeared.

Thoughts on early computing history



When you look back at the major milestones in computing history, we moved quickly. We went from abstract models of computing to stored-program computers in a
decade or less. It was truly amazing.

1903 - Alonzo Church was born in Washington, D.C. (USA)
1928 - The Entscheidungsproblem decision problem was proposed by David Hilbert
1936 - Church publishes "An Unsolvable Problem of Elementary Number Theory", Church's Thesis [1]. It is a paper on untyped lambda calculus. American Journal of Mathematics, Volume 58, No. 2. (Apr., 1936)
1936 - Alan Turning publishes a paper on an abstract machine , On Computable Numbers, with an Application to the Entscheidungsproblem' Proceedings of the London Mathematical Society, Series 2, 42 (1936-37). He proposed the concept of the stored-program.
1936 - 1938 - Alan Turing studies under Alonzo Church
1937 - John von Neumann recommends Alan Turing for Fellowship at Princeton.
1938 - Alan Turing receives Ph.D from Princeton
1946 - Alan Turing presents a paper on the stored-program computer (Automatic Computing Engine).
1937+ - John von Neumann gains knowledge from Alan Turing's papers but Turing was not directly related to the development of ENIAC.
1943 - 1946 - Creation of ENIAC (Electronic Numerical Integrator And Computer). Note: ENIAC was not a stored-program computer.
1944 - John von Neumann became involved with ENIAC
1945 - John von Neumann publishes paper on Electronic Discreet Variable Computer (EDVAC)
1948 - Manchester Mark I developed at Manchester University, first stored-program computer
1949-1960 - Early stored computers were created, some of the based on von Neumann architecture.
1938 - Donald Knuth was born
1957 - Donald Knuth had access to a computer. "I saw my first computer in 1957, which is pretty late in the history game as far as computers are concerned. There were about 2000 programmers in the entire world"
1963 - Donald Knuth began work on the Art of Computer Programming.
1973 - C programming language appeared.

Note: I presented milestones but some of these events were not directly related.

Image of Alan Turing: "It is believed that the use of this image may qualify as fair use under United States copyright law. Other use of this image, on Wikipedia or elsewhere, may be copyright infringement"

[1] "INTRODUCTION Alonzo Church: Life and Work"

--- Berlin Brown

Saturday, May 7, 2011

Thoughts on Artificial Life, AI and AI reboot

"Unthinking Machines Artificial intelligence needs a reboot, say experts."

There are some issues with a top-down approach to automatic artificial complex behavior. The problem with modeling the brain or the brain's neural network is that you are just looking at the end result of millions of years of evolution. We should understand how the human brain became relevant and came to be and then we will find that other animals also have brains and exhibit complex behaviors. Simple animals have smaller brains but we can look how those systems evolved over time. You could go that route, completely model and understand the brain but you will still end up with issues. You will have a broken, less than accurate copy of the brain but then you still are missing other components of the human body. The heart, the nervous system, the lungs, millions of years of evolution. Scientists look at the brain and say, "Hey, that is pretty cool, let's model that". I say, "Hey the earth's biosphere is pretty cool. How did I and the rest of the other intelligent animals get there, let's model that". They are looking at intelligence. But what is intelligence? Why are humans more intelligent than monkeys? Or Crows? Or Dolphins? In reality, they aren't THAT much more intelligent. And even if humans are a lot more intelligent, a lot of other animals have the same hardware. So if we understand the system that created...animals and their hardware, I think that would be more interesting than look at just one animal "brain" and trying to copy that. What parts do you model/copy? No matter how accurate you model the brain, scientists will always play catch up trying to understand the interesting parts of the human brain. And then after 20 years of copying the brain's functionality, we still may have to copy other aspects of the human body that give the brain life.

We need a true bottom up approach that looks at biologically inspired entities if we truly want to understand emergent phenomena. Examine the the microbiology level and chemical reactions and move up. A truly bottom-up approach that looks at the biology of basic organisms and models basic organisms, starting from bacteria to cells is the way to go. And of top of the biology, I would look at inorganic matter and how that relates to organic matter. And then I would look at the evolution of these biologically inspired systems. You could play experiments, where did organic matter come from on earth? We should understand DNA, RNA, mRNA, cells, single celled organisms, water, on and on. Even those basic components are kind of interesting. Combine DNA, cells and other matter together and you have a complex entity. Understanding the reasons for those components and how they interact is the way to go. Evolve systems that generate those interactions. I would model simple creatures, evolve those creatures and then create an environment for those creatures to exist, have them interact and then evolve a system that has some form of brain..or multiple brains.

Even the term "artificial intelligence" leads people in the wrong direction and needs a reboot. I like "autonomous artificial adaptability". We want creatures that adapt to the world around them and do so at their own direction. The concept of intelligence implies "human brain intelligence". Humans are more intelligent than pigs. But pigs are WAY more intelligent than trees. That leap in adaptability is interesting and worth looking at. Think about it this way, the unintelligent parts of the human body are fascinating. And who is to say that there are creatures in the universe that are infinitely more intelligent than humans? We have one brain, is it possible that a creature could have a million brains that all operate independently of one another. We only use a small capacity of our brains. Is it possible a creature could use 100% of their brain capacity. Bacteria and plant life are not normally considered intelligent but they do adapt to the earth's changes conditions. Human beings are far more interesting than bacteria but that doesn't necessarily mean that replicating human brain intelligence doesn't have to be the ultimate goal for strong AI.

At the heart of strong AI will be computational biology, whether the artificially evolved creature has something similar to a brain or neuron cells is irrelevant to the problem, you can still create adaptable, seemingly intelligent creatures with artificial biology through a controlled artificial environment. That is why I think the AI field is missing, the focus has always been the brain. Even if you create an artificial brain that is similar to the human brain, you have the problem of replicating the signal processing mechanisms of the eyes and ears. You have an issue with creating pain receptors and other bits of information that are fed into the brain. Even if you can feed the right bits to the brain, you will hit the next philosophical question, what is this autonomous creature supposed to do? Everything that a person does is ultimately tied to their evolutionary inspired purpose. You eat because you are hungry, the human is hungry. You create societies to make it easier to survive for other humans. All of the adaptability of the human brain and the human are kind of tied to its evolutionary purpose. What will be the goal of this artificial brain? You still have the same problem with a biologically inspired, evolutionary inspired artificial systems but you can control the evolutionary constraints. Maybe the creature doesn't need a brain? But that is OK, it still may have interesting properties that encourage its survival. The computer science AI research community has tunnel vision as it relates to AI, "the human brain, the brain, the brain". If you stop and think, "what is the brain? what is a human?". We are really a collection of cells and bacteria, all wrapped in a nice protective package. Most of the individual cells in the human body are interesting on their own, the brain cells are not that much more interesting than the skin cells or blood cells or anything else.

With most software engineering and even most AI research, the developer is required to program the behavior into the system. The developer is careful to program a response to all known inputs. Even if you model the brain and create a close enough model of the brain, the puppet master will still have a problem of programming and training inputs that only this particular brain can respond to. You have reached the zenith of AI but now you have hit a wall trying to train and feed information to the brain. You are essentially programming the brain with known inputs. With a good biologically inspired model that evolves behavior and operates autonomously and completely independent of the "creator", you don't program any behavior (as much as you can). If you run the system 20, 100 years, we may not know what type of behavior emerges. These systems should have a start button but no kill switch. Killing the system means you start all over and completely new behavior emerges. In theory, The brain model and the bottom-up biological model are similar, you expect emergent behavior. Evolutionary design creates more emergent behavior than starting at the brain and watching what happens next.


\[
\begin{align}
\nabla \times \vec{\mathbf{B}} -\, \frac1c\, \frac{\partial\vec{\mathbf{E}}}
{\partial t} & = \frac{4\pi}{c}\vec{\mathbf{j}} \\
\nabla \cdot \vec{\mathbf{E}} & = 4 \pi \rho \\
\nabla \times \vec{\mathbf{E}}\, +\, \frac1c\, \frac{\partial\vec{\mathbf{B}}}{\partial t} & = \vec{\mathbf{0}} \\
\nabla \cdot \vec{\mathbf{B}} & = 0
\end{align}
\]

Resources

[1] http://en.wikipedia.org/wiki/Avida - Artificial life software platform

[2] http://en.wikipedia.org/wiki/Artificial_life

[3] http://en.wikipedia.org/wiki/Christopher_Langton

[4] googlecode/bottom-alife-demo.zip

-- Ron Paul 2012

Sunday, May 1, 2011

Confirmed Sources, Osama bin Laden is dead

It has just been reported, Osama bin Laden is dead. Now, I can stop looking for him.

"Justice has been done" -- Obama

Unit Testing / Extreme Programming

Unit Testing advocates use language that you hear from religious advocates.

"You can't write software without unit tests". "You must unit test your code".

That sounds a lot like:

"You will go to hell if you sin". "If you don't accept Jesus as your lord and savior, you will go to hell".

Where does that leave us?

You can write software without unit tests. A lot of software has and will be written without unit tests. No human will go to hell for not accepting Jesus as their lord savior.

Same question, where does that leave us?

I may have advocated the use of unit tests. I think the concept is beneficial. BUT there are a couple of issues. Developers not writing unit tests is a big one (I was thinking about creating a spreadsheet, software from github and sourceforge, projects that have unit tests and those that don't). Why don't developers write unit tests? Because they don't have to. That is the main reason. You need a compiler to compile code for languages that need to be compiled. You need some runtime. You need to your code. But you aren't required to write unit tests in order to execute your code.

So, maybe we need some form of system that checks for unit tests before allowing your code to run. In my mind, I kind of feel that unit tests are an extra compilation step or runtime step. Maybe if that sort of system was in place, we might see developers wanting to write more unit tests. I also want more information from the unit tests. It is great to know if the test passed or failed but I would like to see similar possible inputs fed into the unit testing systems and the output (possibly with a visual graph). QuickCheck and ScalaCheck are example tools that automate unit testing.

And like with most with religious practices, sometimes the underlying message is important but the message is delivered improperly.

If I say: "Write unit tests or else". If developers don't see a need or want to write unit tests, they won't. So telling developers to write unit tests doesn't really hold weight.

Now, I would like to hear, "Write unit tests in order to isolate this piece of code and test reasonable inputs and check the output of the tests. This is a core class in our library and we must validate the possible inputs and expected outputs. It will be interesting to look at this particular module, visualize the possible inputs from the calling client code and see what happens. Plus, isolating this particular component and testing it takes less time than running the entire system as a whole". That blurb won't make for a very good headline at an extreme programming conference but I think it is gets the point across, more so than, "Write unit tests or else". But my overly verbose blurb on unit testing still doesn't force developers to write unit tests. Even those that are well versed on testing will not write unit tests when under some critical deadline.

In Summary

I think the concept behind unit tests is beneficial but I still feel that implementation systems behind unit testing are still lacking. This is one possible reason that developers don't always write them (jUnit is one example of a system that lacks).

Response to critics of this post?

I have noticed if you step on any person's religion, critics tend to come at you from all directions. I don't mean to step on the unit testing religion. I am not advocating to stop writing unit tests. But I do want to point out that software is written without them. Maybe if we create a platform that forces the developer to write unit tests as part of some compilation or build step, then developers will start thinking about the practice.

How do you know that developers aren't writing unit tests? If it is possible not to write unit tests, then it is possible that developers won't write them. Also, I tend to want to include the edge cases of software development when talking about unit tests. Are people writing unit tests for embedded software development? Linux device driver development? Any Linux software development? Complex J2EE server environments?