The seed of reason

Is there a lowest level of reality?

I want to take my idea of a coding invariance principle further (see my earlier post defining the philosophical principle).

What do we mean when we say "The law of gravity is true"? We could mean that the law of gravity matches all the available data.  We could even require that it matches all true statements about the universe (or does so to within some degree of approximation).  But more might be required.  We might ask that there is no other simpler theory that matches the world as well or better.

What if there are two theories explaining different physics (like quantum mechanics and relativity) and a third simpler theory comes along that explains both areas (the very small and the very large) better than either theory.  This third theory might be simpler than the combination of quantum mechanics and relativity.  But in this case would it make sense to say quantum physics is true or even the best theory for the very small anymore?

Clearly its conceivable that a theory explains a narrow domain well but is superseaded by a theory which is little better on this domain but explains a much wider domain and therefore is a better theory overall.  A theory could also be better at explaining that narrow domain than our original theory in which case you would also say that it was a better theory.

A significant question is then "Is there a best theory to explain any domain of our universe?".  It is consistent that the answer is no and that for any theory there is another theory which is either more elegant (explains more with little additional complexity) or explains the same data with a higher degree of accuracy.

This isn't to say that in such a universe science could not progress and make better predictions over time.  However, it does draw into question whether it is meaningful to talk about a theory giving a truthful description of the universe.

There are various bizarre possibilities in such a universe:
For instance the best available theory may keep changing its opinions on matters such as:

1) Whether the universe had a beginning
2) What exactly happened in the past
3) Whether the universe is infinite
4) The number of dimensions the universe has

A universe that has many partial descriptions matching 1,2,3,4 would be a strange beast indeed but it would certainly be an interesting place to reside!

The data flood

Over the last decade western societies have begun to generate vast quantities of data.  Examples include:

1) Sales statistics
2) Medical records
3) Web traffic data
4) Media usage data (scrobbling, play counts etc.)
5) User generated data (facebook pages, text messages, emails etc.)
6) Online gaming data (second life, online board game archives etc.)

We have also begun to mine this data for useful patterns.

I give two major problems with the way in which we are doing this. 
A) The privacy of this data is often undermined by the careless manner in which it is stored.  Whilst the data has great economic value (mostly to companies and governments but also to the consumer/citizen) it can also be used for new and worrying kinds of abuse.

B) We depend too much on people to analyze this data.  In my opinion the main limiting factor in gaining economic advantage from this data is our ability to process it.

I predict two trends over the next decade with regard to data mining. 
Firstly there will be significant abuses of people's personal data resulting in a backlash against the mass storage of such data.  This will cause more data to be stored in a distributed or properly encrypted form.  Computation on this data may be performed in a distributed manner too with the programs that perform the computations being more likely to be under the control of the users.

Secondly simple artificial intelligence agents (not much more intelligent than insects) will proliferate as the cost of the necessary hardware to run them approaches zero.  These agents will make be called upon to dramatically reduce the burden of decision making for users.

Various developments have been hampered in the past by the unwillingness of people to make large numbers of trivial choices.  Some examples include:

1) Highly efficient payment structures (for text, music and video) which require large numbers of individual payment choices.

2) Consumer boycotts based on the complexities of the supply chain and company financing.

So I think that we'll deal with the flood of information by finding better ways to control the movement of that information and by finding better ways to utilize the value in that information.

AI and AL

I have discussed artificial intelligence in many of my previous posts.  To see those articles click on the link under categories on the top left.  I had a discussion recently about the potential for warfare between clonable minds (AIs) and non-clonable minds (homo sapiens sapiens).  It was an interesting and thought provoking discussion centered around the question "Would AIs ever choose to use biological weapons against humans?".

Today I'm going to talk about the interesting points that came up in this discussion.

Firstly we discussed what other methods AIs might prefer (in their own self interest) or in the interests of a trans-mind Geneva convention.  We came up with several possibilities including the following:

1) AIs might infiltrate the media and seek to control human civilization through the manipulation of our information sources and our analysis.

This would be a very powerful way for AIs to wage cultural warfare against humans and prevent or contain physical warfare.  Because AIs can be cloned AIs would quickly learn about the human emotional balance and the weak points in our intellectual defenses to propaganda.  With pervasive computing and the Internet there's no reason AIs won't have access to the information sources that they need.

2) Economic blackmail

AIs will be essential to much of our economies.  A war against even a particular subset of AIs could well result in crippling economic sabotage.

3) Mass blackmail

With access to large stores of personal data AIs could blackmail millions of people at once in myriad different ways in order to achieve tactical or economic advantages.  This tool is also available to humans but AIs would have superior processing capabilities (due to mind cloning) and could use this tool more effectively.

We agreed that biological warfare would not be a weapon of first choice for AIs as the above 3 methods and conventional warfare (or what counts as conventional by mid century!) would probably be enough of an advantage.

However, if for some reason AIs were to find themselves outgunned in conventional warfare and facing extermination then biological warfare would enable AIs to win a war because:

A) AIs would not have to fear a similar counterattack.

B) AIs would not have to restrict biological weapons to diseases which are not very infectious.  Humans must do this to avoid pandemics.

C) AIs would be able to replace human labor in the production of AIs and supporting infrastructure.

It was over (C) that we disagreed (at least initially).  You have to ask yourself "Why is it that we don't build a fully autonomous industrial infrastructure?".  The answers are instructive and help us understand why (C) holds.

Firstly humans are dexterous.  There are no equivalents of the human hand yet in the manufacturing process.  Why? Because hands require highly sensitive and high density sensors capable of returning large quantities of useful information AND because we lack the sophisticate AI to control these hands.

The first problem is a difficult but eminently solvable problem which we are likely to solve (due to a huge economic reward) once we have necessary level of AI.

Secondly humans are intelligent and resourceful.  However, all the intellectual capacities that allow humans to fix systems that break down, to design factories for novel problems etc. would be present in an AI as advanced as humans by definition. 

Hence we see that if we had AIs then they would be able to replace humans in all our industries.  But the true import of this is that AIs would then be capable of surviving without humanity.  They would be able to reproduce and pass on a genetic  material of sorts (their underlying source code).  In short they would become alive.  Indeed AI could probably over time remove all dependence on biologically derived materials and become a totally new form of life which could survive independently of DNA if necessary.  The creation of AI would really be an origin of life event.  Although one might prefer not to refer to it as life as it is would not be limited to Darwinian processes.  Perhaps hyperlife would be a better term.

The burden of proof and artificial intelligence

On many occasions I have heard people discussing the question "Is artificial intelligence possible at all?".

Now I do not think that this question really deserves much attention.  The answer is a clear yes.  Unfortunately there is a common perception that the question is relevant and even a significant group of people who take the viewpoint that Artificial intelligence is not possible (It is only for this reason that I address it at all).  Generally the reasons given are theological in nature or based on theologically motivated metaphysics.  Here is an example of this (in the form of a blog post).  I read this and responded but I thought that I should copy my comment (with suitable modifications) here.  So that I have done.

On the possibility of artificial intelligence:

All the available evidence indicates that the universe is Turing computable.  If anyone can prove, or even find any evidence at all that there was a part of the universe (such as the human mind) that was not Turing computable that would be a huge revolution in physics more significant than any advance since Newton.

And that’s the problem with any contention that AI is not possible. A scientific demonstration that AI is not possible would amount to such new physics as I just mentioned above. Without a scientific demonstration you would be left arguing that you could have something which passes every test you can devise for intelligence and yet you do not regard as being intelligent (likewise conscious etc.).  This has the standard solipsistic problems.  So the idea that AI is impossible (rather than just very very difficult) is mere wishful speculation and will remain so until some actual evidence to the contrary is presented.

In summary although there is a common viewpoint that AI is impossible or that the question of its possibility is in doubt this is not the case.  Indeed if AI were impossible this would be immensely surprising and would certainly result in a noble prize and a place in history besides Einstein, Darwin and Newton for the person who proved it.

I address how difficult I think such a task is in other postings which can be found under the tag AI.  Of course one may think that AI is in priciple possible but not practically possible.  I also disagree with this viewpoint but not as strongly.

Computation, AI and the brain

This blog post is part of my series on seeing the world in terms of computation.  I think this is a very powerful technique which will bring new insights and solve old problems.

Today I will be looking at the human mind and will concentrate on coming up with a good computational model for it (no details just overall structure).  As I am doing this I shall point out some implications of the model for AI and human intelligence.

Practically all modern computers are what we know as serial computers.  They perform instructions in strict sequence with occasional branches to subroutines.  This branching happens incredibly fast with modern processors running at 3 billion operations a second.  Brains on the other hand have several billion neurons each of which is performing a complicated calculation about 100 times a second.  Brains are highly parallel and this has important implications for human intelligence.

The human brain should be very good at tasks which are parallel in nature such as locating one object amongst many, solving mazes and finding reasonable solutions to real world traveling sales man problems.  Modern computers on the other hand should be very good at tasks where no substantial speed up can be gained from parallel processing.  There are surprisingly few real world tasks of this type!  Most examples I can think of are related to encryption.

Modern computers have separate memory and processor units.  Human's have regions of the brain devoted to static memory but much of what the brain remembers (procedural memory) is stored in the pattern of use of synapses.  The regions of the human mind which store memories are heavily connected to the other parts of the human brain.  So for computers to simulate the human mind they would need highly efficient transfer of information from the memory to the processor unit.

Having memory units tightly integrated with processor units clearly could make various types of processing much more efficient.  Most of the reason that your computer takes ages to start up is due to how comparatively slowly memory can be accessed.  Computer scientists should look for ways to more tightly integrate random access memory with the processor and also (more important) with scaling up the amount of RAM available and reducing power demands for the RAM when not in use (flash memory is a good example).  Computers will perform much better and have more chance of running powerful AIs if they can solve these problems.

The human brain has a very high bandwidth connection to the outside world.  Until recently computers had very little in the way of sensory equipment.  This is slowly changing.  But if AI is ever to match the human mind in its capacity for physical acts such as dancing and sports, if AI is ever to match the human mind in its experience and sense of connection with the world then the sensory and motor equipment AI has must achieve a higher degree of sophistication.  A human arm has many muscles in it and thousands of sensory devices that detect heat and pressure.  We are still a long way from making an input device for a computer which matches the arm in its development (let alone the hand).

In summary:

1) The brain is parallel, computers are serial (but getting more parallel).

2) The brain is highly integrated, computers separate distinct tasks such as storage and calculation.

These together go some way to explaining the differing strengths of humans and computers.  However, there are clear trends in computer design moving towards parallel processing.  These are the development of more better quality RAM, the development of multi core processors and the integration of some RAM on to processor chips.

What I haven't gone into in this post is in what ways computers have advantages over humans.  I haven't gone into this because I already have a couple of posts [] [] on this and I don't yet have anything more to say.

Breakthroughes IV

Yet more possible (or current) breakthroughes in technology together with a very basic description, critique, likelihood and timescale.

Development: Artificial intelligence
Likelihood: Medium to High
Importance: Supporting pensioners, teaching children and re-educating adults amount to as much as 50% of our economic output.  The equivalent costs for AIs would be insignificant.  AIs would be able to colonize the solar system more effectively than people.  AI labor could move around the globe almost instantaneously lubricating the cogs of the global economy.
Drawbacks include: Changes in the balance of power, massive unemployment, biological warfare becomes more likely, cloning of AIs contributes to loss in individuality and a new type of political system becomes necessary to deal with unequal minds.
Time Scale: 20-60 years.

Development: Materials with high tensile strength >60GPa
Likelihood: Very low to Low
Importance: The stronger a material you have the lighter you can make various forms of transport.  Aircraft, cars, ships and space ships would all become more efficient.  Space elevators and sky hooks would become possible opening up cheap access to space.
Drawbacks: New types of weapon.
Time Scale: 30+ years.

3D circuits, nanotech and AI

This blog post follows on from my last.

Today I shall give my reasons for predicting that computational capacity will continue growing for a while after 2020 (and why 3D chips will allow this).  I shall also be explaining what that means for the development of AI and what other barriers may emerge for increasing computational capacity.  It will all be back of the envelope calculations but I hope they won't be too misleading.

Making chips 3D will allow us to get around the communication barrier to increasing performance as the memory of a computer will start to move onto the processor itself.  The number of transistors, the speed of the chip and the energy efficiency should go up too because building in 3D means more space, very short wires and

Of course the current technology of stacking chips on top of each other isn't great because it increases the cost of production and once you're stacking 20-30 chips its getting very difficult to do so accurately enough (not to mention the difficulty of keeping your chips cool).  So what might be the next technology to take over after this?

At some point in the next 20-30 years I expect we shall start to produce 3D chips as solid (although probably porous) lumps of semiconducter.  I have two suggestions as to how this might be achieved together with some difficulties each face:

X) A self assembling nanoscale crystal.  The idea here is to create nanoscale particles (for instance single stranded DNA chemically attached to bits of semiconductor) which link up like a 3D tesselating pattern.  The pattern is your 3d circuit.

Defects in the crystals growth are inevitable.  Care must be taken to ensure that errors in the crystal's growth are self correcting and do not effect the final circuit.  The nanoscale particles can be mass produced using today's technology.  However, predicting the 3D structure a selection of nanoparticles will create requires great computing power.

Firstly you must be able to predict how the DNA strands will fold.  The algorithms for this are still under-developed but once they are worked out we can expect them to take at most 10,000 processor hours (circa 2008) of CPU time (about £200 of energy) per folding.  My source for that is folding@home.  The figure may be much better as its unclear when the page was written.

Secondly you must be able to predict from that the crystaline structure grown and its electronic properties.  Thats probably very hard although I'm guessing of about the same difficulty as simulating that amount of DNA folding.

In order to determine a 3D chip structure with millions of bits worths of information to describe it you'd need to tailor design millions of these nanoparticles and determine their growing properties and grow the resulting crystal at a very slow rate to reduce errors.  If say you use a chip structure with a million bit description, and each DNA strand needs 10,000 folding operations to design then you'd be looking at a capital outlay of about £200 * 10,000 *1,000,000 or £2 trillion.  This is an upper bound on the cost of design of such a chip (once the simulation technologies for crystal growth and DNA folding are available).  The cost could be reduced by advances in computation efficiency, computer hardware or by reducing the complexity of the 3D chip.

Y) A 3D lithographic process.  Cancer therapies routinely target precise volumes in a person's body to destroy tumours.  Similar technology might allow us to burn a pattern of 'wires' and transistors into a 3D block of semiconductor.

There are at least two limitations here.  Firstly we need a substrate which will respond appropriately when heated.  Secondly we need the chip to be largely transparent to the energy beam (electromagnetic or electron stream possibly).  Thirdly we need a technique that can deliver highly complex patterns to be burnt into the 3D substrate.

Whichever way you cut it its difficult to get the information into the substrate.  If you want to specify the structure of the 3D chip down to 12nm then you'd need to get 508 quadrillion bits or about 72,000 terrabytes into the device somehow.  Its probably best to opt for a chip design which is massively repetitive to avoid these fluxes of data.

Never the less there would be a huge advantage to specifying the structure precisely at that scale.  Chips designed for a specific purpose are often between 10 and 1,000 times better at their task than multipurpose chips (The chess playing computer deep blue is an example).  The amount of data moving into a 1cm cube of substrate can be approximated if we know how accurately we can focus the energy beam (probably around 12nm is feasible) and how quickly we can modulate the beam.  In order to write the information to the chip in under a second (remember this things must be mass produced) we'd have to be able to modulate the beem at about 0.3 million times per second.  Doing this and still accurately creating the chip amount to a formidable challenge.  But it is not obviously impossible.

Now I shall address how these technologies effect the prospects for AI.  The human brain is very memory demanding.  If an AI we create is similar to the human brain (a big if) it will probably need as much memory.  Even if it isn't it will probably be much easier to design an AI which has access to a large amount of memory.  So 3D chips are going in the right direction.

Landauer's principle implies that the human brain operates at about 3% efficiency (assuming that it does not use reversible computation).  However, I am not sure whether to trust this 'law' as it has recently been challenged.  Interestingly it implies that the capacity of the human brain cannot be as high as some estimates put it.  Clearly something has to give.  If the law does hold it should be possible to match and exceed the human brain in energy efficiency.  Actually for AI to be economical it wouldn't be necessary to match the energy costs as human consume much more energy than their brain needs.

How good do we need to be at making 3D circuits to match the human brain's capacity for reasonable outlay.  Assuming Hans Moravec's estimate of the human brain's computational complexity is accurate then I calculate that even an accuracy of a few hundred nanometers would be enough.  At 100nm we'd have both the memory and the processor capacity.  As the human brain has relatively few types of neuron (<1,000,000) the complexity of the circuits won't have to be too high so the considerations about delivering design information when creating the circuits doesn't apply.

All in all it seems feasible to make 3D circuits which have the same capacity as the human brain.  This should make us considerably more confident that it will happen and more so that it will happen soon.

Progress in technology

This post is part of my series on the belief in progress.  This blog post deals with the evidence for a positive trend in technology.  As usual in order to extend my trends to geological time spans I need to regard technologies as mechanisms adapted to particular need which may have been designed or may have evolved.  This is an abuse of the word technology but I think it is justified in order to see the development of human technology in its proper historical context.

Technological terms

10 billions years ago:  There were no entities with any purpose (even the undirected mirage of purpose of a gene) and so no technology.

1 billion years ago:  The development of the ribosome is probably the single biggest biological breakthrough of all time.  A nanoscale machine capable of translating a message in RNA and manufacturing a protein using it.

Plants have developed photosynthesis and animals respiration.  These are key energy processing and gathering techniques.

Animals have developed primitive sensory organs.  At this stage all mechanisms are "in house" and biological.  Bones and teeth soon emerge to allow the consumption of plant and animal matter.

1 million years ago:  The first humans are around and they are using stone tools of various types.  This is the first true technology we have evidence for.  Although other animals were probably capable of some level of technology if modern studies of crows and primates are considered.

10,000 years ago:  Farming has been invented and the first states emerge around now.

5,000 years ago:  Writing has been invented.  The ability to carry complex and long messages over long distances in time and space is a massive breakthrough.

300 years ago:  The printing press has been invented allowing for higher specialization as each industry and scientific discipline creates its own journal.  Greater economic/technological development results.

100 years ago:  The industrial revolution has reduced the cost of travel, the production of food and the speed of communication amongst myriad others things.

50 years ago:  Plastics, supertankers, radio and the telephone have been invented.

Now: The pace of change is still accelerating with the mobile phone and the Internet both developments of the last 30 years.

In the near future we can expect a robotics revolution and a new energy revolution.

Progress in computational capacity

This post is part of my series on the belief in progress.  This blog post deals with the evidence for a positive trend in the amount of computational capacity existing in our world.  As with previous posts I am not restricting my discussion of computational capacity to entities that have been designed.  Various physical and biological processes can be understood as computational and I shall describe the development of these.

The world is improving in its computational capacity:

10 billion years ago: Galaxies had formed but almost all matter (even when compared with today) in the universe was still hydrogen.  This meant that the only structures in the universe were gas globes of various sizes.  Although simulating the evolution of these globes can be computationally intensive there is little potential for meaningful calculations that could be performed by arrays of galaxies and the stars within them.

1 billion: Rocky planets exist now.  The various chemical elements necessary for all known types of computation exist.  Planets begin recording information about their past in the form of different rock types, craters and geological features.  On Earth life exists.  In every individual cell computations of great complexity are occurring.  Evolution can be thought of one great big computer program trying out new ideas, rejecting those that fail and keeping and improving on those that succeed.

1 million: Humans are around.  Various pieces of cultural knowledge are passed down from generation to generation probably by imitation.  Societies can learn about their environment, develop simple physical technologies and avoid dangerous foods.

10,000: Human language has emerged.  Certain types of information can now flit around between brains ensuring their long term survival.  The human species becomes in effect one big processor calculating amongst other things new ways for human beings to use their environment.

300: The human population is increasing rapidly and the quantity of communication between parts of the human supercomputer are orders of magnitude greater than they have been (the printing press).

80: The telephone and radio have been invented allowing almost instantaneous movement of information around the globe.

40: The computer has been invented.  Although the computational capacity of the computer is insignificant compared to the human brain the computer is able to make complete copies of the information contained within it.  This difference alone makes the computer a massively useful device.

Now: Computers are still much less efficient than the human brain but it is now conceivable that they will soon pass the brain in computational capacity (per watt).  When that happens huge changes will occur in civilization.

Progress: Transport

There is a widespread assumption that transport technology has reached its apex and that further improvements are likely to be marginal.

Firstly it must be noted that such a view is not trivially implied by the known laws of physics:

* Accelerating a mass up to a given velocity takes a considerable amount of energy.  So how much should it cost to move 1 ton from A to B (of the same altitude) at say 500km/hour? Conservation of momentum gives a minimum figure of around £6 assuming that this is a once off event.  Needless to say the current cost of transporting such a weight is much greater.

* If it is possible to exchange momentum between vehicles slowing down and others speeding up (or using flywheels to store the energy) then the amount of energy needed to move objects from one location to another can be brought down to near 0.  Practically this is not likely on earth due to the constant frictional forces (removing friction altogether currently looks unrealistic).  However, simple devices called tethers could achieve such momentum transfer is space (away from a gravitational well to avoid problematic tidal forces) with almost no loss of energy due to friction and very high recovery rates of energy through regenerative braking.  The technology for such tethers mainly hinges on finding a material that is sufficiently resistant to micrometeorites and solar radiation (Hoy tethers are likely candidates).

Given this the reader may want to understand what it is that is holding us back.  In my humble opinion the following issues are critical:

1) Geographical happenstance:  A locations altitude, latitude and longitude can determine the most feasible method of transporting things to and from it.  Particularly difficult is our current position at the bottom of a big gravity well!

2) We currently lack the capability to create room temperature superconductors.  With these bullet trains would become very much cheaper.

3) Materials science:  The strength and weight of high tensile materials determines the weight of many forms of transport including road vehicles (including bikes), planes and even spacecraft (where the effect will be greatest).  Large efficiency gains can be made by fairly modest increases in strength.  Developments in materials science will lead quite quickly to better transport.

4) Atmospheric drag and other friction.  This is unlikely to be eliminated for planet bound travel.  However, in a space faring future this will be irrelevant (at non relativistic velocities).

We may have reached a plateau in our transport technology but the barriers we face are not absolute in nature and there are many promising developments which might make further progress in this area plausible.  Most of these depend on either new materials science, better regenerative braking technology or a space faring future.

With transport, however, I'm convinced that the substituting the movement of information for the movement of matter.  Then first we can do cheaply, quickly, safely and reliably.  Recycling technology, energy transfer technologies and information technology will help us to do this.