Symbolic AI vs Connectionism

Symbolic AI and Connectionism are two contrasting approaches to artificial intelligence (AI), both aiming to recreate human intelligence through distinct methods.

Symbolic AI, also known as "Good Old-Fashioned AI" (GOFAI), focuses on the development of rule-based systems that manipulate symbols and apply logical reasoning to solve problems. These systems are designed to imitate human thought processes by utilizing symbolic representations and formal logic, which are believed to underlie human cognition.

On the other hand, connectionism is an approach inspired by the neural networks found in biological systems, specifically the human brain. Connectionist models, often referred to as artificial neural networks (ANNs), comprise interconnected nodes or neurons that process and transmit information. These networks learn to recognize patterns and make decisions by adjusting the strength of connections between nodes based on input data. This learning process is achieved through training, where the network is exposed to a large number of examples, allowing it to gradually improve its performance.

Symbolic AI can be compared to building a LEGO castle using an instruction manual, where each block has a specific meaning and place. On the other hand, Connectionist AI is more like growing a plant, where it learns and adapts over time without specific instructions.

The Symbolic AI crowd believes that if a smart enough programmer wrote down all the instructions for how to think, we would have a formal model of what thinking is. In contrast, the connectionist contingent thinks that there is no way around letting the system learn from data. In this approach, the intelligence doesn't come from the programmer, but from the information provided in the environment.

The ongoing debate between symbolic AI and connectionism has significantly influenced AI research over the years. This intellectual tug-of-war has led to the rise and fall of various methodologies, ultimately shaping the direction and progress of AI development. While symbolic AI emphasizes structured knowledge and rule-based systems, connectionism highlights the importance of learning through experience and adapting to new information. The interplay between these two approaches continues to impact the evolution of AI as researchers strive to develop more advanced and efficient models of intelligence.

The Key Players

Heinz von Foerster

Heinz von Foerster, an Austrian-American physicist, and Marvin Minsky, an American cognitive scientist, were key players in this AI tug-of-war. Von Foerster rose to prominence after attending the Macy Conferences on Cybernetics in New York. The Macy Conferences were a series of highly impactful interdisciplinary meetings held in New York between 1946 and 1953, aimed at exploring the potential of cybernetics. Cybernetics is a field that seeks to find general patterns and structures that can be applied across different systems, such as biological, social, and technological systems, in order to improve their efficiency, adaptability, and overall performance.

Among the influential attendees were incredible thinkers such as John von Neumann, whose work spanned an extreme multitude of fields from physics to computing and economics. In fact all modern computers are built on an architecture called the "Von Neumann architecture". Von Foerster was an upstart from a physics background who only got into computer intelligence and rose to prominence because he had stumbled into the Macy conferences almost by accident.

Von Foerster later founded the Biological Computer Lab (BCL) at the University of Illinois in 1958. The BCL focused on self-organization (a process whereby a system spontaneously forms an orderly structure without external guidance), learning, and artificial neural networks, drawing inspiration from biological systems.

Marvin Minsky & John McCarthy

Marvin Minsky, on the other hand, began his journey as a teenager by building mechanical contraptions focused around computation. After showcasing his early aptitude for engineering and robotics, he studied at Harvard and Princeton, and later co-founded the MIT Artificial Intelligence Laboratory with John McCarthy in 1959. The MIT AI Lab championed the symbolic AI approach, which aimed to understand and replicate human intelligence by breaking down complex tasks into simpler, rule-based subtasks. This method of AI research involved creating algorithms and representations that could simulate cognitive processes. Minsky and his colleagues believed that by emulating these processes, they could eventually develop machines capable of human-like problem-solving and reasoning.

Where von Foerster spoke in the obscure language of the cyberneticians and called his machine learning research "second-order cybernetics", Minsky and McCarthy had a knack for popularizing their field and promising fast advances. Branding their research (and inventing the term) "Artificial Intelligence" they managed to inspire the public with a future that was very close.

The Clash

In the 1960s, the philosophical clash between the BCL and the MIT AI Lab deepened. As symbolic AI gained popularity and funding by promising fast results, the BCL's connectionist approach faced challenges in securing resources for research. The connectionists were more focused on foundational research and didn't promise immediate results. In a constrained funding environment this was a death sentence. The BCL was disbanded in 1974, marking the end of an era in AI research. Reasons for its closure included budget cuts, changes in the academic environment, and the growing dominance of symbolic AI. The demise of the BCL signified a temporary setback for alternative AI approaches.

With the closure of the BCL and the dominance of symbolic AI, resources for alternative approaches like neural networks were scarce. AI research became heavily focused on rule-based systems, limiting the development of techniques that would later give rise to large language models (LLMs) like ChatGPT. Despite the constraints, a few dedicated researchers continued to explore the potential of neural networks.

Impact on Modern AI

During the period of symbolic AI's dominance, researchers such as Geoffrey Hinton, Yann LeCun, and Yoshua Bengio persisted in their exploration of neural networks. These pioneers faced numerous challenges due to limited resources and support, but their dedication to connectionism eventually paid off. In the late 1980s and early 1990s, neural networks regained popularity with the development of more advanced algorithms and learning techniques, such as backpropagation. This resurgence was further fueled by the increasing availability of computational power and data, which allowed researchers to build more sophisticated models and make significant progress in areas like deep learning and natural language processing. As computational power and data availability increased over time, these pioneers were able to develop more advanced algorithms and learning techniques. This persistence eventually led to breakthroughs in deep learning and natural language processing, paving the way for the emergence of powerful large language models like ChatGPT that have transformed the AI landscape.

One of the key factors explaining why neural networks and in general connectionist approaches win so decisively against symbolic approaches is that the limiting factor for intelligence isn't how fast your computer is, but how much information you have available. Achim Hoffman explored this in his 1990s seminal paper on "General Limitations on Machine Learning". Trying to write a system by hand that acts intelligently would require us to input all of the information ahead of time by hand. But mostly intelligence is just learning from data, which means building a system that can find patterns in data. Up till now all computational learning systems have been severely constrained in the data available to them.

Biologists Humberto Maturana and Francisco Varela explored the idea that all life itself is a cognitive process in their 1980s book "Autopoeisis and Cognition". They argue that because that all living systems are fundamentally cognitive because they self-organize in order to create and maintain their structure. This self-organizing nature allows organisms to process information, adapt, and respond to changes in their environment, regardless of whether they possess a nervous system or not.

Neo-Yuddism

With all the successes of GPT-4 and the connectionist approach you'd think the debate is settled. But the Symbolic AI crowd around people like Eliezer Yudkowski feel scared by the dynamic nature of intelligence. They were hoping that it would be possible for very smart humans to write the AI in a very formal and structured way. But now the AI we actually know how to build simply learns from the information we have available in the world.

With a looming public discourse around AI safety, the irony at the heart of the issue is that the Symbolic AI crowd who first invented the marketing term "Artificial Intelligence" in the late 50s to concentrate funding and starve their connectionist competition, are now uses the same term to scare the public of something that's very far from "Artificial Intelligence".

So far what LLMs (Large Language Models) have done is to very effectively compress large sets of information to find the patterns within them. See What it is doing is less like "figuring out" and more like "already knowing".

GPT-4 has consumed the world's information, but it's not actually good at reasoning about it. The progress that has been made is exciting, and the applications are truly inspiring. But just like the AI crowd in the 70s got excited by how close they thought they were to AI, they are now scared by how close they think AI is despite this being far from the truth.

The reality is more complicated: We've made a ton of progress in compressing existing information into queryable models, but we've made very little progress in increasing the amount of information available in a self-organizing way to truly move forward. To acquire more information beyond a certain point requires interaction with the world. The running of experiments. And to run experiments about the nature of reality requires resources, and time, and patience. Because while computation is fast, the world is slow. As LLMs get more data they will learn. But the speed of that learning will be limited by how much data can ingest, and the world simply does not hold enough data for them to progress all that much beyond where they are now.

While AI companies are still improving their techniques, they can squeeze additional information out of the existing data, and through data sharing deals they can acquire more data. But not substantial amounts. Once the incremental improvements and the data run dry, the improvements might be closer to resembling a logistical curve than an exponential one.

The path from here to true AI might be longer than the path from the 70s to here. What I wrote in 2016 still holds true: We are still limited by data, just like we have been all along, and we have not yet found a way to increase the amount of training data available by the orders of magnitude required. We should savor the amazing bounties we can get from the progress that was made and build amazing companies and tools based on LLMs, GANs, and other neural networks without succumbing to Neo-Yuddism because we are scared of something that might not happen in our lifetime.

As you can see, the debate between the Symbolic AI followers who insist that we need to have a formal understanding of how intelligence works before we're allowed to build better information querying systems, and the connectionists who embrace that knowledge acquisition is a dynamic process and both takes time and is always limited by the amount of available information, is far from over.

It's interesting to reflect on how these fundamental philosophical disagreements can reverberate for decades in the discourse and shape the direction of research and funding allocation. It will be intriguing to see if and when this debate can ever be settled.

Few things are more powerful in the world than the ability to shape the narrative, censor others, and push your own messaging. Time and again, we've seen that no matter who controls speech, power corrupts.

Twitter has been at the center of the intelligentsia's struggle over the narrative: No matter who is in charge, we know that the power will be abused: Twitter's previous leadership censored political opponents, Elon censored competitors, and no doubt if someone else gained this power, they too would succumb to wielding it. It's as if the power to control speech was The One Ring, and no one is strong enough to resist it forever. So some of us have concluded that the only reasonable option is to throw The One Ring (Twitter) back into the volcano from which it was forged (The Internet).

So how would you go about doing this? Twitter's former CEO Jack Dorsey announced the Blue Sky project on December 11, 2019. Its aims were simple: To develop an open protocol to address the challenges faced by centralized platforms.

Let's dig into what moving from a centralized service to a decentralized protocol means:

A centralized service runs on computers wholly under control of a single entity. They can change, monitor, amplify and bury all information. They can read your private messages. They can delete content, promote other content, and even impersonate you. They can tell you that they are doing this in your interest, and they can, as Twitter has recently done make assurances that they are publishing their process. But in the end you have to trust that those in power won't read your private messages and that they won't censor and de-platform you because they disagree with you. If the press is indeed the fifth estate, then it seems clear that Democracy can't function if we hand the power to control speech to centralized entities, be they foreign or domestic, well-intended or malicious.

A protocol on the other hand is an agreement for how computers can talk to each other. The word protocol has its origins in describing the set of forms and etiquettes observed when meeting foreign and local dignitaries. Think of it as something along the lines of "First gifts are exchanged, and then you hand the business card with both hands while bowing looking down deferentially as you're handing it to the other person". Protocols are what power the internet. When your E-Mail program sends a message to an E-Mail server, that's what it does: First, it says Hi, then it demonstrates that it's allowed to send email on behalf of the person they are sending as, next they submit the subject line and the body of the message, and finally after receiving acknowledgement that the message was sent, they say goodbye if they are polite.

To "decentralize Twitter" essentially means to transform the platform from a centralized service, controlled by a single entity, to a distributed network where control and decision-making are spread across multiple, independent nodes. This is a big effort and involves redesigning the underlying architecture and protocols to enable peer-to-peer communication, data storage, and content sharing. The challenges involved are definitely big. But the team at Blue Sky told me they have solved it. So let's go deeper.

Blue Sky is built on something they call the AT Protocol (or @ Protocol?), which stands for Authenticated Transfer Protocol, aka ATP. What this protocol needs to define is how users can create and update records of their activities (posts, comments, likes, follows). Blue Sky/ATP calls the place data like this is stored "Signed Data Repositories". Signed, because each user has a cryptographic key to update his data repository. Each data repository is the equivalent to an account on twitter (e.g. @justinbieber or @rihanna).

But how do users "own" these data repositories? In ATP - remember, ATP is the protocol that Blue Sky, the decentralized Twitter replacement is built on - accounts are primarily identified by a cryptographic key. This key has a public component, and a private component. This is very similar to how Bitcoin wallets have a public name/address, and a private key that's required to access the funds contained within that address. The true name of the ATP account is what's called a DID, a Decentralized Identifier, which is just a representation of the public key (there's a bit more to it, but it's best to just think of it as a public key for now).

For example, my DID looks like this

Now a DID like that wouldn't make a great username, so how do we get to user-readable usernames from here? ATP's solution is simple: Just reuse DNS (the Domain Name System), one of the core internet protocols that's already in use everywhere. What DNS does for millions of queries per second already is to resolve names like "amazon.com" to IP numbers like 192.168.0.1. So why not use DNS to resolve did:plc:vzemfzughnwhrtvr63ur44r6 to paulbohm.com? Exactly, that's what ATP does.

So where this puts us at is that bluesky uses domain names (example.com) not emails (user@example.com) as the identifiers most users will see. It also means that you can change your username (Domain Name), while retaining your identity (DID, the equivalent of an IP Address) and all your followers and other information stored in Data Repositories. You can still subsegment each domain using subdomains, so anyone working for example.com could get paul.example.com and maria.example.com entries to show their affiliation.

I think this innovation is quite a big deal: It means that while there is still a bit of a land rush for people to get accounts on the original .bsky.social domain for OG bragging rights, the obvious solution for those who really care about their brand is to get a cool domain name. There's another reason you really want your own domain as username: It makes it easy for people to know it's you. If someone registers celebrity.bsky.social on the network, you can't be sure it's them. But if they show up with their domain name, it's a lot easier to establish if they are who they say they are.

Ok this gets us to about half-time: We've described how Blue Sky/ATP defines identities, and how users own the keys to those identities, but what about servers? Isn't Twitter run in these enormously big data centers with millions of computers crunching all the Tweets? How do you decentralize that? Glad you asked!

Blue Sky's approach to decentralization is using a paradigm called Federation. What this means is that all the data from the Data Repositories needs to be stored somewhere. And ideally not just in one place controlled by one company, but in many places controlled by many people.

So let's recap: Users own accounts which have public and private keys. The public key is public, and everyone knows about it, but the private key is private — only the user has it. This also means that even the servers can't impersonate you since they don't have your key. The private key/DID/Decentralized Identifier is the true name of the account, but the account can also have a domain name — just like websites on the internet have domain names and IP Addresses. Most of the time you never interact with the DID or public key but just with the domain name of other people. The software takes care of it for you.

The data that users sign using their private key gets stored in Data Repositories, and those Data Repositories in turn get stored on Federation Servers. These Federation Servers need to synchronize data between them so that multiple servers in the world share the same world-view. The way this will be accomplished is using peering agreements. This means you need to talk to someone who is already part of the network to give you access to the "firehose" of all the information that happens on the network. Once enough servers have a global view of all the posts, that should become easier: You won't be dependent on any singular entity gatekeeping you away from the global feed of information, but you might have to pay someone for sending you all that data. It's open because you're probably going to get it, but it's probably not free forever.

There are two types of networking the Blue Sky / ATP Federation supports:

Big-World Networking: That's the firehose. Having all the data of the entire network on all the Big-World Servers. That's what you need to do if you want to crunch the data, make it searchable, and provide algorithms that help you discover new content and new people to follow. Not everyone needs to run a Big-World server, and running a Big-World server can be quite costly.

Small-World Networking: This is much more directed and downstream from Big-World: A Small-World Server only gets the information it needs to serve its clients: Posts, DMs, likes, and so on for just the people it serves, and those people its users follow.

It's a lot cheaper to operate a small-world server than a Big-World server. And while it's not explicitly discussed, I'd argue there's probably an argument to be made that there's also a need for Huge-World Servers which not only see all the data, but store it forever; Not every Big-World server will want to store decades worth of data since that can be very costly.

Now what's cool about this architecture is that you're not dependent on any single server. You can take your account (DID/account key) and move to any server you want. If it's a Big-World server, it already has a copy of your data repository. If it's a Small-World server it can ask its upstream Big-World Server for a copy and there you go.

It also means that if you don't agree with the algorithmic filtering on the server that you are on, it's incredibly easy and fast to just pack up and move. Just connect to a new server and there you go.

Now let's talk a bit about the economics of moving a beast with multiple data centers to such a federated/decentralized architecture. How much is it going to cost, optimistically, to run a Big-World server? How much is it going to cost to send all that data between multiple Big-World servers?

For that we'll need to make some assumptions, and I'd love to hear from you if you can make a better guess. This is my napkin math:

Assuming 6,000 tweets per second, an average tweet size of 3 KB, $0.05/GB bandwidth cost, $0.02/GB storage cost, and no data redundancy, the daily data storage needs of Twitter might be estimated as follows:

Write throughput: 6,000 tweets/second * 3 KB/tweet = 18,000 KB/second ≈ 17.58 MB/second

Storage capacity (daily): 17.58 MB/second * 86,400 seconds/day ≈ 1,519.07 GB/day ≈ 1.51 TB/day

Storage capacity (monthly, with 30-day data retention): 1,519.07 GB/day * 30 days ≈ 45,572.1 GB/month ≈ 44.54 TB/month

Daily cost for storage: 1,519.07 GB/day * $0.02/GB ≈ $30.38/day

Daily cost for bandwidth: 1,519.07 GB/day * $0.05/GB ≈ $75.95/day

Total daily cost: $30.38/day (storage) + $75.95/day (bandwidth) ≈ $106.33/day

Monthly cost for storage: $30.38/day * 30 days ≈ $911.40/month

Monthly cost for bandwidth: $75.95/day * 30 days ≈ $2,278.50/month

Total monthly cost: $911.40/month (storage) + $2,278.50/month (bandwidth) ≈ $3,189.90/month

The hardware to handle the write speeds and puts us 2x redundancy might cost around $30k. Add ~$500 of electricity cost (~144kWh/day), and round to a rough order of magnitude: This puts us somewhere in the $4,000-10k/month range for storing 30 days worth of data with 2x redundancy.

That seems at least order of magnitude viable, especially since other Big-World servers and downstream Small-World servers will likely pay for both data and indexing/algorithmic filtering services.

So in conclusion, as crazy as the idea of moving the Twitter behemoth to a decentralized/federated network sounds: It might actually work, and it could create a really cool ecosystem that allows a thriving ecosystem of service providers to build on top of the protocol. Thumbs up from me!

Now, if you're wondering if something similar could be done in other industries, subscribe to my newsletter and follow along as I post more about our project to decentralize ridesharing at Teleport.XYZ.

Can we speed up scientific progress?

In 1874, Lord Kelvin observed:

Scientific wealth tends to accumulate according to the law of compound interest. Every addition to knowledge of the properties of matter supplies [the physical scientist] with new instrumental means for discovering and interpreting phenomena of nature, which in their turn afford foundations of fresh generalisations, bringing gains of permanent value into the great storehouse of [natural] philosophy.

If Kelvin’s statements had sufficiently described the accumulation of scientific wealth, we would expect to see continuous exponential progress.

However, progress only superficially appears to proceed exponentially. Indeed, on closer observation, it is actually made up of many smaller paradigm shifts, with each shift creating an entirely new field.

Such a breakthrough initially facilitates many significant follow-on discoveries, but as the new field matures, scientists are able to make successively fewer and fewer major discoveries within it.

One such breakthrough was the Germ theory of disease, which states that some diseases are caused by microorganisms. Girolamo Fracastoro was one of the first to formulate the theory back in 1546, but it took hundreds of years for the theory to gain acceptance.

Countless versions of the theory were formulated over the centuries. For example, Ignaz Semmelweis, a Hungarian obstetrician working at the Vienna General Hospital, provided evidence that washing hands reduced mortality from childbirth from 18% to 2.2% in his hospital. Semmelweis’s observations conflicted with the established scientific and medical opinions of the time, and some doctors were offended at the suggestion that they should wash their hands. Semmelweis lost his job, was pronounced insane, and died of septicemia (bacterial infection) in an insane asylum only 14 days after being committed.

By 1880 Pasteur and Koch, both already established and well respected scientists, finally provided evidence considered substantial enough for the germ theory to gain acceptance. It was then that a golden age of bacteriology started. We illustrate this in the logistic S-Curve below.

Suddenly huge amounts of resources and manpower began to flood into the field, and between 1879 and 1889, German microbiologists isolated the organisms that cause cholera, typhoid fever, diphtheria, pneumonia, tetanus, meningitis, gonorrhea, as well the staphylococcus and streptococcus organisms. But while the initial discovery was a revolutionary paradigm shift that started a golden age, many further discoveries — at least initially — were merely incremental additions to human knowledge. All the low-hanging fruit quickly had quickly been harvested.

Exponential progress requires exponential innovation.
By 1890 Viruses were discovered.

It’s not enough for a new insight to occur every once in a while. This would lead to merely linear progress. Truly fundamental insights open up new avenues for research.

We believe that we’re living in times of ever-increasing faster exponential progress. But there are those who believe that in many fields this progress has been slowing. 

For a new idea to succeed on the market of ideas, an inventor or scientist needs to first achieve a significant result, then draw sufficient attention to the result, and finally either convince the established cabal in their field of the validity of the result, or alternatively decide to sidestep the academic process altogether, and directly, entrepreneurially, implement their discovery in the form of a product or service.

Depending on the type of project, the inventor might require funds, or even institutional permission, to develop and test their hypothesis and generate sufficient evidence to move forward. But when it comes to funding, in the last decades, radical experiments have been increasingly stymied in favor of safe, predictable, incremental improvements. Institutionalism, centralization, standardization, and risk-averseness have homogenized the kind of research that can get funded. 

True innovation is by definition difficult to recognize. Everything that is obvious has already been discovered. New ideas require rethinking and challenging what we believe to be true and absolute. It is tough for outsiders - and most innovators are outsiders - to garner the level of attention they need to get their ideas funded and accepted. 

Only after an outsider has succeeded do we begin to admire their way of thinking, and search for people who fit the same superficial pattern — looks, thinking style, education, personal history, type of approach —to begin the next wave of innovation. However, the previous generation of rebels often are not the ones who can recognize the significance of the next. By the very nature of innovation, every new wave will be unlike the previous one. Truly new ideas always look unworkable, boring, crazy, and absurd.

It does not help that those invested in the dominant paradigms form cartels that try to keep out upstarts and alternative views. To name but a few examples: If during the last 30 or so years you had, worked on quantum gravity, but did not believe in string theory, or researched vaccines, but did not believe subunit vaccines were the only viable approach, or were an oncologist, but did believe immunotherapy was a valid approach, you’d have struggled to find funding, or even a faculty position.

Not only do the dominant thought collectives at each university often actively conspire to keep out alternative form of inquiry on a local level, but given the invention of the internet and the emergence of globalized citation rankings, it has become international career-suicide to work on anything but the dominant theories in many scientific fields. We’ve traded parallel lines of inquiry for having to look good to as many of our peers as possible.

As discussed at the beginning of this essay, incremental improvements to dominant theories are unlikely to yield the kind of results we do require for sustained exponential progress. It’s only when we challenge the status quo that we can truly move humanity forward. We’re wasting valuable resources by condemning our best and brightest to dig in mines that have already been depleted.

The few that do succeed in bringing us true innovation must serve two masters: Being intellectually far enough outside the group consensus to discover original ideas, but skilled enough as a courtier to attract funding and talent to their radical vision. The kind of innovator that can truly shine both as an independent thinker, and at a court that systematically punishes divergence from group think, is rare indeed. If we want to see more innovation and faster progress, we’ll need to find ways to bridge that gap and make it a goal to have many parallel competing lines of inquiry pursued.

Paul Stefan Bohm (follow me on twitter) writes about self-organization in Computers, Biology and Society.

Thanks to Kumar Thangudu, Christine Peterson, Kai Peter Chang, and Silver Keskküla, Michael Solana, and Geoff Lewis for reading a drafts of this article and providing feedback.

At Teleport.XYZ, we're creating an open ridesharing network called the TRIP Network. Our vision is to have an open network similar to the open nature of the web and email. Just as you can use different web browsers like Chrome or Safari to access your email and website hosted on different providers like Squarespace, Wix, GMail, and Outlook, our ridesharing network will allow anyone to build on top of it without a single company controlling every aspect of it.

We believe that in the future, there will be many service providers offering ridesharing, food delivery, and other services, without any single company taking up to 50% from every transaction. With digital money, this vision is now possible, but connecting the digital world of smart contracts and DAOs to the realms of law, regulation, identity, and finance required the invention of bridges. 

Adapter companies serve as these bridges and are the key to achieving seamless integration of Decentralized Physical Infrastructure Networks with traditional systems. Such adapter companies enable web3-based protocols to operate within established frameworks (such as ridesharing and financial regulation) while retaining their decentralized nature, facilitating seamless collaboration between the old and new economies.

One of the first examples of an adapter company were Bitcoin miners. Despite participating in a decentralized network, they function as traditional businesses, such as LLCs or sole proprietorships, allowing them to rent offices and pay for electricity using a bank account.

In the case of the TRIP Rideshare Protocol we needed to define two primary types of Adapter Companies:

1. Operating Companies that run software to connect drivers and riders, plan routes, set prices, provide customer support, interact with local ridesharing regulation and insurance, and perform background checks on drivers.

2. Client Companies that publish ridesharing software to the Apple and Google App Stores, process credit card transactions, and enable drivers and riders to connect to the Operating Companies to find each other.

In conclusion: To create an open network for ridesharing in which many Operating Companies and many Client Companies can interact, bridges between the world of digital money and smart contracts to the worlds of finance, regulation, insurance, and off-chain computation are required. We call these bridges Adapter Companies and we encourage others who are developing open networks and decentralized protocols that touch the physical world to reuse this as a design pattern.

With the Singularity Summit coming to San Francisco tomorrow, a lot of people are wondering about how close we are to successfully implementing Artificial Intelligence.

Probably you’ve seen a graph like the above before. Computers are getting faster. Does this mean we’re getting closer to true Artificial Intelligence? Not necessarily.

As I’ll show in the following paragraphs, both Artificial Intelligence and the less ambitious Machine Learning (ML) aren’t computationally but data bounded. This has profound implications for viable avenues for research and development.

Generally speaking, the goal of AI and ML research is to acquire a digitally executable copy of a model behavior. The specific model behavior desired can range from classifying (“are there humans in the picture?”, “ixs this news article about economics?”) to predicting values (“how much does a house in this neighborhood sell for?”), to making conversation like a human would (“a chatbot”).

Any such observable behavior can be mathematically described as a mapping from a set of finite input sequences to corresponding outputs, or in other words: a function. And any such function can be translated into a corresponding computer program.

That is, we can represent the model we want to learn (“respond to lines of chat messages like a human would” in the chat bot case) as a function, and then in turn said function as a computer program.

The length of the shortest computer program that provides a mapping equivalent to the model function, can be seen as a measure of the complexity of the model. (For example the shortest program that performs addition on input numbers obviously is shorter — that is, has lower complexity — than the shortest program that parses human acoustic speech)

Now, the goal of machine learning techniques often is seen as helping us acquire a computer program equivalent to the model we want to learn. In general, this feat is performed like this: A human writes a machine learning program L, which given training data T, generates desired program M.

The sleight of hand, in case you haven’t noticed, is that instead of writing M directly, the human has simply decided to write L which writes M for him. The problem is that if the shortest L is shorter than the shortest M, we’re inevitably missing information — there’s many many different candidate M we could be writing — but which one specifically are we supposed to write?

We need to acquire this information from somewhere, and the only place other than L we can get this information from is our training data T. T is a list of examples of M in action. This could be a data set of (houses, their location, and their sales price), or a list of pictures and whether they contain a human face, or log files of human chatroom conversation. The problem is that learning from examples without prior knowledge is notoriously difficult — to learn complex concepts we need an impractical amount of examples. (as proven in [1])

So let us recap here for a second: All the information that goes into producing M is a generator/machine learning function L (written by a human), and training data T (acquired through interaction with humans). The information encoded in L combined with the information extracted from T needs to at least match or exceed the information encoded in M. The only reasonable way to extract information from T is to encode information about its underlying pattern in L. That is: All the work needs to be done by humans.

We face the following conundrum: To get a better model, we either need make the learning algorithm more complex/write more code (which is human work), or we need to gather more or better sample data (which requires human work as well). The complexity of the model M that we want to learn can’t be essentially reduced — it needs to be captured in a combination of program and training data.

Since these are information theoretical limitations, Moore’s law and faster computers do not help. Even a computationally unbounded supercomputer would not be able to break the laws of information theory. Attempts to reduce the necessary work essentially by sophisticated machine learning methods can be compared with attempting to build a perpetuum mobile.

So how do we improve computer models? More often than not the answer is through more and better data. Humans often have knowledge about underlying relationships between data, which helps them write better M and L. But for problems where they themselves haven’t understood the underlying relationships, they are no help. Since the contribution of machine learning techniques to knowledge acquisition tends asymptotically to zero from a viewpoint of algorithmic information theory, more or better data is needed.

Interestingly enough economists Mises and Hayek have discovered the necessity of data to decision making already in the 1920s in their treatises on the Economic Calculation Problem. Since markets force humans to interact, they reveal preferences through interaction. When removing the market, this preference data is never revealed. Thus the market is required to acquire the extract the data needed to do economic planning from people’s heads. In a similar fashion the problem of AI needs to be approached with additional data.

Either this data is directly extracted from a physical medium we know it to be encoded in (e.g. by scanning our body and brain), or we need to find ways to scale the amount of relevant data available to machine learning systems significantly.

A second way besides scanning would be simulation: If we’re fundamentally data limited, and we can’t acquire sufficient data from the physical world through adding sensors or collecting data, maybe we can make computers collect data from world’s of their own creation. In the simple case this would amount to computers playing games of parametrizable difficulty similar to Chess or Go against each other. It’s far easier to specify a game like Go than it is to fully understand it — so in theory we can let the computers explore a search space that provides its own data. The problem is that exploration of such a kind might over time approximate the complexity of a full particle simulation. It’s not clear how deep we’d have to search to find the kind of consciousness patterns we’re looking for. But we need to keep in mind that we might need to simulate an entire universe to find the patterns we are looking for.

Embodiment is the remaining third way, and one that had been chosen for humanity long before it made its own decisions. Here new bodies come loaded with both sensors and computational power. Since structurally there’s no clear boundary between the processes of cognition and living, this could be considered part of an efficient search process for consciousness throughout the entire universe. In this argument, our physical embodiment is actually a fairly efficient vessel, and humanity is more likely than not to become a first super-intelligence. The internet then amounts to said super-intelligence performing brain-surgery on itself, because changing the plumbing makes it capable of acting in a smarter more coordinated fashion, and those of us who develop better information routing technology simply continue the procedure.

In Summary: The real problem of creating good AI is data and complexity, not computational power. The information theoretical bounds described above apply equally to computers of finite and infinite computational power.

In essence, an equivalent question to “can we learn intelligence by observing intelligent behavior” is: Can we learn about the structure of living organisms simply by looking at recorded DNA data. If so, how many DNA samples do we need? A lot. If we could examine the physical organism directly instead, we’d learn a lot faster.

It’ll be interesting to see where all this goes.

Post-Script: I’ve omitted formality in my arguments for the sake of reaching an audience. For a formal argument please see A. Hoffmann, ECAI 1990 and A. Hoffmann, Minds & Machines 2010, whom I’ve often simply paraphrased and in two cases quoted verbatim.

[1] A Ehrenfeucht, D Haussler, M Kearns… 1989 — A general lower bound on the number of examples needed for learnin

Bitcoin is a theoretical and practical breakthrough that makes it possible to decentralize services we couldn’t previously decentralize.

To elaborate: Bitcoin isn’t just a currency but an elegant universal solution to the Byzantine Generals’ Problem in computing, one of the core challenges in reaching consensus in distributed systems. Until recently the problem was thought not to be solvable at all, much less on a global scale. Regardless of its currency aspects, many experts believe Bitcoin is brilliant, in that it technically makes possible what was previously thought to be impossible.

The Byzantine Generals’ Problem goes roughly as follows: N Generals have their armies camped outside a city they want to invade. They know their numbers are strong enough that if at least half of them attack at the same time, they’ll be victorious. But if they don’t coordinate the time of attack, they’ll be spread too thin and will all die. They also suspect that some of the Generals might be disloyal and will send fake messages. Since they can only communicate by messenger, they have no means to verify the authenticity of a message. How, then, can such a large group reach consensus on the time of attack without mutual trust or a central authority, especially when faced with adversaries intent on confusing them?

Bitcoin’s solution is this: All of the Generals start working on a mathematical problem that statistically should take 10 minutes to solve if all of them work on it. Once one of them finds the solution, she broadcasts that solution to all the other Generals. This solution then becomes the next problem. Everyone then stops to work on the previous problem, and proceeds to use the previous solution as the next problem — which again should take another 10 minutes. Every General always starts building upon (using as the problem) the longest chain of solutions he’s seen. After a solution has been built upon (extended) 12 times, every General can be certain that no attacker controlling less than half the computational resources could have created another chain of similar length, because it took a lot of computational resources to extend the chain 12 times. The existence of the 12-block chain of solutions is proof that a majority of Generals have participated in its creation. This is called a proof-of-work scheme.

If this sounds confusing, don’t worry. What it means is simply that consensus is reached because computational resources are scarce. You vote with work. So to rig the vote, an attacker would need to control more computational power than the honest participants have. Also, to ensure that it’s more expensive for an attacker to purchase the computational power needed to attack the system, Bitcoin adds an incentive scheme. Users who contribute computational power get rewarded for their work. Thus, if the value of a Bitcoin rises and attacking the system becomes more profitable, it also becomes more profitable for honest users to add computational resources. Here’s an example of how this works: At any given point, one would expect miners to invest as many resources into mining as is profitable for them. Bitcoin then also is a currency, because it needs incentives to protect the consensus process from attackers. This computational process (“mining”) is not wasteful, but an incredibly efficient way to make attacks economically unprofitable. Bitcoin never uses more computational resources than necessary to protect the integrity of its interactions.

Now let’s return to discussing the value provided by Bitcoin. Essentially it’s a means to make consensus in highly distributed large-scale systems that would otherwise never be able to accomplish that. The value of this is, that it’s now possible to build applications in a decentralized fashion that we previously thought could not be built without a central authority. The most obvious example is of Bitcoin as a medium of exchange for goods and services that can’t be easily bought or sold using cash issued by a central authority. But there’s more. Bitcoin also makes it possible to fully decentralize the DNS (Domain Name System). Every Bitcoin Domain Name comes with a cryptographic key pair, which also allows us to solve the PKI (Public Key Infrastructure) problem: every name you connect to, has an encryption key associated with it that can be verified without having to trust a central authority. Moreover, in case network traffic monitoring prevents people from accessing information either at all or anonymously, Bitcoin makes it feasible to pay for internet relays that anonymize or reroute traffic — that is, it makes it easier to remove central control and fight censorship. The list of benefits goes on, and I’ve been hard-pressed to find any decentralization schemes that would not benefit from Bitcoin integration.

The pattern here is that almost all of these applications can already be built in centralized form. But often the centralized solution comes with many weaknesses. In the PKI case, you have to trust over 300 Certificate Authorities every time you make an https:// connection, many of which are located in countries with repressive governments. If any of those 300 entities are compromised, malevolent attackers will be able to read your email, access your bank account, and violate your privacy. DNS is being censored by governments simply because they can do it. And every time you store value in currency, you’re trusting a central authority to ensure that it isn’t mismanaged and won’t depreciate in value.

So is there really value in Bitcoin? The answer depends entirely on your perspective. Let me ask a counter question: Is decentralization valuable? If you think we’ll increasingly lose trust in the central authorities that manage the infrastructure we rely on, you might expect Bitcoins to rise a lot in value. If, however, you believe that authorities will be able to tackle the challenges of the future better in centralized form, then from your perspective Bitcoins don’t add value. Time will tell which viewpoint is correct.