Neuro-symbolic AI emerges as powerful new approach

He thinks other ongoing efforts to add features to deep neural networks that mimic human abilities such as attention offer a better way to boost AI’s capacities. We investigate an unconventional direction of research that aims at converting neural networks, a class of distributed, connectionist, sub-symbolic models into a symbolic level with the ultimate goal of achieving AI interpretability and safety. To that end, we propose Object-Oriented Deep Learning, a novel computational paradigm of deep learning that adopts interpretable “objects/symbols” as a basic representational atom instead of N-dimensional tensors (as in traditional “feature-oriented” deep learning). For visual processing, each “object/symbol” can explicitly package common properties of visual objects like its position, pose, scale, probability of being an object, pointers to parts, etc., providing a full spectrum of interpretable visual knowledge throughout all layers. It achieves a form of “symbolic disentanglement”, offering one solution to the important problem of disentangled representations and invariance.

Since some of the weaknesses of neural nets are the strengths of symbolic AI and vice versa, neurosymbolic AI would seem to offer a powerful new way forward. Roughly speaking, the hybrid uses deep nets to replace humans in building the knowledge base and propositions that symbolic AI relies on. It harnesses the power of deep nets to learn about the world from raw data and then uses the symbolic components to reason about it. Each of the hybrid’s parents has a long tradition in AI, with its own set of strengths and weaknesses.

Symbolic artificial intelligence

Take, for example, a neural network tasked with telling apart images of cats from those of dogs. The image — or, more precisely, the values of each pixel in the image — are fed to the first layer of nodes, and the final layer of nodes produces as an output the label “cat” or “dog.” The network has to be trained using pre-labeled images of cats and dogs. During training, the network adjusts the strengths of the connections between its nodes such that it makes fewer and fewer mistakes while classifying the images. Ducklings exposed to two similar objects at birth will later prefer other similar pairs. If exposed to two dissimilar objects instead, the ducklings later prefer pairs that differ. Ducklings easily learn the concepts of “same” and “different” — something that artificial intelligence struggles to do.

• All operations are executed in an input-driven fashion, thus sparsity and dynamic computation per sample are naturally supported, complementing recent popular ideas of dynamic networks and may enable new types of hardware accelerations.
• For almost any type of programming outside of statistical learning algorithms, symbolic processing is used; consequently, it is in some way a necessary part of every AI system.
• In the emulated duckling example, the AI doesn’t know whether a pyramid and cube are similar, because a pyramid doesn’t exist in the knowledge base.
• Research problems include how agents reach consensus, distributed problem solving, multi-agent learning, multi-agent planning, and distributed constraint optimization.
• LNN performs necessary reasoning such as type-based and geographic reasoning to eventually return the answers for the given question.

Prolog provided a built-in store of facts and clauses that could be queried by a read-eval-print loop. The store could act as a knowledge base and the clauses could act as rules or a restricted form of logic. As a subset of first-order logic Prolog was based on Horn clauses with a closed-world assumption—any facts not known were considered false—and symbolic ai a unique name assumption for primitive terms—e.g., the identifier barack_obama was considered to refer to exactly one object. These dynamic models finally enable to skip the preprocessing step of turning the relational representations, such as interpretations of a relational logic program, into the fixed-size vector (tensor) format.

Part I Explainable Artificial Intelligence — Part II

This is important because all AI systems in the real world deal with messy data. For example, in an application that uses AI to answer questions about legal contracts, simple business logic can filter out data from documents that are not contracts or that are contracts in a different domain such as financial services versus real estate. “This is a prime reason why language is not wholly solved by current deep learning systems,” Seddiqi said. Our researchers are working to usher in a new era of AI where machines can learn more like the way humans do, by connecting words with images and mastering abstract concepts.

LISP is the second oldest programming language after FORTRAN and was created in 1958 by John McCarthy. LISP provided the first read-eval-print loop to support rapid program development. Program tracing, stepping, and breakpoints were also provided, along with the ability to change values or functions and continue from breakpoints or errors. It had the first self-hosting compiler, meaning that the compiler itself was originally written in LISP and then ran interpretively to compile the compiler code. These old-school parallels between individual neurons and logical connectives might seem outlandish in the modern context of deep learning.

Agents and multi-agent systems

Our initial results are encouraging – the system achieves state-of-the-art accuracy on two datasets with no need for specialized training. The two biggest flaws of deep learning are its lack of model interpretability (i.e. why did my model make that prediction?) and the large amount of data that deep neural networks require in order to learn. Symbolic AI’s adherents say it more closely follows the logic of biological intelligence because it analyzes symbols, not just data, to arrive at more intuitive, knowledge-based conclusions. It’s most commonly used in linguistics models such as natural language processing (NLP) and natural language understanding (NLU), but it is quickly finding its way into ML and other types of AI where it can bring much-needed visibility into algorithmic processes. The concept of neural networks (as they were called before the deep learning “rebranding”) has actually been around, with various ups and downs, for a few decades already. It dates all the way back to 1943 and the introduction of the first computational neuron [1].

As far back as the 1980s, researchers anticipated the role that deep neural networks could one day play in automatic image recognition and natural language processing. It took decades to amass the data and processing power required to catch up to that vision – but we’re finally here. Similarly, scientists have long anticipated the potential for symbolic AI systems to achieve human-style comprehension. And we’re just hitting the point where our neural networks are powerful enough to make it happen. We’re working on new AI methods that combine neural networks, which extract statistical structures from raw data files – context about image and sound files, for example – with symbolic representations of problems and logic. By fusing these two approaches, we’re building a new class of AI that will be far more powerful than the sum of its parts.

Approaches

Symbolic artificial intelligence, also known as Good, Old-Fashioned AI (GOFAI), was the dominant paradigm in the AI community from the post-War era until the late 1980s. “As impressive as things like transformers are on our path to natural language understanding, they are not sufficient,” Cox said. Deep learning is better suited for System 1 reasoning,  said Debu Chatterjee, head of AI, ML and analytics engineering at ServiceNow, referring to the paradigm developed by the psychologist Daniel Kahneman in his book Thinking Fast and Slow.

Hobbes was influenced by Galileo, just as Galileo thought that geometry could represent motion, Furthermore, as per Descartes, geometry can be expressed as algebra, which is the study of mathematical symbols and the rules for manipulating these symbols. A different way to create AI was to build machines that have a mind of its own. René Descartes, a mathematician, and philosopher, regarded thoughts themselves as symbolic representations and Perception as an internal process. While this may be unnerving to some, it must be remembered that symbolic AI still only works with numbers, just in a different way. By creating a more human-like thinking machine, organizations will be able to democratize the technology across the workforce so it can be applied to the real-world situations we face every day. Controversies arose from early on in symbolic AI, both within the field—e.g., between logicists (the pro-logic “neats”) and non-logicists (the anti-logic “scruffies”)—and between those who embraced AI but rejected symbolic approaches—primarily connectionists—and those outside the field.

Further Reading on Symbolic AI

What the ducklings do so effortlessly turns out to be very hard for artificial intelligence. This is especially true of a branch of AI known as deep learning or deep neural networks, the technology powering the AI that defeated the world’s Go champion Lee Sedol in 2016. Such deep nets can struggle to figure out simple abstract relations between objects and reason about them unless they study tens or even hundreds of thousands of examples. Language is a type of data that relies on statistical pattern matching at the lowest levels but quickly requires logical reasoning at higher levels. Pushing performance for NLP systems will likely be akin to augmenting deep neural networks with logical reasoning capabilities.

This is fed to another neural network, which learns to analyze the movements of these objects and how they interact with each other and can predict the motion of objects and collisions, if any. The other two modules process the question and apply it to the generated knowledge base. The team’s solution was about 88 percent accurate in answering descriptive questions, about 83 percent for predictive questions and about 74 percent for counterfactual queries, by one measure of accuracy. It contained 100,000 computer-generated images of simple 3-D shapes (spheres, cubes, cylinders and so on).

Problems with Relational-Level NSI

Cyc has attempted to capture useful common-sense knowledge and has “micro-theories” to handle particular kinds of domain-specific reasoning. Forward chaining inference engines are the most common, and are seen in CLIPS and OPS5. Backward chaining occurs in Prolog, where a more limited logical representation is used, Horn Clauses. The logic clauses that describe programs are directly interpreted to run the programs specified. No explicit series of actions is required, as is the case with imperative programming languages.

This entire process is akin to generating a knowledge base on demand, and having an inference engine run the query on the knowledge base to reason and answer the question. For almost any type of programming outside of statistical learning algorithms, symbolic processing is used; consequently, it is in some way a necessary part of every AI system. Indeed, Seddiqi said he finds it’s often easier to program a few logical rules to implement some function than to deduce them with machine learning.

We chose to focus on KBQA because such tasks truly demand advanced reasoning such as multi-hop, quantitative, geographic, and temporal reasoning. It is one form of assumption, and a strong one, while deep neural architectures contain other assumptions, usually about how they should learn, rather than what conclusion they should reach. The ideal, obviously, is to choose assumptions that allow a system to learn flexibly and produce accurate decisions about their inputs. This creates a crucial turning point for the enterprise, says Analytics Week’s Jelani Harper.