AI Self-Improvement: Taking Constitutional AI further

As I’ve delved deeper into the world of Large Language Models, one peculiar challenge consistently stands out: the leap from knowing something to understanding it. This distinction lies between what’s called ‘declarative’ and ‘procedural’ knowledge. Declarative knowledge is about facts and information, the kind of knowledge that AI is typically quite good at. It’s knowing that Paris is the capital of France or that the Earth orbits the Sun.

Procedural knowledge, on the other hand, is about how to do things. It’s knowing how to ride a bike or how to navigate a complex social interaction. This is where AI often struggles, particularly when it comes to nuanced, context-dependent areas like ethics. Sure, an AI can recite the golden rule, but applying it in a real-world scenario? That’s where things get tricky. And some sort of understanding of ethics is necessary to achieve AI alignment (so that it doesn’t help the user when they are trying to build a dirty bomb or commit suicide, for example).

This is where Constitutional AI comes into the picture, an approach developed by Anthropic that I believe might hold the key to helping AI systems navigate this complex terrain. They used it to train their Claude model (not publicly available yet, as of May 12, 2023) with very interesting results.

Constitutional AI

Constitutional AI, in essence, can be thought of as a self-coaching process for AI systems. The traditional approach to training AI involves human feedback — AI systems rely on humans to rate their responses and guide their learning. Constitutional AI presents an intriguing twist on this concept. Instead of seeking external feedback, AI systems are given a set of principles and trained to look inward, evaluating and refining their own responses.

To provide a more detailed picture, let’s break down this process. When an AI system operating under the Constitutional AI approach generates a response, the process doesn’t stop there. The system prompts itself to reconsider its initial answer, nudging it towards a response that aligns more with being “helpful and harmless”. It then pairs these initial and revised responses and progressively builds a dataset. This dataset acts as a guide for future responses, helping the AI to be more aligned with a nuanced and ethical approach.

This method offers potential advantages in terms of efficiency and control. It’s less costly than traditional reinforcement learning, as it relies less on a large team of human evaluators. Moreover, it provides more control, mitigating the risk of varying interpretations and biases that might be introduced by different human raters.

The Human-AI Divide in Understanding Declarative and Procedural Knowledge

A fascinating divergence between humans and AI lies in how we each grasp and process declarative and procedural knowledge. For us humans, our understanding of procedural knowledge often surpasses our declarative knowledge, especially when it comes to intricate and deeply ingrained subjects like ethics.

From a very young age, we begin to develop an innate understanding of right and wrong. By as early as 19 months, children start showing signs of ethical intuition. Our brains are wired to understand and operate within the complex social and ethical frameworks of our societies. This inherent understanding of ethics is so fundamental to our existence that our legal systems are essentially built upon it. We could even view our legal systems as the bare minimum ethical systems required for functioning within our societies. They operate on the assumption of “legal capacity” — the idea that adults can and should take responsibility for their actions.

When we look at AI, particularly large language models (LLMs), we see a very different picture. These systems begin their ‘lives’ with a vast array of declarative knowledge — facts and information that their creators have painstakingly gathered. Yet, their grasp of procedural knowledge is significantly limited. Given their primary function is text completion, this shouldn’t come as a surprise.

This is where Constitutional AI comes into the picture. It offers a way to leverage the AI’s impressive declarative knowledge to boost its procedural understanding, guiding it to be more “helpful and harmless”. The goal isn’t to create an AI ethical genius, but rather to develop a system that can handle tasks and interactions in a more effective and ethical manner. Constitutional AI, by encouraging AI systems to refine their responses, represents a promising path towards bridging this human-AI divide in understanding declarative and procedural knowledge.

Exploring Applications of Constitutional AI

The potential of Constitutional AI extends far beyond its application to alignment through the lens of ethics. This innovative approach could greatly enhance the procedural understanding of AI systems across diverse fields. Here are two examples:

1. Emotion, Social Skills, and Language Proficiency: Constitutional AI could be instrumental in guiding AI systems to interact more effectively with humans in a wide array of situations. Take, for instance, a scenario where an AI is communicating with an individual with limited knowledge or education — be it a child or a barely literate farmer in a Third World country trying to understand why their chickens are dying and what they can do to help them. While AI systems, like ChatGPT, have vast knowledge and advanced language skills, they often struggle to adjust their language and explanations to the comprehension level of the user. They tend to use complex words and can come off as overly academic, which isn’t helpful in situations that require simplicity and straightforwardness. Constitutional AI could enable these systems to self-evaluate their responses and adjust their language and presentation style to the user’s level of understanding. This would result in a more effective, user-friendly interaction that truly caters to the needs of the individual, regardless of their knowledge level.
2. Creativity and Art: In the realm of art, AI’s capability to replicate various artistic styles and techniques is impressive. However, it often lacks the intuitive understanding of what constitutes “great art”. Constitutional AI could enable AI to act as its own art critic, guiding the system to evaluate and refine its creations based on its extensive knowledge of artistic principles and critiques. Instead of merely reproducing known styles, AI could start generating original works of art that resonate more deeply with human aesthetic sensibilities.

Of course, there are many other domains where this could be applicable.

By harnessing AI’s extensive declarative knowledge to enhance its procedural understanding, we can address some of the current limitations of AI systems. This approach opens up a world of possibilities for the future of AI, promising advancements that could bring AI systems closer to our human way of understanding and interacting with the world.

Related Ideas: BabyAGI, AutoGPT

BabyAGI, or Baby Artificial General Intelligence (and similar ideas, like AutoGPT), operates in an infinite loop, constantly pulling tasks from a list, executing them, enriching the results, and creating new tasks based on the objective and the result of the previous task. Given an objective like “how to solve world hunger,” BabyAGI would engage OpenAI’s GPT-4 to generate a task list towards this objective, execute these tasks, and keep refining and generating new tasks until no new tasks can be generated.

Complementing this, Constitutional AI focuses on self-refinement and improved understanding. It leverages AI’s generative abilities for self-evaluation and refinement of responses, creating a more effective and adaptable AI.

Both BabyAGI and Constitutional AI demonstrate the use of AI to enhance its own efficiency without changing the underlying model architecture. These iterative processes improve AI systems by focusing on problem-solving for BabyAGI and better understanding for Constitutional AI.

The future of AI involves continuous learning and adaptation. Innovations like Constitutional AI and BabyAGI are significant milestones, bringing us closer to creating intelligent, effective, and ethical AI systems.

In conclusion, Constitutional AI is a major advancement in bridging the gap between AI’s knowledge and understanding. By using generative abilities for self-evaluation and response refinement, this approach promises more effective, adaptable, and ethical AI systems. The versatility of ConstitutionalAI extends beyond ethics alignment with potential applications in emotion recognition, social skills development, language proficiency improvement, creativity enhancement, and art exploration. As we refine this approach further it could be instrumental in developing truly smart yet ethical AIs.