The Socratic Method of Large Language Models
During my time at university, philosophy students were sometimes the subject of low level, friendly ridicule. We were accused of acquiring impractical skills, lost in a world of abstract ideas with little bearing on the so-called “real world.” Fast forward to today, and it seems that the tables have turned. In a world where large language models (LLMs) like ChatGPT are becoming increasingly prominent, the skills I gained through my philosophical education (even though I never found the patience to actually write my Master’s thesis) have proven to be invaluable in navigating and making the most of these powerful tools.
In this blog post, let’s explore how the Socratic method and other philosophical skills can be effectively applied when interacting with LLMs. Instead of focusing on the practicality of a philosophy education, we will delve into specific ways that philosophical training can be harnessed to maximize the benefits we reap from LLMs, as well as to enhance our self-awareness and critical thinking.
Applying the Socratic method to squeeze more insight from the models and ourselves
When engaging with philosophical texts, our ultimate goal should be not merely to understand or internalize the author’s ideas but to actively challenge them. We should seek out the weakest parts of their arguments, identify instances of philosophical sleight of hand, and relentlessly scrutinize even the most revered thinkers. The uninspired philosophy student will talk about how we should rely on reasoning to understand the world as we cannot trust our senses, and how skepticism is the only valid approach after reading Meditations on First Philosophy. The inquisitive and engaged student, on the other hand, will contemplate mind-body dualism, consider how Descartes’ reasoning about God might result in a circular argument, and explore numerous other methods to identify weaknesses in the argumentation. (Of course, this means no disrespect to Rene Descartes; it’s just how the game of “philosophy” is really played — by pointing out flaws in argumentation. If one is after making fuss over intellectual authorities, the theology department is in the other building.)
This rigorous approach equips us with the tools necessary to employ the Socratic method, a favorite strategy in the philosopher’s repertoire. The Socratic method is a dialectical technique that uses questioning and critical reasoning to stimulate critical thinking, uncover contradictions or inconsistencies, and ultimately deepen our comprehension of complex ideas. A prime example of the Socratic method in action can be found in Plato’s dialogue, “Phaedo.” In this dialogue, Socrates engages in a discussion about the immortality of the soul, questioning his interlocutors about their beliefs and assumptions. By asking probing questions and dissecting their responses, Socrates exposes inconsistencies in their arguments, guiding them towards a deeper understanding of the nature of the soul and its immortality.
In the same way Socrates used his method on Simmias or Euthyphro to gain deeper insight, we can use it on large language models like ChatGPT to help us extract more profound perspectives. By challenging their outputs and demanding further elaboration, we can reveal nuances and subtleties that a surface-level response might miss.
The Socratic method, while commonly employed to question and challenge the ideas of others, also serves as a powerful tool for self-examination and personal growth. To use LLMs for introspection, we can ask them to assist us in achieving better outcomes. This approach often entails ending prompts with a request for the model to ask questions that would help refine its response, fostering a more interactive and dynamic exchange. The model’s inquiries can highlight areas where our explanations or arguments may be unclear or incomplete, offering valuable feedback to enhance our understanding and communication of ideas. This principle applies not only to philosophical discussions but also to situations such as pitching a startup idea to a VC or persuading our boss to adopt our proposal.
Another way to use LLMs for this purpose is by presenting our thoughts or beliefs to the model and asking it to pose critical questions, pushing us to think more deeply about our views. This process can reveal underlying assumptions, inconsistencies, or gaps in our reasoning, helping us become more aware of the foundations of our beliefs and prompting us to question them further.
For example, we might ask ChatGPT to challenge our position on a controversial topic or to explore potential counterarguments to an idea we hold dear. By engaging with the model in this manner, we expose ourselves to alternative perspectives, encouraging us to think critically and consider our views in a broader context.
In essence, using large language models to apply the Socratic method on ourselves allows us to harness the power of these tools for self-improvement and intellectual growth. By challenging our own beliefs and ideas through thoughtful questioning and reflection, we can strengthen our reasoning, broaden our perspectives, and deepen our understanding of the world around us.
Transforming critique into a constructive process
In a philosophy class, students might spend hours debating the ideas of Wittgenstein, attempting to find flaws or inconsistencies in his arguments. However, Wittgenstein himself is unlikely to respond to these critiques, so the exercise, while intellectually stimulating, can feel at times futile. In contrast, when we critique ChatGPT’s output, the model has the ability to respond, learn, and adapt according to our feedback.
This dynamic interaction allows us to guide the model towards producing content that aligns with our objectives, while also improving its understanding of the subject matter. By pointing out what we don’t like about the model’s output, we provide valuable input that the model can use to refine its responses. In this way, the critique becomes a creative act, shaping the direction of the conversation and pushing the model to generate better content. Moreover, unlike with most human interlocutors, we don’t need to care about hurting the model’s feelings. It will always incorporate our feedback to the best of its abilities, without breaking its cheerful and enthusiastic demeanor.
In conclusion, transforming critique into a constructive process when engaging with large language models adds a creative dimension to our philosophical endeavors. By guiding these models with our feedback, we can harness their potential and foster a collaborative relationship that enhances both our understanding and the model’s performance.
Precision
Clarity of thought and precision in language are highly valued in the world of philosophy, particularly among analytical philosophers. When I was a student I attended a seminar by Professor Jacek Juliusz Jadacki. He believed himself an intellectual descendant of the Lvov-Warsaw school of analytical philosophy, whose philosophers were famous for their focus on clarity in their writing (even if in some cases this might have been merely aspirational). In this class, a designated note taker was tasked with creating a summary of the proceedings, which was read aloud during the next class. However, before that the summary needed to pass Jadacki’s meticulous review, who was dead set to ensure that every word was ideally chosen and every punctuation mark placed correctly. Though this exercise may have felt futile at the time (some people were questioning if the level of insight in these discussions merited all this effort), it instilled in us a deep appreciation for the power of precise language — if only as a way to be off the hook for an assignment.
When working with LLMs, precise communication is invaluable for guiding models towards accurate and insightful responses. Crafting clear, specific prompts and providing pinpointed feedback allows the model to adapt and grow effectively. Applying rigorous standards of clarity and precision, especially when discussing complex or abstract ideas, ensures fruitful, accurate, and intellectually stimulating exchanges with LLMs.
Dealing with the model’s biases
In philosophical texts, biases often arise from the author’s cultural, social, or historical context, as well as from their personal beliefs and values. For example, Aristotle made patently false claims, such as stating that women have fewer teeth than men, while Immanuel Kant held various racist and sexist views. However, it would be unwise to dismiss the entirety of Kant’s work due to his misguided notions of gender or racial hierarchies. (In Kant’s defense, it’s worth noting that late in life, around his 70th birthday, Kant abandoned his thesis of racial hierarchy and began to criticize European colonialism. He never made parallel revisions to his account of the status of women, sadly.) When philosophizing, one must learn to identify these biases and separate them from the core ideas, allowing us to extract valuable insights and intellectual tools from the texts.
Similarly, large language models like ChatGPT are not immune to biases. They inherit biases from the data they were trained on, reflecting the cultural, historical, and cognitive biases present in that data. When engaging with LLM-generated content, philosophical skills enable us to detect and address these biases, questioning the underlying assumptions and preventing distortions from affecting our understanding.
Conclusion
The paradoxical nature of large language models like ChatGPT, simultaneously powerful and weak, serves as a reminder that they are tools for the mind — some of the most potent intuition pumps ever invented, to borrow Dennett’s metaphor. While they possess the ability to generate impressive content, they can also produce superficial output, taking the path of intellectual least resistance. However, when we employ the skills and techniques honed in the study of philosophy, we can transform these models into valuable resources for personal growth and intellectual development.
By applying the Socratic method and other philosophical strategies, we can prompt LLMs in more inspired ways, eliciting deeper insights and fostering collaboration between users and models. Precision in language further refines model-generated content, while the ability to identify and address biases allows us to extract valuable knowledge from a diverse array of sources. Embracing these philosophical techniques enables us to harness the full potential of LLMs, enhancing our understanding of the world around us across all disciplines, not just philosophy.
