Can Transformers Find Logical Rules? | Paulo Pirozelli | Neuro-Symbolic Wednesdays
💬 Discord: https://discord.gg/h8NVzwnysW 💻 GitHub: https://github.com/centaurinstitute 🤝 LinkedIn: https://www.linkedin.com/company/centaur-ai-institute/ 📢 New Initiative: Neuro-Symbolic Agentic Protocol ⭐ Star us on GitHub https://github.com/centaurinstitute/neuro-symbolic-agentic-protocol — 📘 This week, we are going to have Paulo Pirozelli presenting Can Transformers Find Logical Rules? 🦾 Transformer models have become the dominant architecture in natural language processing, underpinning large-scale systems with remarkable performance across a wide range of tasks. However, it remains unclear whether these models genuinely perform logical reasoning or primarily exploit statistical regularities in data. In this presentation, Paulo will discuss the ability of transformer models to follow formal processes of logical inference based on two studies. 🔬 The first, Assessing Logical Reasoning Capabilities of Encoder-Only Transformer Models, investigates whether encoder-only models can apply logical inference rules. The results indicate that, although these models can recognize patterns associated with logical validity, their performance is strongly dependent on the training distribution and shows limited generalization. 🔬 The second study, A Study of Modus Ponens in Transformer Models, isolates the rule of modus ponens to evaluate structured reasoning in a controlled setting. Even in this simplified scenario, the findings suggest that transformer models tend to rely on statistical heuristics and shortcuts rather than performing explicit logical deductions. 🎓 Paulo Pirozelli received his PhD (2018) in Philosophy from the University of São Paulo (USP). He was a postdoctoral researcher at the Center for Artificial Intelligence (C4AI-USP) from 2020 to 2024. He previously worked as a professor at the Instituto Mauá de Tecnologia, teaching courses in Artificial Intelligence and Data Science. He is currently Assistant Professor of Philosophy of Science in the Department of Philosophy at the University of São Paulo. 📃 Paper links: https://link.springer.com/chapter/10.1007/978-3-031-71167-1_2 https://proceedings.mlr.press/v288/pirozelli25a.html — 🗨️ Join us for an interactive session exploring Neuro-Symbolic AI, the emerging paradigm that blends the strengths of neural networks with symbolic reasoning. We will discuss how hybrid approaches can enhance generalization, interpretability, and reasoning, and how these methods are shaping the future of intelligent systems. Whether you’re a researcher, engineer, or simply curious about the cutting edge of AI, you’ll find an engaging space to learn, connect, and exchange ideas. Event Format: 🔍 Reading Group 🎙️ Panel Discussion 🧩 Tutorials/Workshop Series 💬 Research Roundtable 🛠️ Open-Source Project Review 📄 Paper Pitch 🤝 Collaboration Hour 🍵 Networking Mixer #AI #NeuroSymbolic #FutureOfAI