728 x 90

Agentic profiles for effective AI governance – Nature

Agentic profiles for effective AI governance – Nature

Feigenbaum, E. A. The art of artificial intelligence: themes and case studies of knowledge engineering. In Proc. 5th International Joint Conference on Artificial Intelligence (IJCAI ’77) Vol. 2, 1014–1029 (Morgan Kaufmann, 1977). Russell, S. J. & Norvig, P. Artificial Intelligence: A Modern Approach (Prentice Hall, 1995). Sutton, R. S. & Barto, A. Reinforcement Learning: An

  • Feigenbaum, E. A. The art of artificial intelligence: themes and case studies of knowledge engineering. In Proc. 5th International Joint Conference on Artificial Intelligence (IJCAI ’77) Vol. 2, 1014–1029 (Morgan Kaufmann, 1977).

  • Russell, S. J. & Norvig, P. Artificial Intelligence: A Modern Approach (Prentice Hall, 1995).

  • Sutton, R. S. & Barto, A. Reinforcement Learning: An Introduction (MIT Press, 1998).

  • Wooldridge, M. in Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence (ed. Weiss, G.) 27–79 (MIT Press, 1999).

  • Sumers, T. R., Yao, S., Narasimhan, K. & Griffiths, T. L. Cognitive architectures for language agents. In Trans. Machine Learning Research (2024).

  • Yee, L., Chui, M. & Roberts, R. Why agents are the next frontier of generative AI. McKinsey & Company https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/why-agents-are-the-next-frontier-of-generative-ai/ (2024).

  • Salesforce. Salesforce unveils Agentforce–what AI was meant to be. Salesforce https://www.salesforce.com/uk/news/press-releases/2024/09/12/agentforce-announcement/ (2024).

  • Spataro, J. New autonomous agents scale your team like never before. The Official Microsoft Blog https://blogs.microsoft.com/blog/2024/10/21/new-autonomous-agents-scale-your-team-like-never-before/ (2024).

  • Casper, S. et al. The AI agent index. Preprint at https://doi.org/10.48550/arXiv.2502.01635 (2025).

  • Gabriel, I. et al. The ethics of advanced AI assistants. Preprint at https://doi.org/10.48550/arXiv.2404.16244 (2024).

  • Kirk, H. R., Gabriel, I., Summerfield, C., Vidgen, B. & Hale, S. A. Why human–AI relationships need socioaffective alignment. Humanit. Soc. Sci. Commun. 12, 728 (2025).

    Article 

    Google Scholar 

  • Chan, A. et al. Harms from increasingly agentic algorithmic systems. In Proc. 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’23) 651–666 (Association for Computing Machinery, 2023).

  • Uuk, R. et al. A taxonomy of systemic risks from general-purpose AI. Preprint at https://doi.org/10.48550/arXiv.2412.07780 (2024).

  • Kasirzadeh, A. Two types of AI existential risk: decisive and accumulative. Philos. Stud. 182, 1975–2003 (2025).

    Article 

    Google Scholar 

  • Weidinger, L. et al. Taxonomy of risks posed by language models. In Proc. 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’22) 214–229 (Association for Computing Machinery, 2022).

  • Bird, C., Ungless, E., & Kasirzadeh, A. Typology of risks of generative text-to-image models. In Proc. 2023 AAAI/ACM Conference on AI, Ethics, and Society (AIES ’23) 396–410 (Association for Computing Machinery, 2023).

  • Hammond, L. et al. Multi-agent risks from advanced AI. Preprint at https://doi.org/10.48550/arXiv.2502.14143 (2025).

  • Kolt, N. Governing AI agents. Notre Dame Law Rev. https://doi.org/10.2139/ssrn.4772956 (2025).

  • Kraprayoon, J., Williams, Z., & Fayyaz, R. AI agent governance: a field guide. Preprint at https://doi.org/10.48550/arXiv.2505.21808 (2025).

  • Chan, A. et al. Visibility into AI agents. In Proc. 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’24) 958–973 (Association for Computing Machinery, 2024).

  • Chan, A. et al. Infrastructure for AI agents. In Trans. Machine Learning Research (2025).

  • SAE International. Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles. SAE International https://www.sae.org/standards/content/j3016_202104 (2021).

  • Council of the European Union. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act). Eur-LEX http://data.europa.eu/eli/reg/2024/1689/oj (2024).

  • Tabassi, E. Artificial intelligence risk management framework (AI RMF 1.0). National Institute of Standards and Technology http://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf (2023).

  • Goldman, A. I. A Theory of Human Action (Prentice Hall, 1970).

  • Davidson, D. I. in Agent, Action, and Reason (eds Binkley, R. W., Bronaugh, R. N. & Marras, A.) 1–37 (Univ. Toronto Press, 1971).

  • Dretske, F. Explaining Behavior: Reasons in a World of Causes (MIT Press, 1988).

  • Dung, L. Understanding artificial agency. Philos. Q. 75, 450–472 (2025).

    Article 

    Google Scholar 

  • Ginet, C. On Action (Cambridge Univ. Press, 1990).

  • O’Connor, T. Persons and Causes: The Metaphysics of Free Will (Oxford Univ. Press, 2002).

  • Lowe, E. J. Personal Agency: The Metaphysics of Mind and Action (Oxford Univ. Press, 2008).

  • Frankfurt, H. G. Freedom of the will and the concept of a person. J. Philos. 68, 5–20 (1971).

    Article 

    Google Scholar 

  • Taylor, C. in The Self: Psychological and Philosophical Issues (ed. Mischel, T.) 103–135 (Blackwell, 1977).

  • Grossman, S. J., & Hart, O. D. in Foundations of Insurance Economics (eds Dionne, G. & Harrington, S. E.) 302–340 (Springer, 1992).

  • Kauffman, S. & Clayton, P. On emergence, agency, and organization. Biol. Philos. 21, 501–521 (2006).

    Article 

    Google Scholar 

  • Bouvard, V. et al. A review of human carcinogens—part B: biological agents. Lancet Oncol. 10, 321–322 (2009).

    Article 
    PubMed 

    Google Scholar 

  • Meincke, A. S. in Philosophy of Science (eds Christian, A., Hommen, D., Retzlaff, N. & Schurz, G.) 65–93 (Springer, 2018).

  • Damşa, C. I., Kirschner, P. A., Andriessen, J. E. B., Erkens, G. & Sins, P. H. M. Shared epistemic agency: an empirical study of an emergent construct. J. Learn. Sci. 19, 143–186 (2010).

    Article 

    Google Scholar 

  • Elgin, C. Z. Epistemic agency. Theory Res. Educ. 11, 135–152 (2013).

    Article 

    Google Scholar 

  • Sosa, E. Epistemic agency. J. Philos. 110, 585–605 (2013).

    Article 

    Google Scholar 

  • Nickel, J. W. in Ethnicity and Group Rights (eds Shapiro, I. & Kymlicka, W.) 235–256 (New York Univ. Press, 1997).

  • List, C. & Pettit, P. Group Agency: The Possibility, Design, and Status of Corporate Agents (Oxford Univ. Press, 2011).

  • Tollefsen, D. Groups as Agents (Polity, 2015).

  • Bratman, M. Shared Agency: A Planning Theory of Acting Together (Oxford Univ. Press, 2014).

  • Shapiro, S. J. in Rational and Social Agency: The Philosophy of Michael Bratman (eds Vargas, M. & Yaffe, G.) 257–293 (Oxford Univ. Press, 2014).

  • Le Besnerais, A., Moore, J. W., Berberian, B. & Grynszpan, O. Sense of agency in joint action: a critical review of we-agency. Front. Psychol. 15, 1331084 (2024).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Russell, S. J. Rationality and intelligence. Artif. Intell. 94, 57–77 (1997).

    Article 

    Google Scholar 

  • Rao, A. S. & Wooldridge, M. in Foundations of Rational Agency (eds Wooldridge, M. & Rao, A.) 1–10 (Springer, 1999).

  • Silver, D. et al. Mastering the game of Go without human knowledge. Nature 550, 354–359 (2017).

    Article 
    ADS 
    CAS 
    PubMed 

    Google Scholar 

  • Brown, N. & Sandholm, T. Superhuman AI for multiplayer poker. Science 365, 885–890 (2019).

    Article 
    ADS 
    MathSciNet 
    CAS 
    PubMed 

    Google Scholar 

  • Meta Fundamental AI Research Diplomacy Team (FAIR) et al. Human-level play in the game of Diplomacy by combining language models with strategic reasoning. Science 378, 1067–1074 (2022).

    Article 
    ADS 
    MathSciNet 

    Google Scholar 

  • Lake, B. M., Ullman, T. D., Tenenbaum, J. B. & Gershman, S. J. Building machines that learn and think like people. Behav. Brain Sci. 40, e253 (2017).

    Article 
    PubMed 

    Google Scholar 

  • Park, J. S. et al. Generative agents: interactive simulacra of human behavior. In UIST ’23: Proc. 36th Annual ACM Symposium on User Interface Software and Technology https://doi.org/10.1145/3586183.3606763 (ACM, 2023).

  • Wang, X. & Zhou, D. Chain-of-thought reasoning without prompting. Adv. Neural Inf. Process. Syst. 37, 66383–66409 (2024).

    Article 

    Google Scholar 

  • Zelikman, E., Wu, Y., Mu, J. & Goodman, N. D. STaR: self-taught reasoner bootstrapping reasoning with reasoning. In Proc. 36th International Conference on Neural Information Processing Systems (NIPS’22) 15476–15488 (Curran Associates Inc., 2022).

  • Anthropic. Claude 3.5 Sonnet Model Card Addendum https://www-cdn.anthropic.com/fed9cc193a14b84131812372d8d5857f8f304c52/Model_Card_Claude_3_Addendum.pdf (2024).

  • Kavukcuoglu, K. Gemini 2.5: Our most intelligent AI model. Google https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025/ (2025).

  • Jaech, A. et al. OpenAI o1 system card. Preprint at https://doi.org/10.48550/arXiv.2412.16720 (2024).

  • Berlin, I. in Four Essays On Liberty 118–172 (Oxford Univ. Press, 1969).

  • Dennett, D. C. Freedom Evolves (Viking, 2003).

  • Vagia, M., Transeth, A. A. & Fjerdingen, S. A. A literature review on the levels of automation during the years. What are the different taxonomies that have been proposed? Appl. Ergon. 53, 190–202 (2016).

    Article 
    PubMed 

    Google Scholar 

  • Klyubin, A. S., Polani, D. & Nehaniv, C. L. Empowerment: a universal agent-centric measure of control. In Proc. 2005 IEEE Congress on Evolutionary Computation 128–135 (IEEE, 2005).

  • Chalmers, D. J. Reality+: Virtual Worlds and the Problems of Philosophy (Allen Lane, 2022).

  • Spirtes, P., Glymour, C. N., & Scheines, R. Causation, Prediction, and Search (MIT Press, 2000).

  • Pearl, J. Causality: Models, Reasoning, and Inference 2nd edn (Cambridge Univ. Press, 2009).

  • Bareinboim, E., Correa, J. D., Ibeling, D., & Icard, T. in Probabilistic and Causal Inference: The Works of Judea Pearl (eds Geffner, H., Dechter, R. & Halpern, J. Y.) 507–556 (Association for Computing Machinery, 2022).

  • Jaeger, J., Riedl, A., Djedovic, A., Vervaeke, J. & Walsh, D. Naturalizing relevance realization: why agency and cognition are fundamentally not computational. Front. Psychol. 15, 1362658 (2024).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Sacerdoti, E. D. Planning in a hierarchy of abstraction spaces. Artif. Intell. 5, 115–135 (1974).

    Article 

    Google Scholar 

  • Georgievski, I. & Aiello, M. HTN planning: overview, comparison, and beyond. Artif. Intell. 222, 124–156 (2015).

    Article 

    Google Scholar 

  • Kwa, T. et al. Measuring AI ability to complete long software tasks. Preprint at https://doi.org/10.48550/arXiv.2503.14499 (2026).

  • Deb, K., Sindhya, K. & Hakanen, J. in Decision Sciences: Theory and Practice (eds Sengupta, R., Gupta, A. & Dutta, J.) 145–184 (CRC Press, 2016).

  • Li, M. & Vitányi, P. An Introduction to Kolmogorov Complexity and Its Applications (Springer, 2019).

  • Shannon, C. E. A mathematical theory of communication. Bell Syst. Tech. J 27, 379–423 (1948).

    Article 
    ADS 
    MathSciNet 

    Google Scholar 

  • Papadimitriou, C. H. in Encyclopedia of Computer Science (eds Ralston, A., Reilly, E. D. & Hemmendinger, D.) 260–265 (Wiley, 2003).

  • Holland, J. H. in Complexity and Industrial Clusters (eds Curzio, A. Q. & Fortis, M.) 25–34 (Physica, 2002).

  • Legg, S. & Hutter, M. Universal intelligence: a definition of machine intelligence. Minds Mach. 17, 391–444 (2007).

    Article 

    Google Scholar 

  • Goertzel, B. Artificial general intelligence: concept, state of the art, and future prospects. J. Artif. Gen. Intell. 5, 1–48 (2014).

    Google Scholar 

  • Hutter, M. Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability (Springer, 2005).

  • Morris, M. R. et al. Position: Levels of AGI for operationalizing progress on the path to AGI. In Proc. 41st International Conference on Machine Learning (ICML’24) 36308–36321 (JMLR.org, 2024).

  • Ehtesham, A., Singh, A., Gupta, G. K., & Kumar, S. A survey of agent interoperability protocols: Model Context Protocol (MCP), Agent Communication Protocol (ACP), Agent-to-Agent Protocol (A2A), and Agent Network Protocol (ANP). Preprint at https://doi.org/10.48550/arXiv.2505.02279 (2025).

  • Anderljung, M. et al. Frontier AI regulation: managing emerging risks to public safety. Preprint at https://doi.org/10.48550/arXiv.2307.03718 (2023).

  • Kasirzadeh, A. Measurement Challenges in AI Catastrophic Risk Governance and Safety Frameworks (Tech Policy Press, 2024).

  • Luo, Z., Kasirzadeh, A., & Shah, N. B. The more you automate, the less you see: hidden pitfalls of AI scientist systems. Preprint at https://doi.org/10.48550/arXiv.2509.08713 (2025).

  • Bowman, S. R. et al. Measuring progress on scalable oversight for large language models. Preprint at https://doi.org/10.48550/arXiv.2211.03540 (2022).

  • Carpenter, J. in Living with Robots: Emerging Issues on the Psychological and Social Implications of Robotics (eds Park, R. de Visser, E. J. & Rovira, E.) 75–90 (Academic, 2020).

  • Bereska, L. & Gavves, E. Mechanistic interpretability for AI safety – a review. Preprint at https://doi.org/10.48550/arXiv.2404.14082 (2024).

  • Hacker, P., Edwards, L. & Kasirzadeh, A. AI, digital platforms, and the new systemic risk. In Proc. 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26) https://doi.org/10.1145/3805689.3806506 (ACM, 2026).

  • Frey, C. B. & Osborne, M. A. The future of employment: how susceptible are jobs to computerisation? Technol. Forecast. Soc. Change 114, 254–280 (2017).

    Article 

    Google Scholar 

  • Webb, M. The impact of artificial intelligence on the labor market. SSRN Scholarly Paper at https://doi.org/10.2139/ssrn.3482150 (2020).

  • Tyler, S. et al. Use of artificial intelligence in triage in hospital emergency departments: a scoping review. Cureus 16, e59906 (2024).

    PubMed 
    PubMed Central 

    Google Scholar 

  • Salge, C., Glackin, C., & Polani, D. in Guided Self-Organization: Inception (ed. Prokopenko, M.) 67–114 (Springer, 2014).

  • Luccioni, S., Jernite, Y. & Strubell, E. In Proc. 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’24) 85–99 (Association for Computing Machinery, 2024).

  • Franklin, S. & Graesser, A. in Intelligent Agents III Agent Theories, Architectures, and Languages (eds Müller, J. P., Wooldridge, M. J. & Jennings, N. R.) 21–35 (Springer, 1997).

  • Shavit, Y., et al. Practices for governing agentic AI systems. Research Paper, OpenAI https://cdn.openai.com/papers/practices-for-governing-agentic-ai-systems.pdf (2023).

  • Masterman, T., Besen, S., Sawtell, M. & Chao, A. The landscape of emerging AI agent architectures for reasoning, planning, and tool calling: a survey. Preprint at https://doi.org/10.48550/arXiv.2404.11584 (2024).

  • Huang, X. et al. Understanding the planning of LLM agents: a survey. Preprint at https://doi.org/10.48550/arXiv.2402.02716 (2024).

  • Li, J. et al. EvoCodeBench: an evolving code generation benchmark with domain-specific evaluations. Adv. Neural Inf. Process. Syst. 37, 57619–57641 (2024).

    Article 

    Google Scholar 

  • Sharkey, L. et al. Open problems in mechanistic interpretability. Preprint at https://doi.org/10.48550/arXiv.2501.16496 (2025).

  • For more tech updates, stay tuned to our blog.

    Posts Carousel

    Latest Posts

    Top Authors

    Most Commented

    Featured Videos