Evaluating AI Capabilities as Affordances

About This Event

Sina Fazelpour (Northeastern University) will give a talk entitled, "Evaluating AI Capabilities as Affordances: Implications for Decision Support and Redteaming."

The rapid organizational uptake of generative AI models has focused research and regulatory attention on evaluating the capabilities and limitations of these tools. For example, the recent Presidential Executive Order has called upon the National Institute of Standards and Technology to create guidelines for evaluating and auditing AI capabilities and risks. Yet, standard approaches that focus on assessing static model properties on benchmark tasks and datasets are insufficient for achieving this task. In this talk, after highlighting some of the key assumptions and choices that underpin standard evaluations, Fazelpour will propose a complementary affordance-based approach in terms of the relational dynamics of technology, users, and environments. Fazelpour will apply the framework to the evaluation of predictive models used as decision support as well as to red-teaming evaluations of generative AI. This will show how this perspective can offer a more functionally and ethically robust approach to AI evaluation.

Featured Guests

Sina Fazelpour (Northeastern University)

Co-sponsors

• Department of Philosophy, University of Rochester
• Goergen Institute of Data Science, University of Rochester