OpenAI has introduced an internal evaluation designed to measure how well artificial intelligence models handle practical tasks such as choosing a school after a move, coordinating volunteers and preparing a meal plan.
The company said its GPT-6 Astra model scored 93.1% on a test involving the selection of a kindergarten after relocation. It scored 98% on organizing community volunteers preparing for a flood and 90.4% on creating a meal plan for a parent recently diagnosed with diabetes.
OpenAI presented the results Monday as it competes with Meta, Anthropic and Google to develop AI agents that can carry out multistep activities in the real world. The company’s evaluation is intended to help determine whether ChatGPT can handle errands such as building a grocery cart or completing online forms, rather than only generating text or answering questions.
“Because so many people are using the app, we want to capture all the different use cases and the ways that they’re using it, and really understand like how we’re improving over time.” — Jenny Yang, OpenAI researcher
Yang said the testing is also meant to identify areas where the models still perform poorly and guide improvements. Standardized evaluations are commonly used by AI developers to train and optimize systems for specific capabilities.
Measuring ordinary activities
OpenAI previously released GDPVal, a benchmark focused on work-related skills in fields including manufacturing, government, finance and sales. The new evaluation turns to activities that are less structured and harder to score consistently because they can involve many choices and an unclear sequence of steps.
To develop the tests, OpenAI reviewed surveys about how people use their time, including the American Time Use Survey from the Bureau of Labor Statistics and the United Kingdom’s Multinational Time Use Survey. The company also asked ChatGPT users what they would want an AI assistant to do. Activities such as sleeping and eating were excluded because they are not tasks an AI system can perform for a user.
- 93.1%: GPT-6 Astra’s score on selecting a kindergarten after a move
- 98%: score on managing volunteers preparing for a flood
- 90.4%: score on creating a meal plan for a parent with a recent diabetes diagnosis
The benchmark arrives as AI companies face pressure to show that increasingly capable models can perform useful actions reliably while also addressing safety concerns and the cost of developing and operating the systems. OpenAI said the evaluation will help it track how its models improve across the varied ways people use ChatGPT.
