Anthropic Reveals Claude Handles 95% of Internal Business Analysis Tasks with High Accuracy

Anthropic has made public its success in leveraging its Claude AI model for internal business analysis, reporting that the AI now handles 95% of such queries with an impressive 95% accuracy. This revelation, detailed in a recent blog post, highlights a strategic shift from solely relying on model power to implementing robust structural enhancements. The core of this success lies in what Anthropic terms 'Skills.' These are essentially structured folders containing documentation, scripts, and resources that Claude can access and interpret as needed. Before the implementation of this 'Skills' mechanism, Claude's performance in internal evaluations for business analysis tasks was notably low, hovering around 21% accuracy. The introduction of Skills dramatically boosted this to over 95%, with some specific fields even reaching close to 99% accuracy. This approach underscores Anthropic's realization that the primary bottleneck for accurate AI-driven data analysis wasn't the model's inherent intelligence, but rather its access to and understanding of relevant, up-to-date context. The company found that simply giving Claude full-text search capabilities over historical SQL files did not significantly improve accuracy, indicating that raw data access was insufficient without proper structuring. However, maintaining this high level of accuracy is an ongoing engineering challenge. Anthropic observed that the system's accuracy could decay rapidly, dropping from 95% to about 65% within a month if the Skill documents were not regularly updated to reflect changes in the underlying data models. To counter this, the team now treats the maintenance of Skill documents as a critical engineering task, integrating them into the same code repository as the data model to ensure synchronization. This proactive maintenance, along with techniques like adversarial review where Claude challenges its own assumptions, helps sustain the high accuracy, albeit with an increase in token consumption and latency.