GLM 5.2 demonstrates near-human accuracy in VAT return preparation

Performance Evaluation of GLM 5.2

GLM 5.2, an open weights AI model, was evaluated for its effectiveness in preparing quarterly value-added tax (VAT) returns for a small UK business. The VAT process generally requires expertise and can be expensive, with typical fees ranging from £750 to £2,100 per quarter for SMEs. In a notable achievement, GLM 5.2 processed 59 transactions within 68 minutes, with a minimal cost of about $2.73.

Benchmark Methodology

The evaluation utilized data extracted by Claude Fable 5 from accounting software that maintained accurate records of transactions. These records were already prepared by humans, showcasing the distinction in complexity between human and AI processing. In this scenario, the model was primarily given 'user notes,' simplifying its task in comparison to its human counterparts, who had the challenge of locating and verifying relevant invoices.

Accuracy of Results

GLM 5.2 produced a VAT return that was rated nearly perfect; it was off by only 7 pence (~10 US cents) compared to the human-prepared results. This small margin of error is particularly significant because it illustrates the model's potential to perform essential accounting tasks with high precision, equal to that of human bookkeepers.

Implications for Small Businesses

The ability of GLM 5.2 to perform VAT returns accurately at a fraction of the cost indicates a substantial opportunity for small businesses in the UK. By potentially reducing dependency on expensive external accounting services, GLM 5.2 could transform how SMEs manage their tax compliance and contribute to overall business efficiency.