Produced in partnership with Northern Trust Asset Management.

More than three-quarters of the world’s data is textual – company filings, earnings call transcripts, financial news, press releases and social media – and the promise of natural language processing (NLP) is that machines can now read nearly all of it. But according to Northern Trust Asset Management, the barriers to converting that vast body of unstructured information into durable alpha are steeper than they appear, and asset owners should apply due diligence to any manager claiming an NLP edge.

Markets have always run on words; what has changed is who – or what – does the reading. For most of financial history that work fell to analysts, one document at a time. The first generation of machine reading counted words against sentiment dictionaries, most famously the finance-specific lexicon built by academics Tim Loughran and Bill McDonald, which recognised that a word like “liability” carries no negative signal in an annual report. Neural embeddings later allowed models to capture meaning and similarity rather than raw word counts, and large language models (LLMs) have pushed machine comprehension further again.

Each wave has lowered the barrier to entry – but that, Northern Trust argues, is precisely the problem.

“The advantage does not come from access to textual data, but from the quality, sensibility and intuition behind the economic questions you ask and the rigor with which you test the answers,” the firm says in a recent report.

Raw inputs – news feeds, transcripts and filings – are widely available, and the models increasingly commoditised, and anyone with basic coding skills can point an AI agent at publicly available company information.

Three traps

The report draws on a recent paper in the Journal of Portfolio Management – ‘Natural Language Processing for Asset Managers: Turning Text into Alpha’, authored by Northern Trust Asset Management’s quantitative researchers – which, the firm says, identifies the structural challenges that make text a treacherous hunting ground for quantitative investors.

Overfitting is endemic: text datasets have enormous dimensionality and a relatively short history, so models can look brilliant in-sample while capturing noise rather than signal. Publication bias compounds the problem, because academic NLP findings tend to be reported only when they work, leaving the graveyard of failed strategies invisible to most market participants. And crowding happens fast; a vendor’s pre-packaged sentiment score begins losing its informational edge from the moment of first sale.

Northern Trust’s research surveyed the landscape of text-based signals, from classic bag-of-words sentiment derived from earnings calls to peer similarity measures built on neural embeddings, and found a few categories stand out for consistency. Graph theory approaches model text as a network to uncover dependencies and information flow that simple word sequences and sentiment miss. Text-based measures of firm similarity, derived from companies’ descriptions of their own businesses, identify competitive dynamics and revenue exposures that Global Industry Classification Standard categories overlook. And the specificity and confidence of forward-looking language in 10-K and 10-Q filings correlates with subsequent earnings quality.

What the survivors share, the firm says, is a clear economic rationale set out in advance. Management teams that hedge their language are signalling something; investment decisions are always peer-relative, and firms describe their own businesses more richly than any discrete industry classification.

The evidence is weaker elsewhere. High-frequency sentiment derived from news wires suffers from rapid crowding and implementation friction. Social media signals, despite their popularity, have shown limited robustness in institutional equity settings once transaction costs are accounted for. And generic vendor sentiment scores, applied without adjustment for sector or firm characteristics, tend to degrade quickly as the data becomes widely distributed. The value added at the margin is often meaningfully smaller than research abstracts suggest – which, the report says, is not a reason to ignore the space but a reason to approach it with discipline.

Northern Trust says that institutional investors allocating to strategies that lean on textual signals should approach due diligence with a four-part checklist: sensibility, predictability, consistency and additivity. All four criteria should be met and understood before a signal is deployed – a high bar in a field where the temptation to over-engineer is unusually strong and live track records are unusually short.