Alternative data for hedge funds & portfolio management
The years 2020 and 2021 marked a significant departure from typical market conditions. The period began with the continuation of the longest bull market in modern financial history, which originated from the market bottom following the sub-prime mortgage crisis in early March 2009. The S&P 500 closed at 676.53 on March 9th, 2009, and reached 3,386.15 on February 19th, 2020. This represents a remarkable 500% growth over slightly more than a decade, surpassing the previous best performance of 417% achieved during the 1990s.
The arrival of March 2020 brought unprecedented challenges as what was initially dismissed as a mild flu-like illness confined to mainland China spread globally. Financial markets experienced sudden disruption, though they have shown varying degrees of recovery since then, with some sectors performing better than others. The rapid shift toward a socially distanced, virtual-first environment has accelerated existing trends in the financial industry. Notably, buy-side spending on alternative data has increased by over 160% since 2018, reaching $1.7 billion in 2020 alone.
According to the Alternative Investment Management Association (AIMA) report “Casting the Net: How Hedge Funds Are Using Alternative Data,” more than half of survey respondents actively utilize alternative data, while an additional 14% are exploring implementation options.
While definitions of alternative data may vary, there is broad consensus that it originates from sources beyond traditional financial information. Standard sources typically include SEC filings, financial performance reports compiled by organizations like Bloomberg and Refinitiv/Thomson Reuters, internal proprietary portfolio performance data, and other established sources. As discussed in previous analysis, alternative data extends far beyond social media platforms. Datasets sourced exclusively from social networks may lack sufficient quality and require careful preparation and processing before application in financial services.
Throughout the history of modern capitalism, and even in ancient times (notably, Babylonian merchants used Euphrates river depth measurements to inform commodity pricing), businesses have sought competitive advantages. They achieve this by identifying patterns and trends not readily available through conventional analysis or not yet leveraged by competitors. The current situation differs due to the unprecedented data explosion we’re experiencing. IDC estimates that 1.2 zettabytes of data were created in 2010, but this figure is projected to reach 175 zettabytes by 2025. This growth combines with advances in data collection, processing, and analysis technologies, including sophisticated data extraction methods, knowledge graph technology, natural language processing (NLP), entity resolution, and continuous improvements in computing power from providers like Dell EMC, HPE, and AWS.
As Michael Megaw from SS&C recently noted, “Alternative data has become a disruptor in the hedge fund industry,” positioning organizations that successfully adopt and integrate it into their research methodologies to achieve significant advantages over industry laggards.
However, caution is warranted. The current wave of enthusiasm may create misconceptions about what this data can accomplish for financial services organizations, whether in capital markets, investment management, banking, or insurance. Alternative data does not replace existing datasets or proven quantitative research methodologies. Rather, it serves as a powerful enhancement tool, augmenting traditional research methods with deeper, more comprehensive insights.
Market trends indicate continued growth across all aspects of alternative data – availability, sell-side providers, adoption rates, and workflow integration. As the initial excitement of accessing this “miracle” data subsides and it becomes more widespread throughout capital markets and investment management, a shift toward prioritizing data quality over quantity and diversity appears inevitable. This focus on value will likely lead to market consolidation, particularly given that data quality issues cost organizations an average of $9.7 million annually.
All hedge funds are not created equal
Hedge funds vary significantly in their capabilities and resources. A straightforward way to differentiate them is by examining assets under management (AUM) to distinguish large institutional players from smaller, specialized firms.
Logic suggests that mid-size and large funds, particularly those managing $5 billion or more and $10 billion or more respectively, possess greater resources to harness and monetize alternative datasets effectively. This assumption appears accurate according to EY’s latest Global Alternative Fund Survey, which shows that while 44% of funds overall have dedicated full-time employees for alternative data initiatives, this figure jumps to 60% for funds exceeding $10 billion AUM. For smaller funds ($2 billion AUM or less), the percentage drops to approximately 32%. Nevertheless, this demonstrates that smaller players remain agile enough to allocate resources strategically and compete effectively.
Regarding performance, particularly for alternative investment funds that are more likely to leverage alternative data, 58% of investors report that their managers met or exceeded performance expectations during pandemic-related market volatility.
Top types of alternative data used by hedge funds
Thousands of alternative datasets are available from various vendors. One leading provider offers over 1,500 ready-to-consume datasets, highlighting the importance of proper categorization.
Most hedge funds regularly utilize at least one of the following data types:
- Web data
- Transaction data/Consumer spending
- Social Media & related sentiment data
- App usage
- Web traffic
- Geo location
- Satellite imagery
- Email receipt
Generating Alpha & Risk Management
Delivering returns attributable solely to a hedge fund manager’s expertise and skill represents the ultimate goal of generating alpha. To achieve this, fund managers must identify and exploit competitive edges by spotting overlooked or underestimated opportunities and allocating appropriate portfolio weights to their investment strategies.
In a recent Eagle Alpha study, Olga Kokareva from Quantstellation effectively demonstrated how alternative data usage varies significantly between fundamental and quantitative hedge fund approaches.
“It’s important to understand that usage of alternative data by fundamental hedge fund managers and by quantitative hedge funds are two very different processes. Fundamental hedge fund managers normally use alternative data to reinforce their investment thesis that they derived from their regular research process. For example, a manager can hold a long position in a retailer, and they are thinking about closing it, but they are not sure. So, instead of waiting for the next quarterly report, they can start looking at foot traffic data or credit card data. If the sales numbers are indeed going down, they might close this position earlier.”
Conversely, quantitative hedge funds derive their investment hypotheses purely from available data insights, typically employing advanced machine learning models. This approach has existed since the late 1970s and 1980s, making the integration of alternative datasets to enhance probabilistic models a natural evolution of existing practices.
Alternative data applications extend beyond stock selection to encompass comprehensive risk management. From a capital allocation perspective, investment risk management remains a cornerstone of hedge fund operations. The concept of risk-adjusted returns, exemplified by the Sharpe ratio developed by Nobel Prize winner William Sharpe in 1990, continues to be fundamental to sound portfolio management.
The challenge remains imperfect information, which renders mathematical constructs like the Sharpe ratio inherently limited. While perfect information is empirically impossible, alternative data promises to reveal hidden risks that can significantly impact risk-reward calculations. Insurance and lending organizations have already begun layering alternative data onto traditional datasets for this purpose. Hedge funds lag somewhat in applying alternative data for risk management, with only 23% of market leaders using it to enhance risk management processes.
Risks & Challenges
Effective alternative data utilization – whether for investment generation or operational efficiency – requires five essential components:
- Adequate human capital
- Appropriate infrastructure
- Sound processes, including Master Data Management
- Regulatory compliance navigation
- Demonstrated ROI to investors
While detailed examination of each element merits separate discussion, the key challenges involve infrastructure, processes, and human capital requirements.
The previously mentioned AIMA study identified infrastructure and human capital adequacy as primary challenges to realizing alternative data benefits (49% of market leaders and 54% of other market participants).
When examining specific challenge components, 77% of market leaders and 54% of other market participants cited the inability to back-test alternative data as the primary obstacle. Many large datasets lack sufficient historical depth for meaningful contribution to historical-based models.
Another critical insight reveals that over half of market leader respondents and more than 60% of other market participants emphasized difficulties in sourcing quality datasets. Data quality is crucial not only for hedge fund performance but reflects broader issues stemming from the data explosion phenomenon.
Organizations now face data overwhelm; obtaining data isn’t problematic, but deriving actionable insights is. This requires clean, insight-ready data. The frequent absence of quality data costs organizations nearly $10 million annually on average, according to Gartner. IBM estimated five years ago that poor data quality costs the United States alone $3 trillion yearly.
For hedge funds specifically, even after identifying appropriate alternative datasets, master data management questions emerge. Data governance and stewardship, semantic consistency across databases and systems, permanency risk (future dataset availability), and data robustness and consistency (mapping to fixed references like CUSIPs in the US or SEDOLs in the UK and Ireland) all require attention.
This situation underscores the need for organizations to maintain adequate human capital for maximizing alternative data value. The approximately 450% increase in dedicated full-time employees for dataset management over recent years demonstrates the competitive race for qualified candidates.
Where to from here
Recent studies by AIMA and the EY 2020 Global Alternative Fund Survey suggest that alternative data adoption within hedge funds will continue expanding in the near term.
While dataset numbers may continue growing, a flight to quality appears inevitable, leading to market consolidation. With major players like Bloomberg doubling down on alternative data and Refinitiv consolidating their offerings, the era of alternative data-augmented investment strategies is clearly established for the long term.
How can Webparsers help
At Webparsers, we specialize in delivering custom data feeds specifically optimized for clients by converting unstructured web data into structured formats, or enabling organizations with internal data collection teams through robust, resilient, always-on infrastructure designed for web data extraction.
We assist some of the largest financial services institutions in navigating web data extraction complexity for alternative data use cases, ensuring compliance standards are met while maintaining healthy data pipelines.