XLP Publishes Web Scraping and Data Aggregation Technology Report

XLP Strategy Research Group announces the release of a technology report, entitled “The Data Scraping Marketplace.” The report describes the rapidly growing marketplace for software known as “data scrapers” aka "web scrapers" which are capable of taking unstructured data from websites and converting it into structured data output for further analysis. Nearly unknown to the current business strategists and managers, data scraping has grown from an obscure back office tool or software developer method to the forefront of desk traders, market analysts, and competitive intelligence and strategy professionals. The report follows recent revelations that many of the world’s largest hedge fund managers have developed sophisticated tools for web scraping and the automated aggregation of data previously undisclosed.

The 30 page report provides a detailed description of the leading technologies in the market for “scraping” tools, as well as the technologies now used and deployed for “anti-scraping” (and associated techniques to prevent the harvesting of data from web sites). The report is written for Fortune 500 executives and strategists to understand the practical implications and opportunities for the automated collection and structuring of web based data for use in investment decision making, M&A diligence processes, and competitive intelligence. For professionals, the research report also includes a detailed terminology appendix providing the many technical names that “web scraping” is called internationally, including "data munging", "web harvesting", "data aggregation", "automated aggregation", etc.

Selected analyses from the research include:

• Overview of major software and web technology trends that enabled the creation of sophisticated data scraping platforms
• A prioritized list of all major commercially available and open source software packages, tools, and platforms providing data scraping capability (e.g. Connotate and Import.io)
• A feature comparison of the most popular scrapers, including an assessment of which are best suited for various common data types and datasets
• A technical and architectural analysis of common data scraping methods, including related technologies required to defeat anti-scraping technology
• Prioritization of some of the most valuable business data sets utilized by XLP Strategy Group for its competitive intelligence and research services
• Ranked and sorted patents and patent claims pertaining to data scraping, including the most commonly cited, potentially infringed, and directly relevant
• A series of opinions and interview notes from technical engineers identifying which patents are most likely infringed by certain data scraping software products
• Price list of common scrapers, and an estimation of which features in scraper platforms are most commonly priced as “premium”
• A list of the most frequently mentioned clients in financial services, tech, software, and consumer goods who purchase data scraping software
• Reviews and ranking of features and technologies for the top 5 anti-scraping and web data protection companies on the market
• A landscape of scraping technology investors, venture capitalists and capital market players acquiring scraping companies
• A review of the leading data scraping trade shows and big data conferences in which technologies are often featured and announced
• Wallstreet equity analysts covering companies that offer data scraping services, including the companies they cover, their sentiments on publicly traded companies
• A review and assessment of all known (over 200) strategic partnerships struck between corporations and data scraping providers
• Overview of the leading publications, and authors frequently carrying articles and reviews on data scraping methods, tools, and applications

Included in the report are 30 page analyst report, all accompanying raw data in Excel spreadsheets, and interview notes with industry professionals and investors. XLP Strategy Group research analysts are available for interview and further discussion.

Request More Information

Due to the competitive nature of XLP Strategy Research, distribution will be limited. Please contact XLP Strategy at info at xlpcapital.com for more information and pricing.