Statistics

Big Data, Data Science and Artificial Intelligence

Big Data, Data Science, and Data Integration

Challenges and opportunities

There is increasing demand from policy and decision-makers for data that are more frequent, timely and granular. National Statistical Offices (NSOs) are increasingly exploring the use of big data, data science and artificial intelligence (AI) to help address these user needs. At the same time, growing and evolving data requirements for monitoring national development plans and the Sustainable Development Goals (SDGs), alongside budget constraints and declining response rates on traditional data collection methods, such as censuses and surveys, are accelerating the shift towards innovative sources and methods. Technological advancements have made new data sources available and enabled official statisticians to produce statistics using more complex approaches. However, using these sources and methods requires a wide range of new technical and institutional capabilities.

Rapid advances in AI also bring new opportunities and risks. Increasingly, users seek information through AI interfaces rather than official portals, raising the stakes for ensuring that official statistics remain discoverable, well-described and correctly interpreted. This places greater emphasis on machine-readable data and metadata, transparency about methods and limitations, and governance frameworks that manage risks related to privacy, bias, security, and trust.

Our response

In response to these needs, ESCAP is supporting NSOs in Asia and the Pacific to responsibly leverage big data, data science and AI to improve the production and use of official statistics. This includes strengthening technical and institutional capacity, developing practical guidance and tools, and convening peer learning and collaboration across the region.

Strengthening institutional capacity through technical assistance: ESCAP provides national and regional support to help NSOs adopt innovative data sources and AI-enabled methods across the statistical value chain. Technical assistance has included the use of sources such as satellite imagery and other geospatial data, scanner and web-scraped price data, and mobile positioning data to support the production of priority disaggregated statistics.
Knowledge products and practical tools: Building on the technical assistance provided to countries, and in partnership with UN and non-UN stakeholders, ESCAP has developed a series of knowledge products, including Working Papers, Stats Briefs, step-by-step guides, e-learnings and other practical tools. These resources cover topics including small area estimation using geospatial data, the use of Earth Observation data for land cover and crop estimation, and the use of alternative data sources for price statistics.
Regional peer learning and communities of practice: ESCAP supports peer exchange through regional events and communities. In 2025, ESCAP established a DASH (Data and Statistics Horizon) team on AI for Official Statistics to support the sharing of experience and expertise across Asia and the Pacific (🤖 Explore: Case studies – AI for Official Statistics | submit additional case studies). Previously, ESCAP hosted a Data Integration Community of Practice (DI-CoP) from 2020 to 2024, which supported virtual peer learning and collaboration. Resources from DI-CoP remain accessible via this link.