But clouds got in my way: Bias and bias correction of VIIRS nighttime lights data in the presence of clouds


Ayush Patnaik, Ajay Shah, Anshul Tayal, Susan Thomas

The VIIRS nighttime lights dataset constitutes progress in the measurement of night lights radiance, with monthly data at a pixel of roughly 0.5km × 0.5km. We identify a downward bias in the reported radiance when the number of cloud-free images in a month is low. This bias often takes on large values from -10% to -30%. We develop a cautious bias-correction scheme which partially addresses this problem. This scheme is applied upon the pixel-level dataset to create an improved dataset. The bias-corrected data hews closer to the ground truth as seen in household survey data.

Citation: But clouds got in my way: Bias and bias correction of VIIRS nighttime lights data in the presence of clouds, Ayush Patnaik, Ajay Shah, Anshul Tayal, Susan Thomas, xKDR Working Paper 7, October 2021.

Research

Paper talk


Reproducible research


A package that implements these methods

A 20-minute paper talk that has the gist of the idea.

xKDR Forum YouTube Channel,
6th October 2021

A 5-minute walk through of the reproducible research included in the paper.

xKDR Forum YouTube Channel,
8th October 2021

Our paper does two things: The first open-source implementation of conventional cleaning methods, and a new bias-correction scheme. Both these are implemented in this Julia package. This video is an introduction to this package.

xKDR Forum YouTube Channel,
7th October 2021

In the public domain

The pros and cons of big data used as economic signals
by Niranjan Rajadhyaksha
Mint,
12th January 2022
But clouds got in my way: Bias and bias correction of nighttime lights data in the presence of clouds
by Ayush Patnaik, Ajay Shah, Anshul Tayal, Susan Thomas
The Leap Blog,
11th October 2021
Webinar on Measuring economic activities from outer space
by Ayush Patnaik
Department of Finance and Business Economics, Delhi University,
YouTube
,
15th September 2021