UnseenCode: Invisible On-screen Barcode with Image-based Extraction

Hao Cui, Huanyu Bian, Weiming Zhang, Nenghai Yu

University of Science and Technology of China

Abstract

Screen-camera communication techniques achieve one-way communication with widely-used screen and camera devices. Popular barcode methods use visible spatial patterns to represent data, which has been considered obtrusive to human observers. Recent works borrow ideas from visual light communication (VLC), and use inter-frame pixel change to modulate data. To recover pixel change, the receiver has to record and process video frames. Such video-based extraction has high hardware requirements and lacks reliability. Inspired by VLC-based methods, we propose UnseenCode, an invisible on-screen barcode scheme. It leverages inter-frame embedment from VLC-based methods to embed invisible barcodes into arbitrary on-screen contents. Unlike VLC-based methods, UnseenCode does not require video-based extraction. We propose an image-based extraction method based on cross-component correlation of color images. Any off-the-shelf smartphones with camera capability can be used to read UnseenCode by capturing on-screen contents. We propose the full implementation of UnseenCode for evaluation. Experimental results show that UnseenCode decoding algorithm is reliable and applicable under various screen and camera settings. UnseenCode provides up to 2.5 kbit capacity with less than 5% error rate.

Demo

I (CUI Hao) am working on the Android UnseenCode Scanner for demonstration.

Video demo: demo.webm

It is not done yet and there is a major technical problem with camera exposure control. I cannot yet figure out why Android camera2 API failed to control ISO settings on my smartphones, so the decoding reliability is not as good as offline evaluation in the paper. Hopefully I can get over it soon.

Source Code

We promise that the code (performance evaluation and Android demo) will be available before INFOCOM 2019.