Pixelor: A Competitive Sketching AI Agent. So you think you can sketch?

SIGGRAPH Asia, 2020
Generative Models

Ayan Kumar Bhunia

SketchX, CVSSP, University of Surrey

Ayan Das

SketchX, CVSSP, University of Surrey

Umar Riaz Muhammad

SketchX, CVSSP, University of Surrey

Yongxin Yang

SketchX, CVSSP, University of Surrey

Timothy Hospedales

University of Edinburgh, UK

Samsung AI Center, Cambridge

Tao Xiang

SketchX, CVSSP, University of Surrey

Yulia Gryaditskaya

SketchX, CVSSP, University of Surrey

Yi-Zhe Song

SketchX, CVSSP, University of Surrey


July 30, 2020

Paper Supplementary Code


We present the first competitive drawing agent Pixelor that exhibits human-level performance at a Pictionary-like sketching game, where the participant whose sketch is recognized first is a winner. Our AI agent can autonomously sketch a given visual concept, and achieve a recognizable rendition as quickly or faster than a human competitor. The key to victory for the agent is to learn the optimal stroke sequencing strategies that generate the most recognizable and distinguishable strokes first. Training Pixelor is done in two steps. First, we infer the optimal stroke order that maximizes early recognizability of human training sketches. Second, this order is used to supervise the training of a sequence-to-sequence stroke generator. Our key technical contributions are a tractable search of the exponential space of orderings using neural sorting; and an improved Seq2Seq Wasserstein (S2S-WAE) generator that uses an optimal-transport loss to accommodate the multi-modal nature of the optimal stroke distribution. Our analysis shows that Pixelor is better than the human players of the Quick, Draw! game, under both AI and human judging of early recognition. To analyze the impact of human competitors’ strategies, we conducted a further human study with participants being given unlimited thinking time and training in early recognizability by feedback from an AI judge. The study shows that humans do gradually improve their strategies with training, but overall Pixelor still matches human performance.


BibTeX citation:
@inproceedings{kumar bhunia2020,
  author = {Kumar Bhunia, Ayan and Das, Ayan and Riaz Muhammad, Umar and
    Yang, Yongxin and Hospedales, Timothy and Xiang, Tao and
    Gryaditskaya, Yulia and Song, Yi-Zhe},
  publisher = {Association for Computing Machinery},
  title = {Pixelor: {A} {Competitive} {Sketching} {AI} {Agent.} {So} You
    Think You Can Sketch?},
  booktitle = {SIGGRAPH Asia, 2020},
  volume = {39},
  number = {6},
  date = {2020-07-30},
  url = {https://dl.acm.org/doi/pdf/10.1145/3414685.3417840},
  langid = {en}
For attribution, please cite this work as:
Kumar Bhunia, Ayan, Ayan Das, Umar Riaz Muhammad, Yongxin Yang, Timothy Hospedales, Tao Xiang, Yulia Gryaditskaya, and Yi-Zhe Song. 2020. “Pixelor: A Competitive Sketching AI Agent. So You Think You Can Sketch?” In SIGGRAPH Asia, 2020. Vol. 39. Association for Computing Machinery. https://dl.acm.org/doi/pdf/10.1145/3414685.3417840.