As the paper explains, “Recent deep image-to-image translation techniques allow fast generation of face images from freehand sketches. However, existing solutions tend to overfit to sketches, thus requiring professional sketches or even edge maps as input. To address this issue, our key idea is to implicitly model the shape space of plausible face images and synthesize a face image in this space to approximate an input sketch. Our method essentially uses input sketches as soft constraints and is thus able to produce high-quality face images even from rough and/or incomplete sketches.” 

It’s not clear how the software will handle race. Of the 17,000 sketches and their corresponding photos created so far, the majority have been Caucasian and South American faces. This could be a result of the source data (bias is an ongoing problem in the world of AI), or down to the complexity of face shapes — the researchers don’t provide any further details.

In any case, the technology is due to go on show at this year’s (virtual) SIGGRAPH conference in July. According to the project’s website, code for the software is “coming soon,” which suggests we could see its application in the wild in the coming months — not only as a fun app to play around with, but also potentially in law enforcement, helping to rapidly generate images of suspects.