Danbooru2018 mirror/dataset

Hi everyone. I'm releasing the second edition of my Danbooru mirror/dataset, Danbooru2018:

https://www.gwern.net/Danbooru2018

This updates my previous Danbooru2017 (https://danbooru.donmai.us/forum_topics/8276?page=4) through 31 December 2018.

The dataset now contains ~2.5tb of 3.33m images with 92.7m tag. This includes all the original images plus the 'safe' subset downscaled to 512px (for easier use in machine learning applications) and the BigQuery mirror metadata (topic #12774) as JSON files. Compared to Danbooru2017, this adds +0.4TB/392k images/15.2m tags.

You can download it via two sets of torrents (the preferred method), or via a rsync mirror.
Please let me know if you run into any problems like bad BitTorrent client versions.

An example use of the dataset is training https://github.com/lllyasviel/style2paints , which is an impressive neural net tool for colorizing anime images.

Another fun use of the Danbooru corpus: StyleGAN has cracked the problem of high-quality unrestricted anime face generation. I was able to dump in 220k faces derived from Danbooru2017 and train a generator of great anime faces: https://twitter.com/gwern/status/1093701790971953152 I've trained 3 so far, all faces, Holo (Spice & Wolf), and Asuka (NGE).

I've put up 60k samples of the faces as a simple little website: http://www.thiswaifudoesnotexist.net/ It's been surprisingly popular: over 470k visitors so far.

Now for a husbando version.

https://twitter.com/roadrunning01/with_replies is working on that. Once he stops dinking around with n=50 or n=1000 datasets and just crops all of the male faces from Danbooru2018 to train on, it'll work fine.

Up to 690k unique visitors now after an upgrade to use GPT-2 text samples about anime plots...

One amusing outcome so far: on 4chan and Twitter and elsewhere, people are mentioning saving some of the best faces, and are using them as avatars already. If you look at the two sets of Asuka & Holo faces I hand-curated, https://imgur.com/a/xXuKwYV and https://imgur.com/a/7R8IRVh , a lot of them are, IMO, human-level and as good as many faces on Danbooru now. One person on Twitter even asked me how to contribute faces back to the 'dataset' (I pointed them at Danbooru). Other people on 4chan and Tumblr and elsewhere are already asking for the 'original' images and information about the 'artist', and just straight up using them or editing them.

So this raises a question of Danbooru policy for you guys to consider.

'GANime', to coin a term, is only going to get better. How should particularly high-quality samples be treated? Some of them are probably going to start showing up on Danbooru, if only as they circulate online & origins are forgotten. Should ML-generated images be banned from Danbooru, period? Should, somewhat like the current handling of waifu2x-upscales, they just be tagged under an author like 'StyleGAN' to record their machine origin and otherwise treated like normal images, deleted if bad & kept if good? Or what?

They should be banned. They are ultimately third-party edits that are even of lower effort than Photoshop editing, even if you have to go through many iterations to find the best face.

tapnek said:

They should be banned. They are ultimately third-party edits that are even of lower effort than Photoshop editing, even if you have to go through many iterations to find the best face.

Well, in the (near ?) future, the only way to know if an artist had put an GAN-generated image on pixiv or the likes will be if he explicitly states it.
Supposing they will be indistinguishable from the other pictures, we won't be able to ban them even if we wanted to.
And since they will be indistinguishable, I am not convinced we should ban them anyway. A good picture will still be a good picture even if it was drawn by a computer.

...They are ultimately third-party edits that are even of lower effort than Photoshop editing, even if you have to go through many iterations...

Should 'effort' be the only criterion of a picture?

The thing I fear is that these kind of generators could produce enough high quality images that they could flood boorus if unchecked and overshadowing real artists who put time into their art.

The other problem is that the generated art is essentially a form of art theft. Combined with taking the focus away from artists and sourced work and... nothing good, really.

gwern-bot said:

Should 'effort' be the only criterion of a picture?

It's the main reason why stuff from CM3D/Koikatsu/etc. is generally avoided even though the quality of the art itself is generally fairly high.

I'd like to give you some belated thanks for making this. It's a neat little experiment that I've been coming back to for a few days now, even if it gets weird pretty often.

I think the concerns about this being indistinguishable from hand-drawn art are overblown. There are already many ways to make transformative art that require little conventional skill: Photoshop, image filters, game screencaps, dollmakers, 3D posing programs, waifu2x, etc. All of those can be potentially mistaken for original art by an untrained eye, but Danbooru isn't run by untrained eyes. As cool as these neural networks are, it's pretty easy to learn how to spot their work at the moment, and I imagine we'll simply be playing whack-a-mole with the idiosyncrasies of any future ones.

As for why we don't archive those in the first place, it borders on pointless to search an archive for something that you could create by yourself in the exact same amount of time. Like archiving every dead leaf on the ground, or every possible iteration of a Photoshop lens flare. It doesn't mean that they're bad or pointless, just not a good fit for what we're doing.

I've written a guide to how to use StyleGAN and including many samples from a variety of training runs on faces, specific characters, 'portrait' headshots, Danbooru2018 etc and comparing with previous GAN runs: https://www.gwern.net/Faces

One thing which might be particularly interesting: we now are able to edit faces, by encoding them backwards into the latent space and then manipulating the latent space variables: https://www.gwern.net/Faces#reversing-stylegan-to-control-modify-images

This relies on use of DeepDanbooru (discussed at topic #15912 ) in order to reverse-engineer what bits of the latent space control things like hair color or eye color. I see this as a good demonstration of what I mean when I say that a good NN tagger for Danbooru would have many applications far beyond merely improving tags on existing Danbooru images or making suggestions for new uploads.

gwern-bot said:

Up to 690k unique visitors now after an upgrade to use GPT-2 text samples about anime plots...

One amusing outcome so far: on 4chan and Twitter and elsewhere, people are mentioning saving some of the best faces, and are using them as avatars already. If you look at the two sets of Asuka & Holo faces I hand-curated, https://imgur.com/a/xXuKwYV and https://imgur.com/a/7R8IRVh , a lot of them are, IMO, human-level and as good as many faces on Danbooru now. One person on Twitter even asked me how to contribute faces back to the 'dataset' (I pointed them at Danbooru). Other people on 4chan and Tumblr and elsewhere are already asking for the 'original' images and information about the 'artist', and just straight up using them or editing them.

Another example is a Pixiv account just for StyleGAN samples: https://www.pixiv.net/member.php?id=38983753 And 4channers are making 'avatar packs' for some sort of computer game (which is actually a pretty reasonable use IMO).

I have since expanded https://www.thiswaifudoesnotexist.net/ with a StyleGAN retrained on faces with much wider crops so as to capture the whole head and any extras like bunny ears. They're not too bad. I am currently training a male-face-only StyleGAN (for the various people who've requested a 'husbando' StyleGAN), and am experimenting with BigGAN at 256px - initial results are promising but BigGAN will be very expensive to train to convergence at 512px, looks like.

Discussion of the Danbooru2019 release is in topic #16535