Danbooru

Filesize inconsistencies between pixiv and other sites

Posted under General

I don't know why buehbueh completely changed their comment, but I've also seen cases where the PNGs from Pixiv had been scaled down from the version released on Twitter. Is the Twitter version still considered inferior when it has a higher resolution?

fossilnix said:

I don't know why buehbueh completely changed their comment, but I've also seen cases where the PNGs from Pixiv had been scaled down from the version released on Twitter. Is the Twitter version still considered inferior when it has a higher resolution?

Most of what I originally asked was already covered from earlier by someone else, so I reworded just the part I thought mattered. Yours is another concern I had and I wasn't sure how to word it.

If there is a case where Twitter is the source with a superior resolution and duplicate visual quality, I'd say it should be the parent in those instances.

Twitter is recompressed and as such always inferior. It's similar to upscales made with the waifu tool. Those also are third party edits and despite higher resolution and better image quality still get parented to the untampered original.

Schrobby said:

Twitter is recompressed and as such always inferior. It's similar to upscales made with the waifu tool. Those also are third party edits and despite higher resolution and better image quality still get parented to the untampered original.

waifu2x still loses the original image data and some of the small details in get blurred out. Twitter pngs have the same image data as the original.

Schrobby said:

Twitter is recompressed and as such always inferior. It's similar to upscales made with the waifu tool. Those also are third party edits and despite higher resolution and better image quality still get parented to the untampered original.

It's inferior when we talk about jpg. For the png, the recompression is lossless : the quality is the same (by the definition of png). At worst, the size of the file will be higher but I don't think the difference is really big.

Waifu2x uses a special algorithm to make the upscales look better compared to using regular scaling algorithms like bicubic. Its not comparable to what twitter does since it just recompresses or downscales.

But I would still uses Twitter if its the only source of the image or if the resolution of the image is greater than what I can get from Tumblr.

This is not about twitter not being used anymore. There was discussion about that, but the conclusion was to parent twitter to original not recompressed art once it becomes available.

This is about uploading said original art, from pixiv or elsewhere, after the twitter ones. Toks thinks those should not be uploaded, because the pixels are identical. I think they should be uploaded because they are the untampered original files released by the artists. The TOS says the original uncompressed artwork is preferred.

Schrobby said:

The TOS says the original uncompressed artwork is preferred.

You're looking at the changelog. The changelog is just a log of what was changed in the rest of the TOS, and the wording of the changelog wasn't discussed ahead of time, so it's not surprising that the wording there is ambiguous.

The TOS itself says "Poorly compressed: Any image where compression artifacts are easily visible."

OK, then let's discuss. I just put those two quotes here since I think they are important:

EB said:

If they are otherwise exactly the same (same resolution, no revisions, etc.), then the source that keeps the original file size should be the parent, so Pixiv.

Ars said:

Whether or not the image quality changes is irrelevant. The fact of the matter is that the change in filesize can be considered a third-party edit and those have always been made the child of the original should it be uploaded.

Let me add this quote :

EB said:
All uploaders should make sure they actually compare the images before deciding not to upload, though. "Double uploads" are good when the image here has noticeable compression artifacts or there are other appreciable differences.

I can see two problems :
1/ The upload of same quality but different filesize pictures :
The problem is not really one with a lossly compression (jpg), because the quality will be changed, the best one being the parent.
With a lossless compression (png) , the problem is that the md5 of the two image will be different, so the uploader will not know that he post a duplicate.
Must we upload the original picture if the same from twitter already is ? For me it's a non-sense : it will create the duplicate that we want to prevent.

From now I will assume we speak about a png.

2/ The determination of the parent with two images already uploaded :
Between the image from twitter and the others, which one is the parent ? The answer was already given : the one with the original filesize, not the one from twitter.
Besides, it does not mean that if the one from twitter is the only one in danbooru, the original must be uploaded.

I agree with Toks that, because pngs are lossless and visually the same, a duplicate isn't needed even if an "original" is uploaded later, if the resolution is the same and there are no revisions to the image. It'd be like uploading identical md5 mismatch images that have no revision, that sort of thing doesn't add anything to Danbooru as an image repository when the same visually-identical image is already present.

Though, a compromise could be to use the system that animepaper used when it was around, where there was one image and you could seamlessly switch between identically sized preview images of different parent/child images appended to the post, rather than cluttering DB with parents and children with differently sourced images or slight revisions (For example, there was that one image where the artist made like 15 revisions). That might take too much work on programming though (I have no idea if it's feasible).

My comments before were mostly about what to do (which to make the parent) if they happen to get uploaded (sometimes accidentally). There's not much point IMO to knowingly uploading identical-looking lossless images, but it's not really something I'm against either. For images that were originally uploaded as lossless PNGs on Twitter, you should still look at the artist's later uploads on other sites to see if revisions have been made. I should also point out, though, that sometimes the Twitter version actually should be the parent, if it is lossless but the version the artist posts on other sites is lossy. I've come across that on a few occasions (post #2016088 is one example).

No matter how you look at it the twitter recompression is a third party edit. You also have to keep in mind that PNGs are not necessarily lossless, see https://tinypng.com/
Since we don't know what Twitter does and their method could change anytime without notice we'd basically have to do a pixel comparison for every upload to make sure. That would be overkill.

If having a duplicate really is such a big problem a solution could be to migrate all votes and comments from the twitter image to the parent and delete it. I see no need for that, though.

Schrobby said:

No matter how you look at it the twitter recompression is a third party edit. You also have to keep in mind that PNGs are not necessarily lossless, see https://tinypng.com/
Since we don't know what Twitter does and their method could change anytime without notice we'd basically have to do a pixel comparison for every upload to make sure. That would be overkill.

If having a duplicate really is such a big problem a solution could be to migrate all votes and comments from the twitter image to the parent and delete it. I see no need for that, though.

As for the second suggestion, I'd think that after the reaction to Dakutree's same thing, that might not be a good idea, not to mention it punishes users who don't do single-tag uploads from pixiv.

tapnek said:

Twitter recompression isn't that serious of a problem when it comes to duplicates. Besides, those can be deleted if they're uploaded by those that can bypass the mod queue.

We just talked about deletion and that it's a bad idea. Did you actually read the last posts?

Schrobby said:

We just talked about deletion and that it's a bad idea. Did you actually read the last posts?

Sorry, I should have clarified that it's deletion through being unapproved, something that has happened to me.

tapnek said:

Sorry, I should have clarified that it's deletion through being unapproved, something that has happened to me.

Unapproving twitter posts is also a bad idea. There was discussion about discouraging twitter uploads, but sometimes it's the only source. You never know if and when the art becomes available elsewhere.

That's what I'm trying to say. Although, in those cases I mentioned, the Twitter source I uploaded was the superior source to the Tumblr one I also uploaded at the time as the Pixiv source wasn't uploaded yet.

1 2 3 4 5