Any plans to rework AI metadata storage?

This was brought up in the Discord server before but this discussion shouldn't be left in there.

The current fields for AI metadata are becoming less and less useful as more software is being developed and more extensions/tools are being used to create images. As it stands currently the metadata provided by default is not even enough to recreate an image 1:1 if hires fix is used in the webui, as we can't always assume the base gen is the final resolution or double that. This is even more difficult when the image is cropped or outpainted like post #34240 (the original base gen resolution of which was 512x768) or post #37245 (does not contain the WxH metadata on AIBooru or in the original image's EXIF).

Because very few users are even going to embed the original metadata in their images (I believe only @iodoff and I have done this) there should at minimum be enough of the fields required to replicate an image 1:1. Width and height are the pretty major ones missing but since there's so many possible fields I don't think everything can continue being stored in separate columns. The metadata userscript is basically required to get anything useful out of embedded metadata, but ideally this would all be indexed somehow. I've had to write my own tools to tag images using controlnet and adetailer and whatnot by parsing the media asset pages one by one, as I don't think the API endpoints even expose the full metadata fields currently. comfyui is also becoming more and more prevalent, especially within the SDXL community (anime finetune pending though lol), and should ideally have the workflow visually embedded on the page (the frontend uses a single file JS lib to render the graph so a read-only view is doable).

I realize this is all a huge task, and I would like to contribute but I'm currently very unfamiliar with Danbooru's codebase, but frankly I find the metadata fields worthless in their current state and are only upheld by userscripts and supporting tags (lora, controlnet, etc). I might also be in the minority here though which is why I open this topic here for discussion.

I don't think there is even a way of doing this robustly. Metadata relies mostly on the willingness of the uploader to describe the process (unless it's literally just txt2img). So I don't think reproducing the image 1:1 is an achievable goal in most cases.

That said, it would be nice to have some additional fields to fill (at least the initial size) during upload. Or maybe a way to add arbitrary key:value pairs?
This and a metadata parser integrated without an additional script could make it more user friendly. Throwing a comfyui visualizer might be possible too but I don't think it's necessary. I think that displaying and allowing the workflow to be downloaded as a json would be plenty, since anyone who cares about comfy workflows has comfy installed anyway.

mkbin said:
comfyui is also becoming more and more prevalent, especially within the SDXL community (anime finetune pending though lol), and should ideally have the workflow visually embedded on the page (the frontend uses a single file JS lib to render the graph so a read-only view is doable).

comfyui metadata is just a big mess. Reading it without reconstructing the entire workflow is just impossible.
https://i.imgur.com/9WAsR3F.png

I also don't think it's necessary to add workflow view on aibooru, there are already other sites for that.

mkbin said:

or post #37245 (does not contain the WxH metadata on AIBooru or in the original image's EXIF).

original resolution is 960x540

and 1920x1080 after hires fix

I couldn't embed the metadata because I had to use jpg due to the maximum file size limit

Edit: I replied when there was only one reply, so this is in response to @antlers_anon

I can agree with a lot of that. The metadata in a way is becoming overall less important as better images tend to involve more work like inpainting or external control, sometimes requiring additional image inputs which can't always be embedded in metadata (and the solution mentioned in this topic is perfectly fine for anyone bothering to make that metadata available imo). Some people like myself though would still like the ability to recreate the "base gen" when possible, even if much further work went into the final image. There's been many times on 4chan the creator will share multiple stages thru the process or even the whole inpainting process with every intermediate image because another user was interested in it.

Additional/optional key:value pairs actually might be a very good approach to this as you describe. It's flexible enough for images with only a few fields and for those with potentially everything. Anyone who uses comfyui would likely have to fill in that data themselves if they want it searchable is my only concern there, but that's an existing issue with comfyui as it is even currently. Adding a .json download for those workflows is a much lighter solution too since you're correct only comfyui users would want those in the first place.

I feel overall maybe the lack of motivation for this is most uploads here are not self-uploads and even less of those contain some form of metadata included. But, I also think it's discouraging to those who _are_ making self-uploads and have no decent & searchable way to present that metadata.

iodoff said:

comfyui metadata is just a big mess. Reading it without reconstructing the entire workflow is just impossible.
https://i.imgur.com/9WAsR3F.png

I also don't think it's necessary to add workflow view on aibooru, there are already other sites for that.

Yeah I don't think there's a reason to parse it, it was never meant to be made user-readable. And you're correct a workflow view isn't 100% necessary, but having the ability to download it as @antlers_anon mentioned seems like a good compromise. Opening the media asset page for images that do contain the workflow is not a great experience and isn't something that should require a userscript imo.

iodoff said:

I couldn't embed the metadata because I had to use jpg due to the maximum file size limit

You can still use EXIF. AIBooru isn't going to read it (currently) but it'd be there for people who do download the original file.

mkbin said:

You can still use EXIF. AIBooru isn't going to read it (currently) but it'd be there for people who do download the original file.

I know, but i'm too lazy to edit the script that I use to add the metadata to work with jpg :c