Suggestion: Create a Project

[iwr-redmond]

With all of the features that I am gratuitously finding for you to spend your time on, it might be helpful to use the Projects feature to sort these ideas into categories that make sense for you.

This organizational tool would help third parties, such as me!, to understand what your priorities might be, and to better fit FRs with your vision for dgenerate. The Projects tool can also be valuable in helping you to prioritize your work and think about how issues relate to one another.

Some examples from other repositories include:

• FooocusPlus, which uses a hybrid version-based and thematic setup
• Invoke, which used projects to group issues into larger features
• GPT4All, which used a yearly calendar with issues sorted by priority

[Teriks]

I agree, I will have to take a moment to see what I can do with the projects feature. I am on the cusp of what I can store in my head in terms of changes history / and need todo

[iwr-redmond]

Of the recent FRs, I would suggest deprioritizing:

#17 High VRAM requirements
#16 Niche feature, albeit a useful one
#19 Similar Kolors already supported

If #20 and #22 are as simple as they appear, then these would add value for all platforms, especially while bitsandbytes remains a work-in-progress for non-Nvidia users. Otherwise they are also nice-to-haves.

My personal hobby horse remains #13 and the K-Diffusion integration. Hopefully the preexisting code I referenced in this comment will make the integration process easier to achieve. I reckon "o1" (whether described as such or not) is a major feature upgrade that could reinvigorate SD1.5. However, as with every FR this depends on how much work is involved.

[iwr-redmond]

I see that you have been hallelujahing the county with the projects feature. It's very helpful for me as well, so thank you for that.

A few notes on the state of quantization in Diffusers:

• torchao: keep an eye on ao#1082 and to a lesser extent ao#556, as there is currently no support for CUDA on Windows which seems like a major issue for dgenerate; once CUDA is available there are ready-made inference scripts here
• GGUF: the SD.Next docs make the point that this format is being poorly used at the moment, an issue that was recently replicated in FooocusPlus#76; it's noteworthy that the highly professionalized InvokeAI uses NF4 for its Starter Models

[iwr-redmond]

Regarding your note about improving the platform-specific code for setup.py, you may wish to review the ComfyUI-LTS setup.py code for useful starting points.

[Teriks]

They are using platform detection in setup.py and distributing as source. Though, I don’t think they have a PyPI package.

I could definitely distribute the package this way as a lot of packaging quirks are solvable when the install process through pip is allowed to just run some code to perform the install procedure.

PyPI still supports sdist and probably always will even if it is not encouraged to use it.

Some downsides to this may be installation speed of the package may take a moderate hit, I vaguely remember going to wheel dist because of the install speed or lack of package caching but I intend to test what it’s like. Currently nothing is getting compiled, it’s just the overhead of pip building the package metadata.

The speed would essentially be identical to an install with pip install git+https://github.com/Teriks/dgenerate as pip is doing the same thing there.

Looking at some of my dependencies, such as diffusers, they are going to bring in some sort of version of torch automatically if my project doesn’t declare it up-front as a dependency, so maybe having it be an extra hoping to completely exclude it from the default install is unviable. And custom configured installs just have to uninstall what is installed and install something new.

[iwr-redmond]

They don't have a PyPI package, but I think this is by choice rather than because of the design of their setup.py code. ComfyUI-LTS updates every other day to stay in sync with ComfyUI-master.

[iwr-redmond]

Of the next round of recent FRs, I would suggest deprioritizing:

• [Feature Request / Future] Fooocus SDXL Modules #27 Quality improvements and extra features, but evolutionary rather than an o1-style architecture enhancement
• [Feature Request / Future] LanPaint Universal Inpainting #26 Will save disk space with fewer checkpoints and memory when Controlnets would be used, but still very new
• [Feature Request] Supporting quantization for DiT models (Flux, SD3.5) #14 (ONNX & GGUF components)* Bitsandbytes is a great first step

Hopefully #25 is easy to implement and will eliminate the need for addressing the complexities that have become apparent in #16. If #28 becomes similarly difficult, it can also be deprioritized.

* I reckon they might each need their own FR in the future to make tracking them easier

[Teriks]

I am thinking of making another repository to host pre-release windows builds for testing.

I have hit a slightly busy streak, but I am working through these ideas bit by bit.

[iwr-redmond]

It's hard to code well by neon. Take your time!

[iwr-redmond]

Regarding your comment about localization, you may wish to consider the relatively new transx package, which supports translations incorporating parameters and environment variables.

[iwr-redmond]

Are sigmas supported in the Flux pipeline? The option isn't in the example. I'll write up an FR if not, as its use bumps up the quality noticeably.

[Teriks]

Are sigmas supported in the Flux pipeline? The option isn't in the example. I'll write up an FR if not, as its use bumps up the quality noticeably.

directly passing from the cli into the pipeline is not supported currently, so I'll need to implement when I get around to digging into the rendering backend to implement the speedup algorithms, probably the best time to work on that since I will be adding arguments

[iwr-redmond]

Thanks @Teriks! Written up as #33.

[Teriks]

Nightly build should go out every day at 3AM cst if there are changes in this repository. The actual artifacts may take a bit to upload on the release as they are large.

Pre-release will always be at: https://github.com/Teriks/dgenerate/releases/tag/pre-release

The github release is simply modified when a new change set is built

Ended up just rolling my own jenkins master at home with a docker cloud

[Teriks]

It’s going to end up zipping up old source code at the bottom of the artifacts list automatically due to githubs release API, but I’ll update the pre-release notes to mention this, and probably add a generated link to the commit later instead of just the branch. Currently the commit sha is in the artifact names.

[iwr-redmond]

You have been busy as a hound in flea season working on version 5.0.0, and I have been busy strutting around sitting "working" to ensure that the wish list is always growing!

It may be time soon to create a new "Research Postponed" category in the current project in order to make things more manageable.

My thoughts are:

Feature Request	Complexity	Suggestion	@iwr-redmond notes	@Teriks notes	Decision?
#13 o1 K-Sampler	Hard	Research	Complete if wrapper code works*
#14 GGUF/ONNX	Hard	Postpone	Bitsandbytes is rapidly maturing
#16 PatchMatch	Hard	Close WONTFIX	Implement #25 instead
#17 PuLiD	Moderate	Postpone	Niche feature
#18 Torchruntime	Easy	Complete	Easily support more GPUs
#19 HyDiT	Easy	Postpone	Similar to Kolors
#25 Resynthesizer	Easy	Complete	Good alternative to PatchMatch
#26 LanPaint	Moderate	Postpone	Immature tech, needs #13 first
#27 Fooocus SDXL	Hard	Postpone	Large scope, needs #13 first
#28 Previews	Moderate	Postpone	Speedups make this less urgent
#31 SAM masks	Easy	Complete	Good match for CLI
#32 HD Inpaint	Moderate	Postpone	HiDiffusion a good first step
#34 GPUs	Easy	Complete	Better non-NVIDIA support
#35 FreeU	Easy	Complete	Already supported in diffusers
#37 Upscale Demo	Easy	Complete	Easy extra feature after #35
#38 Flux CNU v2	Easy	Complete	Easy upgrade to Union CN

* Implementing the original K-Diffusion may be in scope if one of the existing wrapper pipelines can be adapted for dgenerate. This would also permit easy integration of other non-diffusers K-Samplers, e.g. LanPaint, Fooocus, FramePack, etc.

You may wish to edit this table to add your own thoughts. I have left columns for this purpose, and you are very welcome to correct anything you think is incorrect.

[Teriks]

torchao quantization is also quite buggy in diffusers at the moment so I am wondering about that one, I have not had time to debug it but it may just not be very mature.

I mostly agree with this, previews could be fun though not entirely necessary, generally I deal with this by just getting an idea from a batch of low inference steps renders. Might not be too hard so I may poke at that one.

I have not had time to look through every single one but DirectML support is likely straight forward, and would give me a way to test that I am not assuming CUDA improperly in parts of the code base without having to rent a mac :)

The main thing limiting me on the hard ones, is that I am currently only able to get about an hour or two in on this a few days out of the week, (contributions definitely welcome if comfortable, I will try to review)

I plan to work on parameterizing Flux sigmas next.

CNU2 pipeline should be easy to add as well, though I might not have time to flesh out img2img or inpaint for that pipeline if it has not been done yet.

HyDiT is not too bad either, just similar to CNU2 in that img2img and inpaint might not exist.

[iwr-redmond]

Generally if something I code barely works, it's so unusual that you would be forgiven for wondering if it's a setup...

On CNU2, the sample code on the model page suggests that the existing diffusers pipeline may be enough. It's unclear why the repo includes a custom pipeline script.

On HyDiT, there doesn't appear to be any img2img or inpainting code available. However, there are three third-party Controlnets that include one for inpainting.

[iwr-redmond]

Generally if something I code barely works, it's so unusual that you would be forgiven for wondering if it's a setup...

...but I can review code for usefulness and have rediscovered a K-Diffusion adapter for Diffusers that I have previously, albeit unknowingly!, tested. It was developed by @gabe56f and @Stax124 for diffusers 0.24.0 and was the default scheduling option for the VoltaML software.

[iwr-redmond]

Regarding your comment about spaCy, according to a recent commit support is being added for Python 3.13, although the timeline for wheel availability is still unclear (spaCy#13755).

[iwr-redmond]

Is it possible to use Daniel Bowring's fork?

[Teriks]

That would work, not sure if gpt4all is up to date now that I think about it.

Eventually I think I will pick up a mac mini, that would alleviate these native build blockers

[iwr-redmond]

GPT4all isn't Python version specific, so it will hopefully work as-is.

[Teriks]

the fork works with a bit of monkey patching to transformers/diffusers, since they cannot detect it: 9dfe90c

[Teriks]

actually requires patching near stdlib level for transformers / diffusers to be completely happy: eb9d34b

pushing a newer 3.13 build out shortly, previous patching leaves Kolors and a few things broken

[iwr-redmond]

It was a good idea to link to the dev version from the main Readme file. You may also wish to link to the branch documentation.

I noticed a small bug in the console UI: the documentation links go to a non-existent 'v5.0.0' branch, instead of the correct 'version_5.0.0'.

[Teriks]

When installing from source manually and not the pre-release installer or wheel (dev environment), there is a missing flag file in the source directory that gets created by the release scripts. dgenerate/release.json

This points the Console UI to the right location for documentation etc. depending on if it is a full release or a pre-release.

It is assuming it is a full release in the dev environment, and simply using __version__ to create the links, version v5.0.0 in this
case which is not released.

Creating "release.json" with these contents will point the Console UI to the correct resources, it will instead
point at the doc associated with the branch and not a non-existent tag.

{"version": "5.0.0", "commit": "75709ca5", "branch": "version_5.0.0", "pre_release": true}

With a wheel install or windows installer, it should point to the correct place

it is a bit of a chicken or the egg problem for the pre-release logic in terms of tag existence