Model Manifest #582
MFlux Model Manifest:
Recommend a MFlux Model Manifest (MMM) which is the central lookup YAML file in the code.
- IMO it is a terrible idea to ask a newbie user to download a massive BF16 safetensor model to convert it themselves into MFlux format so they can actually use MFlux.
- We need to default to a suitably sized MFlux model from HF (MFlux-Community Org) - which is referenced in the Model Manifest
- Default steps then also go in the Model Manifest
- Include upstream BF116 safetensor HF model name, License etc etc
- (future) adding a new model to MMM can trigger CI to add the full model quant suite to MFlux HF collection
- We could even offer value-add features like supported/suggested LoRAs .. also on MFlux HF collection
- MFlux users with custom LoRAs could simply alter the YAML model config... and have their custom LoRAs available (ideally with CLI auto-completion)
All reactions
Replies: 11 comments 17 replies
All reactions
I don't understand why you tacked this onto my PR, only to duplicate it here, which duplicated me creating an issue base don your comment...
All reactions
Good question
Current State:
notice issue >> create Issue ticket >> code
Proposed Future State:
notice issue >> create Discussion >> Discuss/Socialize >> Agree >> Prioritize >> create Issue ticket >> code
IMO Currently everyone on MFlux is rushing too much - this is is sub-optimal and often resulting is misteps that waste time and/or need redos (nvidia PiD, --metadata & --no-metadata)
To improve the overall product quality and development effort efficiency, IMO we need to slow down a little bit and discuss things before rushing in and coding them.
Further the product is drifting... not being intentionanlly designed - this is not a recipe for success. Yes, we need to be responsive to SOTA and what users want... but no one can do good product design by just shooting from the hip - without consideration and discussion.
All reactions
That's just more paperwork.
feat -> discussion first, PR after
something broken/buggy -> issue, PR to fix issue
Agree on the propensity to rush, it's how we end up breaking things and frustrating each other. :)
All reactions
Yes,
feat -> discussion first, PR after
something broken/buggy -> issue, PR to fix issue
aaah: the classic tension beetween young devs versus old devs
Young devs just want to write code (even if it's the wrong code)
Old devs & product people... want to slow things down and make a good product design decisions, avoid wasting time, avoid technical debt, make a better product
But like the Tortoise and the hare fable goes... "slow and steady wins the race"
Fire, fire, fire, ready, aim, fire, fix mistake, fix mistake, delete code (techical debt)
versus
Ready, Aim, Fire
This is not about introducing process, bureaucracy, or paperwork... it is about making good design decisions so we can code efficiently and make a good product.
All reactions
I can’t tell if you're lumping me in as a "young dev", being sarcastic, agreeing with me, or if a human even wrote that. I'm a human, using tools to grunt work, but reviewing and testing everything I touch. Have a group discussion about every bug fix is excessive. It either passes a PR review, or doesn’t; that is the documentation and discussion, so having another discussion is where I push back. This isn't "seasoned practice > newbie rush to market"; it's "we are doing this for 0ドル and free time is valuable".
I'm already frustrated as hell with the sheer fire hose of prose in every comment, edit, conversion, etc., all almost certainly spewed out by LLMs not instructed for brevity (yes, I'm guilty too). If this is the cadence you all want, I'll check out. The dual CI modules spitting literal pages of text on everything they touch, each, is just a headache.
All reactions
@plz12345,
I'm talking more abstractly about tendencies, approaches, cultures. Sometimes development team cultures shoot from the hip. Sometimes they have a more considered and methodical culture. Sometimes like you say, there's too much paperwork and process.
You clearly are a really good developer and I'm not making any criticism or comment about you personally. I'm giving my genral observation about the way the MFlux dev team might be approaching things... and suggesting a slightly difference approach.
100% agree: A group discussion about every bug fix is absurd. I'm certainly not suggesting that!!
I am proposing that we consider and discuss before adding new models and new functionality - that we design them so they work better (eg resolving --metadata & --no-metadata before it is PR approved and released code)
Yeah, the AI generated slop PR comments are WAY more verbose than necessary!! I find reading AI slop really hardwork, makes me tired. Not enjoyable.
Dual CI sounded like a good idea... but yeah. It is really exhausting.
Agree. I am not enjoying the fire hose PR cadenace.
BUT: We (you guys) have broken the back of the backlog. We're down to nearly zero PRs and issues. YAY!!
I say we hit the pause button. Have some human conversations about what to do next...
Of course not everyone speaks English. Personally I'd be happy if people speak from their heart in their OWN language... I'm happy to AI translate it myself a few different ways myself to really understand what they're saying.
THB: I am happy though because I think we've started to have the conversations about some stuff that really matters; what we focus on, how we work together, how we communicate, etc.
BTW: Until further notice, everything I write is 100% human written - unless it is discosed/stated otherwise.
All reactions
this is why I am in "audit mode" this week making sure the project gets a health check before declaring a new release with so many recent landed changes.
I make the effort to write or dictate human responses such as this paragraph right here, but for the pull requests I'm going to let the AI do the grunt work of making line-by-line changes. I try to guide the PR comments to be well formatted, well targeted at the intended reader, and formatted to be optimal for human reading.
All reactions
-
👍 2
Yes, all of my comments are 100% my own words, though 80% AI dictated.
All reactions
oh boy...
All reactions
yeah, sorry. Talking about AI is boring.
I wonder how many billion people are having the same conversations at the moment?!
All reactions
on the manifest itself, two things worth deciding early.
the cost isn't the yaml, it's whether everything else derives from it. if MODEL_CHOICES, MODEL_INFERENCE_STEPS, the completions list and the mflux-save dispatch are computed from the manifest, adding a model is one entry and this pays for itself. if they stay hand-written next to it, it's the sixth table and we're worse off. the drift bugs from last week all came from that same shape.
and we already have the disappearing-model case in the tree. microsoft/Lens was withdrawn from HF, so lens-turbo resolves to Comfy-Org/Lens, a third-party mirror. that's a supported model whose weights live or die by someone else's copy, and today it's one line in model_config.py that nobody would look at until a user's download fails. a manifest with upstream + mirror declared, plus a scheduled job that HEADs every declared repo, turns that into something we find out about instead of them.
which is also the strongest argument for the pre-quantized side of your proposal: an org copy is what keeps a model alive after the original goes away. worth being deliberate there though, republishing a checkpoint the vendor pulled is a call to make on purpose, not a CI default.
as for requiring a model on HF before a PR lands, I'd make it a field, not a gate. gemma is gated, flux1-dev is non-commercial, ideogram too, so a hard rule blocks exactly the models with the awkward licenses. and uploading to the org collection needs write access there, so the gate would mean only people with those credentials can port a model. CI can publish the quant suite after merge where the license allows it.
All reactions
yeah-no
MFlux-Converter converts & quantizes models using "main".
We only want to be generating the HuggingFace library from known good, tested and released versions of MFlux. It is not intended for dev tests.
We don't want to be polluting the HuggingFace library with potentially bad models. There's GPU costs involved and a lot of fiddling around to delete bad model sets - so I really don't want to go in that direction.
HF is a "Prod" release
All reactions
Naming constistency is easily solved
We lock in all naming in Step 2 - "MFlux maintainers discuss and approve or reject.. documenting in GH Discussions as a record"
"a contract" as the AIs call it
All reactions
Then it's a race condition, as we don't want a new model addition to be a 3-stage process, ether. So adding a model means we add the family/model to manifest, and then your tooling needs to use that naming when you create the BF16 repo, quants, etc.
All reactions
what's a "race condition".. pls explain
for reference.. pls see the model config artifacts currently behind the MFlux-Converter: https://huggingface.co/buckets/mflux-community/ci
All reactions
chicken/egg scenario == race condition
All reactions
|
MFlux-Converter config files: https://huggingface.co/buckets/mflux-community/ci
|
All reactions
Proposed new model workflow
1) Someone suggests a new model is supported.
2) MFlux maintainers discuss and approve or reject.. documenting in GH Discussions as a record
exact names, paths are locked in - python objects, CLI, upstream source, etc etc
3) MFlux maintainer (@ianscrivener) clones the model to a private MFux-Community HF Collection... and adds the models to Model Manifest. This avoids any missing models or gated model issues for development and archives the model for the future (avoind the MS/Lens withdrawn style issue)
4) MFlux Dev codes the model integrations as usual
5) MFlux maintainer (@ianscrivener) creates a fulll MFlux quant suite (using MFlux-Conv app)
3 & 4 can occur in parallel
IMO: MFlux should be supporting select good models... not every model that exists.