added `init-image-strength` and `model` so they can be fully displayed on png info inspectors or stable diffusion browsers such as https://breadboard.me
just running `aimg --help` or `aimg --version` was very slow due to all the imports being brought in eagerly
Before changes `aimg --help`
`2.24s user 4.05s system 184% cpu 3.416 total`
After changes:
`0.04s user 0.02s system 8% cpu 0.625 total`
Used `PYTHONPROFILEIMPORTTIME=1 aimg --help` to find time consuming imports.
Also switched to using `scripts` instead of `entrypoints` since the scripts are much faster.
Made duplicate SAMPLER_TYPE_OPTIONS that can be loaded without loading all the samplers themselves.
Likely a breaking change - not sure.
Loading png images for captioning will cause the following error since PIL loads png images in RGBA mode.
```python
File "./miniconda3/lib/python3.8/site-packages/torchvision/transforms/functional_tensor.py", line 940, in normalize
return tensor.sub_(mean).div_(std)
RuntimeError: The size of tensor a (4) must match the size of tensor b (3) at non-singleton dimension 0
```
If a model has a remote filename with characters that might not be supported on
the local cache filesystem, replace them during translation of URL to cached
file path.
Closes#202.
If one uses `hf_hub_download` only referencing specific commits the `refs` folder will not be created even though data will be cached via `snapshots` and `blobs`. Subsequent calls to `try_to_load_from_cache` will return None even though the desired data was in the cache.
Example:
```python
# download something
hf_hub_download(repo_id=repo, revision=commit_hash, filename=filepath, token=token)
# returns None
try_to_load_from_cache(repo_id=repo, revision=commit_hash, filename=filepath)
```
https://github.com/huggingface/huggingface_hub/pull/1306
- feature: finetuning your own image models
- feature: image prep command. crops to face or other interesting parts of photo
- fix: back-compat for hf_hub_download
- feature: add prune-ckpt command
- feature: allow specification of model config file
- feature: negative prompting. `--negative-prompt` or `ImaginePrompt(..., negative_prompt="ugly, deformed, extra arms, etc")`
- feature: a default negative prompt is added to all generations. Images in SD-2.0 don't look bad anymore. Images in 1.5 look improved as well.
- feature: use different default steps and image sizes depending on sampler and model selceted
- fix: #110 use proper version in image metadata
- refactor: samplers all have their own class that inherits from ImageSampler
Previously the `+` and `-` characters in a mask (example: `face{+0.1}`) added to the grayscale value of any masked areas. This wasn't very useful. The new behavior is that the mask will expand or contract by the number of pixel specified. The technical terms for this are dilation and erosion. This allows much greater control over the masked area.
You can use `{}` to randomly pull values from lists. A list of values separated by `|` and enclosed in `{ }` will be randomly drawn from in a non-repeating fashion. Values that are surrounded by `_ _` will pull from a phrase list of the same name. Folders containing .txt phraselist files may be specified via
`--prompt_library_path`. The option may be specified multiple times. Built-in categories:
3d-term, adj-architecture, adj-beauty, adj-detailed, adj-emotion, adj-general, adj-horror, animal, art-movement,
art-site, artist, artist-botanical, artist-surreal, aspect-ratio, bird, body-of-water, body-pose, camera-brand,
camera-model, color, cosmic-galaxy, cosmic-nebula, cosmic-star, cosmic-term, dinosaur, eyecolor, f-stop,
fantasy-creature, fantasy-setting, fish, flower, focal-length, food, fruit, games, gen-modifier, hair, hd,
iso-stop, landscape-type, national-park, nationality, neg-weight, noun-beauty, noun-fantasy, noun-general,
noun-horror, occupation, photo-term, pop-culture, pop-location, punk-style, quantity, rpg-item, scenario-desc,
skin-color, spaceship, style, tree-species, trippy, world-heritage-site
Examples:
`imagine "a {red|black} dog" -r 2 --seed 0` will generate both "a red dog" and "a black dog"
`imagine "a {_color_} dog" -r 4 --seed 0` will generate four, different colored dogs. The colors will eb pulled from an included
phraselist of colors.
`imagine "a {_spaceship_|_fruit_|hot air balloon}. low-poly" -r 4 --seed 0` will generate images of spaceships or fruits or a hot air balloon
Credit to [noodle-soup-prompts](https://github.com/WASasquatch/noodle-soup-prompts/) where most, but not all, of the wordlists originate.
If input images didn't need resizing because they were already smaller than max width/height then they didn't get normalized to a multiple of 64. This caused an exception like the following:
```Sizes of tensors must match except in dimension 1. Expected size 4 but got size 3 for tensor number 1 in the list.
```
while the previous version did produce much better blending it also makes images that lack detail for some reason.
tests: Added more tests to help catch this sort of thing earlies
fix: found that median blur is really slow, so I made sure we only do it on downsampled masks. Was taking like 3 minutes to run on the large pearl girl picture on M1
- docs: update examples
- 🎉 fix: inpainted areas correlate with surrounding image, even at 100% generation strength. Previously if the generation strength was high enough the generated image
would be uncorrelated to the rest of the surrounding image. It created terrible looking images.
- fix: mask boundaries are more accurate
Specify advanced text based masks using boolean logic and strength modifiers. Mask descriptions must be lowercase. Keywords uppercase.
Valid symbols: `AND`, `OR`, `NOT`, `()`, and mask strength modifier `{*1.5}` where `+` can be any of `+ - * /`. Single-character boolean
operators also work. When writing strength modifies know that pixel values are between 0 and 1.
- feature: apply mask edits to original files
- feature: auto-rotate images if exif data specifies to do so
- fix: accept mask images in command line
If the x_sample was a bfloat on the gpu but the cpu doesn't support bfloat, that can cause a TypeError
```
File "/home/stdiff/.local/lib/python3.10/site-packages/imaginairy/api.py", line 292, in imagine
x_sample.cpu().numpy(), "c h w -> h w c"
TypeError: Got unsupported ScalarType BFloat16`
```
Seems to be caused by incompatible types in group_norm when we use autocast.
Patch group_norm to cast the weights to the same type as the inputs
From what I can understand all the other repos just switch to full precision instead
of addressing this. I think this would make things slower but I'm not sure. So maybe
the patching solution I'm doing is better?
https://github.com/pytorch/pytorch/pull/81852
- simpler logging suppression for `transformers` library
- suppress logging noise for running tests
- get test running for all samplers on mps and cuda platforms
- refactor safety model env variable to allow classification
- improved image logging functionality. can just stick log_latent wherever you want
- improved some variable naming
- moved all the samplers together
- vendored k-diffusion library
Bryce
cocktailpeanut
Pavel Yakovlev
Jiaming Sun
Mahmoud Al-Qudsi
mebelz
Paul Mestemaker
Kian-Meng Ang
Maxime Mouchet
Christian Käser
Jay Drennan
wseagar