feature: add art-scene, desktop-background, interior-style, painting-style phraselists
fix: file globbing works in the interactive shell feature: make compilation animations simple slide showspull/288/head
parent
de2a72f718
commit
eb26d5a7c5
@ -0,0 +1,34 @@
|
||||
|
||||
## Notable Stable Diffusion Implementations
|
||||
- https://github.com/ahrm/UnstableFusion
|
||||
- https://github.com/AUTOMATIC1111/stable-diffusion-webui
|
||||
- https://github.com/blueturtleai/gimp-stable-diffusion
|
||||
- https://github.com/hafriedlander/stable-diffusion-grpcserver
|
||||
- https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines/stable_diffusion
|
||||
- https://github.com/lkwq007/stablediffusion-infinity
|
||||
- https://github.com/lstein/stable-diffusion
|
||||
- https://github.com/parlance-zz/g-diffuser-lib
|
||||
- https://github.com/hafriedlander/idea2art
|
||||
|
||||
## Image Generation Algorithms
|
||||
|
||||
| date | name | group | use type | FID | params |
|
||||
|------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|---------|----------|-----|--------|
|
||||
| 2023-02-22 | [Composer](https://damo-vilab.github.io/composer-page/) [[paper]](https://arxiv.org/pdf/2302.09778.pdf) [[code]](https://github.com/damo-vilab/composer) | Alibaba | Private | 9.2 | 3.4B |
|
||||
| 2022-11-24 | Stable Diffusion 2 | StabilityAI | Opensource
|
||||
| 2022-09-28 | Dall-E-2 | OpenAI | API |
|
||||
| 2022-08-22 | Stable Diffusion | StabilityAI | Opensource
|
||||
| - | Latent Diffusion |
|
||||
|
||||
|
||||
## Further Reading
|
||||
- https://github.com/vdumoulin/conv_arithmetic/blob/master/README.md
|
||||
- [Prompt Engineering Handbook](https://openart.ai/promptbook)
|
||||
- Differences between samplers
|
||||
- https://www.reddit.com/r/StableDiffusion/comments/xbeyw3/can_anyone_offer_a_little_guidance_on_the/
|
||||
- https://www.reddit.com/r/bigsleep/comments/xb5cat/wiskkeys_lists_of_texttoimage_systems_and_related/
|
||||
- https://huggingface.co/blog/annotated-diffusion
|
||||
- https://github.com/jessevig/bertviz
|
||||
- https://www.youtube.com/watch?v=5pIQFQZsNe8
|
||||
- https://jalammar.github.io/illustrated-transformer/
|
||||
- https://huggingface.co/blog/assets/78_annotated-diffusion/unet_architecture.jpg
|
@ -0,0 +1,141 @@
|
||||
## Todo
|
||||
|
||||
- Inference Performance Optimizations
|
||||
- ✅ fp16
|
||||
- ✅ [Doggettx Sliced attention](https://github.com/CompVis/stable-diffusion/compare/main...Doggettx:stable-diffusion:autocast-improvements#)
|
||||
- ✅ xformers support https://www.photoroom.com/tech/stable-diffusion-100-percent-faster-with-memory-efficient-attention/
|
||||
- https://github.com/neonsecret/stable-diffusion
|
||||
- https://github.com/CompVis/stable-diffusion/pull/177
|
||||
- https://github.com/huggingface/diffusers/pull/532/files
|
||||
- https://github.com/HazyResearch/flash-attention
|
||||
- https://github.com/chavinlo/sda-node
|
||||
- https://github.com/AminRezaei0x443/memory-efficient-attention/issues/7
|
||||
|
||||
- Development Environment
|
||||
- ✅ add tests
|
||||
- ✅ set up ci (test/lint/format)
|
||||
- ✅ unified pipeline (txt2img & img2img combined)
|
||||
- ✅ setup parallel testing
|
||||
- add docs
|
||||
- 🚫 remove yaml config
|
||||
- 🚫 delete more unused code
|
||||
- faster latent logging https://discuss.huggingface.co/t/decoding-latents-to-rgb-without-upscaling/23204/9
|
||||
- Interface improvements
|
||||
- ✅ init-image at command line
|
||||
- ✅ prompt expansion
|
||||
- ✅ interactive cli
|
||||
- Image Generation Features
|
||||
- ✅ add k-diffusion sampling methods
|
||||
- ✅ tiling
|
||||
- ✅ generation videos/gifs
|
||||
- ✅ controlnet
|
||||
- scribbles input
|
||||
- segmentation input
|
||||
- mlsd input
|
||||
- [Attend and Excite](https://attendandexcite.github.io/Attend-and-Excite/)
|
||||
- Compositional Visual Generation
|
||||
- https://github.com/energy-based-model/Compositional-Visual-Generation-with-Composable-Diffusion-Models-PyTorch
|
||||
- https://colab.research.google.com/github/energy-based-model/Compositional-Visual-Generation-with-Composable-Diffusion-Models-PyTorch/blob/main/notebooks/demo.ipynb#scrollTo=wt_j3uXZGFAS
|
||||
- ✅ negative prompting
|
||||
- some syntax to allow it in a text string
|
||||
- [paint with words](https://www.reddit.com/r/StableDiffusion/comments/10lzgze/i_figured_out_a_way_to_apply_different_prompts_to/)
|
||||
- https://github.com/cloneofsimo/paint-with-words-sd
|
||||
- https://multidiffusion.github.io/
|
||||
- images as actual prompts instead of just init images.
|
||||
- not directly possible due to model architecture.
|
||||
- can it just be integrated into sampler?
|
||||
- requires model fine-tuning since SD1.4 expects 77x768 text encoding input
|
||||
- https://twitter.com/Buntworthy/status/1566744186153484288
|
||||
- https://github.com/justinpinkney/stable-diffusion
|
||||
- https://github.com/LambdaLabsML/lambda-diffusers
|
||||
- https://www.reddit.com/r/MachineLearning/comments/x6k5bm/n_stable_diffusion_image_variations_released/
|
||||
- Image Editing
|
||||
- ✅outpainting
|
||||
- https://github.com/parlance-zz/g-diffuser-bot/search?q=noise&type=issues
|
||||
- lama cleaner
|
||||
- ✅ inpainting
|
||||
- https://github.com/Jack000/glid-3-xl-stable
|
||||
- https://github.com/andreas128/RePaint
|
||||
- ✅ img2img but keeps img stable
|
||||
- https://www.reddit.com/r/StableDiffusion/comments/xboy90/a_better_way_of_doing_img2img_by_finding_the/
|
||||
- https://gist.github.com/trygvebw/c71334dd127d537a15e9d59790f7f5e1
|
||||
- https://github.com/pesser/stable-diffusion/commit/bbb52981460707963e2a62160890d7ecbce00e79
|
||||
- https://github.com/SHI-Labs/FcF-Inpainting https://praeclarumjj3.github.io/fcf-inpainting/
|
||||
- ✅ text based image masking
|
||||
- ✅ ClipSeg - https://github.com/timojl/clipseg
|
||||
- https://github.com/facebookresearch/detectron2
|
||||
- https://x-decoder-vl.github.io/
|
||||
- Maskless editing
|
||||
- ✅ instruct-pix2pix
|
||||
-
|
||||
- Attention Control Methods
|
||||
- https://github.com/bloc97/CrossAttentionControl
|
||||
- https://github.com/ChenWu98/cycle-diffusion
|
||||
- Image Enhancement
|
||||
- Photo Restoration - https://github.com/microsoft/Bringing-Old-Photos-Back-to-Life
|
||||
- Upscaling
|
||||
- ✅ realesrgan
|
||||
- ldm
|
||||
- https://github.com/lowfuel/progrock-stable
|
||||
- [txt2imghd](https://github.com/jquesnelle/txt2imghd/blob/master/txt2imghd.py)
|
||||
- latent scaling + reprocessing
|
||||
- stability upscaler
|
||||
- rivers have wings upscaler
|
||||
- stable super-res?
|
||||
- todo: try with 1-0-0-0 mask at full image resolution (rencoding entire image+predicted image at every step)
|
||||
- todo: use a gaussian pyramid and only include the "high-detail" level of the pyramid into the next step
|
||||
- https://www.reddit.com/r/StableDiffusion/comments/xkjjf9/upscale_to_huge_sizes_and_add_detail_with_sd/
|
||||
- ✅ face enhancers
|
||||
- ✅ gfpgan - https://github.com/TencentARC/GFPGAN
|
||||
- ✅ codeformer - https://github.com/sczhou/CodeFormer
|
||||
- ✅ image describe feature -
|
||||
- ✅ https://github.com/salesforce/BLIP
|
||||
- 🚫 CLIP brute-force prompt reconstruction
|
||||
- The accuracy of this approach is too low for me to include it in imaginAIry
|
||||
- https://github.com/rmokady/CLIP_prefix_caption
|
||||
- https://github.com/pharmapsychotic/clip-interrogator (blip + clip)
|
||||
- https://github.com/KaiyangZhou/CoOp
|
||||
- 🚫 CPU support. While the code does actually work on some CPUs, the generation takes so long that I don't think it's
|
||||
worth the effort to support this feature
|
||||
- ✅ img2img for plms
|
||||
- ✅ img2img for kdiff functions
|
||||
- Other
|
||||
- Enhancement pipelines
|
||||
- text-to-3d https://dreamfusionpaper.github.io/
|
||||
- https://shihmengli.github.io/3D-Photo-Inpainting/
|
||||
- https://github.com/thygate/stable-diffusion-webui-depthmap-script/discussions/50
|
||||
- Depth estimation
|
||||
- what is SOTA for monocular depth estimation?
|
||||
- https://github.com/compphoto/BoostingMonocularDepth
|
||||
- make a video https://github.com/lucidrains/make-a-video-pytorch
|
||||
- animations
|
||||
- https://github.com/francislabountyjr/stable-diffusion/blob/main/inferencing_notebook.ipynb
|
||||
- https://www.youtube.com/watch?v=E7aAFEhdngI
|
||||
- https://github.com/pytti-tools/frame-interpolation
|
||||
- guided generation
|
||||
- https://colab.research.google.com/drive/1dlgggNa5Mz8sEAGU0wFCHhGLFooW_pf1#scrollTo=UDeXQKbPTdZI
|
||||
- https://colab.research.google.com/github/aicrumb/doohickey/blob/main/Doohickey_Diffusion.ipynb#scrollTo=PytCwKXCmPid
|
||||
- https://github.com/mlfoundations/open_clip
|
||||
- https://github.com/openai/guided-diffusion
|
||||
- image variations https://github.com/lstein/stable-diffusion/blob/main/VARIATIONS.md
|
||||
- textual inversion
|
||||
- https://www.reddit.com/r/StableDiffusion/comments/xbwb5y/how_to_run_textual_inversion_locally_train_your/
|
||||
- https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/sd_textual_inversion_training.ipynb#scrollTo=50JuJUM8EG1h
|
||||
- https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/stable_diffusion_textual_inversion_library_navigator.ipynb
|
||||
- https://github.com/Jack000/glid-3-xl-stable
|
||||
- fix saturation at high CFG https://www.reddit.com/r/StableDiffusion/comments/xalo78/fixing_excessive_contrastsaturation_resulting/
|
||||
- https://www.reddit.com/r/StableDiffusion/comments/xbrrgt/a_rundown_of_twenty_new_methodsoptions_added_to/
|
||||
- ✅ deploy to pypi
|
||||
- find similar images https://knn5.laion.ai/?back=https%3A%2F%2Fknn5.laion.ai%2F&index=laion5B&useMclip=false
|
||||
- https://github.com/vicgalle/stable-diffusion-aesthetic-gradients
|
||||
- Training
|
||||
- Finetuning "dreambooth" style
|
||||
- [Textual Inversion](https://arxiv.org/abs/2208.01618)
|
||||
- [Fast Textual Inversion](https://github.com/peterwilli/sd-leap-booster)
|
||||
- [Low-rank Adaptation for Fast Text-to-Image Diffusion Fine-tuning (LORA)](https://github.com/cloneofsimo/lora)
|
||||
- https://huggingface.co/spaces/lora-library/Low-rank-Adaptation
|
||||
- Performance Improvements
|
||||
- [ColoassalAI](https://github.com/hpcaitech/ColossalAI/tree/main/examples/images/diffusion) - almost got it working but it's not easy enough to install to merit inclusion in imaginairy. We should check back in on this.
|
||||
- Xformers
|
||||
- Deepspeed
|
||||
-
|
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,24 @@
|
||||
|
||||
blueberries
|
||||
fruit salad
|
||||
gold coins
|
||||
leaves
|
||||
autumn leaves
|
||||
wood grain
|
||||
tree bark
|
||||
sunflowers
|
||||
strawberries
|
||||
abstract art
|
||||
ferns
|
||||
jellybeans
|
||||
floral pattern
|
||||
the galaxy
|
||||
pebbles
|
||||
cobblestone
|
||||
stacks of books
|
||||
paint splatters
|
||||
a forest
|
||||
blue sky
|
||||
glitter
|
||||
diamonds
|
||||
grass
|
@ -0,0 +1,44 @@
|
||||
18th century european
|
||||
traditional european
|
||||
maximimalist
|
||||
minimalist
|
||||
modern
|
||||
traditional
|
||||
contemporary
|
||||
eclectic
|
||||
new traditional
|
||||
transitional
|
||||
midcentury modern
|
||||
art deco
|
||||
scandinavian
|
||||
farmhouse
|
||||
modern farmhouse
|
||||
bohemian
|
||||
industrial
|
||||
scandifornian
|
||||
french country
|
||||
shabby chic
|
||||
desert modern
|
||||
english countryside chic
|
||||
modern coastal
|
||||
new mediterranean
|
||||
naturalist
|
||||
southwestern
|
||||
coastal
|
||||
rustic
|
||||
modern rustic
|
||||
Art Nouveau
|
||||
gothic
|
||||
victorian
|
||||
indian
|
||||
moroccan
|
||||
japanese
|
||||
spanish
|
||||
maverick
|
||||
hollywood regency
|
||||
urban modern
|
||||
asian zen
|
||||
1950s
|
||||
futuristic
|
||||
Neoclassical ornate golden palace
|
||||
messy and outdated
|
@ -0,0 +1,13 @@
|
||||
abstract painting
|
||||
acrylic painting
|
||||
cave painting
|
||||
fresco painting
|
||||
impressionist painting
|
||||
mural painting
|
||||
oil painting
|
||||
post-impressionism painting
|
||||
realist painting
|
||||
still life painting
|
||||
surrealist painting
|
||||
tempera painting
|
||||
watercolor painting
|
Loading…
Reference in New Issue