How to force ComfyUI to use xformers?

[Snimm]

When I installed comfy it showed loading xformers [version] when I started it. I started messing with the flags because I had trouble loading the refiner, however I was not able to turn on xformers after. There is no flag to turn on xformers just to disable it. When I load it even with no flags I still don't see xformers showing up, How do I fix it? How to force Comfy to use xformers?

I do have xformers installed. pip install xformers shows requirement already satisfied, I am on Ubuntu, RTX 3060 GPU with 16 RAM.

[comfyanonymous]

If it's not using xformers it either means it's not installed or you launched it with a parameter like: --use-pytorch-cross-attention

[Snimm]

@comfyanonymous
I am not using any flags. However it seems to be using pytorch cross attention anyway. I have not edited any config files.

python main.py
Total VRAM 12050 MB, total RAM 15951 MB
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3060 : cudaMallocAsync
Using pytorch cross attention
Starting server

To see the GUI go to: http://127.0.0.1:8188

Xformers was being used when I first installed it. To show xformer is installled.

(sd) (base) sonnet@sonnet-MS-7A70:/media/sonnet/AAAA42E0AA42A8A1/Ai/ComfyUI$ pip install xformers
Requirement already satisfied: xformers in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (0.0.20)
Requirement already satisfied: numpy in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from xformers) (1.25.1)
Requirement already satisfied: pyre-extensions==0.0.29 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from xformers) (0.0.29)
Requirement already satisfied: torch==2.0.1 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from xformers) (2.0.1)
Requirement already satisfied: typing-inspect in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from pyre-extensions==0.0.29->xformers) (0.9.0)
Requirement already satisfied: typing-extensions in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from pyre-extensions==0.0.29->xformers) (4.7.1)
Requirement already satisfied: filelock in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (3.12.2)
Requirement already satisfied: sympy in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (1.12)
Requirement already satisfied: networkx in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (3.1)
Requirement already satisfied: jinja2 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (3.1.2)
Requirement already satisfied: nvidia-cuda-nvrtc-cu11==11.7.99 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (11.7.99)
Requirement already satisfied: nvidia-cuda-runtime-cu11==11.7.99 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (11.7.99)
Requirement already satisfied: nvidia-cuda-cupti-cu11==11.7.101 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (11.7.101)
Requirement already satisfied: nvidia-cudnn-cu11==8.5.0.96 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (8.5.0.96)
Requirement already satisfied: nvidia-cublas-cu11==11.10.3.66 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (11.10.3.66)
Requirement already satisfied: nvidia-cufft-cu11==10.9.0.58 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (10.9.0.58)
Requirement already satisfied: nvidia-curand-cu11==10.2.10.91 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (10.2.10.91)
Requirement already satisfied: nvidia-cusolver-cu11==11.4.0.1 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (11.4.0.1)
Requirement already satisfied: nvidia-cusparse-cu11==11.7.4.91 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (11.7.4.91)
Requirement already satisfied: nvidia-nccl-cu11==2.14.3 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (2.14.3)
Requirement already satisfied: nvidia-nvtx-cu11==11.7.91 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (11.7.91)
Requirement already satisfied: triton==2.0.0 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from torch==2.0.1->xformers) (2.0.0)
Requirement already satisfied: setuptools in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch==2.0.1->xformers) (67.7.2)
Requirement already satisfied: wheel in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch==2.0.1->xformers) (0.40.0)
Requirement already satisfied: cmake in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from triton==2.0.0->torch==2.0.1->xformers) (3.27.0)
Requirement already satisfied: lit in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from triton==2.0.0->torch==2.0.1->xformers) (16.0.6)
Requirement already satisfied: MarkupSafe>=2.0 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from jinja2->torch==2.0.1->xformers) (2.1.3)
Requirement already satisfied: mpmath>=0.19 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from sympy->torch==2.0.1->xformers) (1.3.0)
Requirement already satisfied: mypy-extensions>=0.3.0 in /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages (from typing-inspect->pyre-extensions==0.0.29->xformers) (1.0.0)

[2blackbar]

ITs not using xformers like it should so you wont see speed gain
Should be half the speed, with xformers 1.5 secs, without xformers 3 secs, its like this for me in auto1111 but here it just does not work even if it says its installed

[http202]

Just found the solution, there is missing xformers folder in ComfyUI_windows_portable\python_embeded\Lib\site-packages, i simply copied xformers and xformers-0.0.22.post4.dist-info (just to be safe) folders from AppData\Local\Programs\Python\Python310\Lib\site-packages, for you it would probably be /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages., with no flags comfyui started using xformers by default.

[ghostsquad]

I'm running a 3060, and just got started in ComfyUI (coming from A1111). Xformers has a significant performance improvement. Would love to see this fixed in a standard way. 🙂

[http202]

i guess the only fix is for developers of standalone comfyui include xformers by default, or make it search and remember path for already installed version on launch (since its already checks for it on install ¯_(ツ)_/¯).
Automatic1111 also does this: i have ver 22 instaled but it uses ver 20 in \side-pakcages folder which it includes by default (you could just copy from a1111 folder since you have it)

[comfyanonymous]

Xformers is no longer shipped with the standalones because most of the functionality is a part of pytorch now so it's not needed anymore.

[LIQUIDMIND111]

Xformers is no longer shipped with the standalones because most of the functionality is a part of pytorch now so it's not needed anymore.

so there is no way of making it "faster' woth SDP or XFORMERS?

[ltdrdata]

Xformers is no longer shipped with the standalones because most of the functionality is a part of pytorch now so it's not needed anymore.

so there is no way of making it "faster' woth SDP or XFORMERS?

You can install xformers manually. Then ComfyUI will use xformers automatically. However, using xformers doesn't offer any particular advantage because it's already fast even without xformers.

[stephantual]

I did some thread necromancy to find this. It might be worth pointing that out in the loading log on server start, because I was scratching my head as to 'why wasn't it using xformers' followed by 'but why is it using cross attention even though i didn't flag it'.

We all have FOMO on speed :) Good to know we're not missing anything out!

[currenself89]

lol I can imagine you scraching your head

[joebreaker]

Just found the solution, there is missing xformers folder in ComfyUI_windows_portable\python_embeded\Lib\site-packages, i simply copied xformers and xformers-0.0.22.post4.dist-info (just to be safe) folders from AppData\Local\Programs\Python\Python310\Lib\site-packages, for you it would probably be /media/sonnet/AAAA42E0AA42A8A1/Ai/sd/lib/python3.10/site-packages., with no flags comfyui started using xformers by default.

Dear friend, you've just saved my mind! I've almost lost my faith by this day! Thank you so much! Dancing)))

[ZeroBomb]

On my machine, today, with fully up-to-date everything (pytorch 2.4, CUDA 12.4), generations take roughly 2-to-3x slower without xformers. It is not the case that xformers is somehow no longer needed.

That being said, there is definitely a need to have a force-xformers flag. I have just finished compiling/installing xformers, only for Comfy to decide not use it for some unknown reason. However, when I run Comfy with the --quick-test-for-ci flag, it confirms that xformers is available to the system.