According to Avoid Oversubscribing Threads, you set OMP_NUM_THREADS, MKL_NUM_THREADS, and OPENBLAS_NUM_THREADS.

You should also set NUMBA_NUM_THREADS and maybe more: https://github.com/joblib/joblib/blob/ed0806a497268005ad7dad30f79e1d563927d7c6/joblib/_parallel_backends.py#L59-L67