[Bug]: RT-DETRv2 model does not produce output on GPU

[mdtoom]

OpenVINO Version

2024.6.0.17404

Operating System

Ubuntu 20.04 (LTS)

Device used for inference

GPU

Framework

ONNX

Model used

https://drive.google.com/drive/folders/1EWAeGVq3-Blh4pCxUVxoN-oDRZU9dxHx?usp=sharing

Issue description

I have converted the default RT-DETRv2-S model (model - config) to ONNX using the command for the RT-DETR repo (https://github.com/lyuwenyu/RT-DETR/tree/main/rtdetrv2_pytorch):

python tools/export_onnx.py -c configs/rtdetrv2/rtdetrv2_r18vd_120e_coco.yml -r configs/rtdetrv2/rtdetrv2_r18vd_120e_coco_rerun_48.1.pth --check

And then converted the model to openvino using:

ovc model.onnx --input image[1,3,640,640]

to get a model.xml and a model.bin, a link to which you can find in the model used.

When I then run detections on those as described in the step-by-step reproduction I get correct detections when running on CPU, for example:

and no detections on GPU.

I expect the detections to be the same.

CPU: 12th Gen Intel(R) Core(TM) i7-1265U
GPU: Alder Lake-UP3 GT2 [Iris Xe Graphics]

Step-by-step reproduction

Run the following code:

import numpy as np
import cv2 as cv

from openvino import Core

def preprocess(im, target_size=(640, 640)):
    origin_shape = im.shape[:2]
    resize_h, resize_w = target_size
    im_scale_y = resize_h / float(origin_shape[0])
    im_scale_x = resize_w / float(origin_shape[1])
    out_im = cv.cvtColor(im, cv.COLOR_BGR2RGB)
    out_im = cv.resize(
        out_im.astype("float32"),
        None,
        None,
        fx=im_scale_x,
        fy=im_scale_y,
        interpolation=cv.INTER_LINEAR,
    )

    scale = 1.0 / 255.0
    out_im *= scale
    out_im = out_im.transpose((2, 0, 1)).copy()
    return np.expand_dims(out_im.astype("float32"), 0)


def draw(images, labels, boxes, scores, thrh=0.6):
    for i, im in enumerate(images):

        scr = scores[i]
        lab = labels[i][scr > thrh]
        box = boxes[i][scr > thrh]

        for b in box:
            top_left = int(b[0]), int(b[1])
            bottom_right = int(b[2]), int(b[3])

            print(f"{top_left} {bottom_right}")
            cv.rectangle(im, top_left, bottom_right, (0, 255, 0), 3)
            cv.putText(im, str(lab[i]), top_left, cv.FONT_HERSHEY_COMPLEX, 1, (0, 0, 255), 2, cv.LINE_AA)

        cv.imwrite(f"results_ov_{i}.jpg", im)



def main(args):
    """main"""

    im_pil = cv.imread(args.im_file)
    org_shape = (im_pil.shape[0], im_pil.shape[1])

    im = preprocess(im_pil)

    ie_core = Core()
    model = ie_core.read_model(model=args.model_file)
    compiled_model = ie_core.compile_model(model=model, device_name=args.device)

    inputs = dict()
    inputs["images"] = np.array(im).astype("float32")
    inputs["orig_target_sizes"] = (
        np.array(org_shape)[::-1].reshape(1, 2).astype("float32")
    )

    results = compiled_model(inputs=inputs)

    draw([im_pil], results["labels"], results["boxes"], results["scores"], 0.6)


if __name__ == "__main__":
    import argparse

    parser = argparse.ArgumentParser()
    parser.add_argument("--model-file", type=str)
    parser.add_argument("--im-file", type=str)
    parser.add_argument("--device", "-d", type=str, default="CPU")
    args = parser.parse_args()
    main(args)

with

python3 <file> --model-file model.xml --im-file <image> -d GPU

and you expect the same detections as when you run it on a CPU

python3 <file> --model-file model.xml --im-file <image> -d CPU

But in GPU there are no detections.

Relevant log output

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.