Skip to content

Fix the RCNN export_onnx path under PIR - #9497

Open
aboccag wants to merge 1 commit into
PaddlePaddle:developfrom
aboccag:pr/rcnn-export-onnx-pir
Open

Fix the RCNN export_onnx path under PIR#9497
aboccag wants to merge 1 commit into
PaddlePaddle:developfrom
aboccag:pr/rcnn-export-onnx-pir

Conversation

@aboccag

@aboccag aboccag commented Aug 5, 2026

Copy link
Copy Markdown

Two one-liners that make export_onnx=True work again on the PIR path.

RPNHead carries a __shared__ config key export_onnx. With it set, the RCNN
family emits a straight-line, batch-size-1 graph — a Python loop over FPN
levels, then concat/topk/gather — instead of the batch loop with TensorArray
accumulation. That is what makes an ONNX export of a two-stage detector possible
at all, and it had simply never been run under PIR.

Each change restores consistency with code a few lines above it in the same
function:

  • rpn_head.pypaddle.shape(onnx_topk_rois)[0] is a 0-d tensor under
    PIR and the caller indexes rois_num[0], so the export fails with
    IndexError: list index out of range. The non-ONNX branch three lines up
    already slices [0:1].

  • post_process.pyconcat rejects the 0-d scalars that indexing
    scale_factor now yields: ValueError: The axis is expected to be in range of [0, 0). The non-ONNX branch already unsqueezes them, and carries a
    TODO(PIR) comment asking for exactly this fix.

Without these, faster_rcnn and friends export a graph containing while +
TensorArray + slice_array_dense, which no ONNX exporter can follow.

Verified on Paddle 3.3 with FLAGS_enable_pir_api=1: the whole two-stage family
exports, converts with paddle2onnx, and matches Paddle inference. For
faster_rcnn_r50_fpn_1x_coco, over 20 COCO images, 140 boxes compared, worst
pairwise IoU 0.999992 and worst score delta 2.7e-06.

RPNHead carries a __shared__ config key `export_onnx`. With it set, the RCNN
family emits a straight-line, batch-size-1 graph -- a Python loop over FPN
levels, then concat/topk/gather -- instead of the batch loop with TensorArray
accumulation. That is what makes an ONNX export of a two-stage detector
possible at all, and it had simply never been run under PIR.

Two one-liners, each restoring consistency with code a few lines away in the
same function:

  * rpn_head: `paddle.shape(onnx_topk_rois)[0]` is a 0-d tensor under PIR, and
    the caller indexes rois_num[0] -- IndexError: list index out of range. The
    non-ONNX branch three lines up already slices [0:1].

  * post_process: concat rejects the 0-d scalars that indexing scale_factor
    now yields -- "The axis is expected to be in range of [0, 0)". The
    non-ONNX branch already unsqueezes them, and carries a TODO(PIR) comment
    saying exactly this.

Without these, faster_rcnn and friends export a graph containing while +
TensorArray + slice_array_dense, which no ONNX exporter can follow.
@CLAassistant

CLAassistant commented Aug 5, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants