Skip to content

bug: Snowflake: NUMBER columns return as object dtype via Arrow (Decimal128 not cast to float64) #11990

@zixiaoshawnshi

Description

@zixiaoshawnshi

Summary

When executing an ibis expression against Snowflake that includes NUMBER
columns, the resulting pandas DataFrame has object dtype instead of
float64, making arithmetic on the results fail without explicit casting.

Root cause

Snowflake NUMBER maps to Arrow Decimal128. When ibis calls .to_pandas()
on the Arrow result, PyArrow converts Decimal128 to Python decimal.Decimal
objects, yielding object dtype. There is no automatic promotion to float64.

Steps to reproduce

  import ibis
  con = ibis.snowflake.connect(...)
  t = con.table("MY_TABLE")  # has NUMBER(38, 8) columns
  df = t.execute()
  print(df["my_number_col"].dtype)  # object, expected float64
  df["my_number_col"].sum()         # TypeError: unsupported operand type

Workaround

  decimal_cols = [c for c, dtype in t.schema().items()
                  if "decimal" in str(dtype).lower()]
  t = t.mutate(**{c: t[c].cast("float64") for c in decimal_cols})

Expected behavior

NUMBER columns should return as float64 in pandas. The Snowflake backend
could apply a Decimal → float64 promotion at the Arrow-to-pandas boundary,
consistent with how most analytical use cases treat numeric columns.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions