Summary
When executing an ibis expression against Snowflake that includes NUMBER
columns, the resulting pandas DataFrame has object dtype instead of
float64, making arithmetic on the results fail without explicit casting.
Root cause
Snowflake NUMBER maps to Arrow Decimal128. When ibis calls .to_pandas()
on the Arrow result, PyArrow converts Decimal128 to Python decimal.Decimal
objects, yielding object dtype. There is no automatic promotion to float64.
Steps to reproduce
import ibis
con = ibis.snowflake.connect(...)
t = con.table("MY_TABLE") # has NUMBER(38, 8) columns
df = t.execute()
print(df["my_number_col"].dtype) # object, expected float64
df["my_number_col"].sum() # TypeError: unsupported operand type
Workaround
decimal_cols = [c for c, dtype in t.schema().items()
if "decimal" in str(dtype).lower()]
t = t.mutate(**{c: t[c].cast("float64") for c in decimal_cols})
Expected behavior
NUMBER columns should return as float64 in pandas. The Snowflake backend
could apply a Decimal → float64 promotion at the Arrow-to-pandas boundary,
consistent with how most analytical use cases treat numeric columns.
Summary
When executing an ibis expression against Snowflake that includes
NUMBERcolumns, the resulting pandas DataFrame has
objectdtype instead offloat64, making arithmetic on the results fail without explicit casting.Root cause
Snowflake
NUMBERmaps to ArrowDecimal128. When ibis calls.to_pandas()on the Arrow result, PyArrow converts
Decimal128to Pythondecimal.Decimalobjects, yielding
objectdtype. There is no automatic promotion tofloat64.Steps to reproduce
Workaround
Expected behavior
NUMBERcolumns should return asfloat64in pandas. The Snowflake backendcould apply a Decimal → float64 promotion at the Arrow-to-pandas boundary,
consistent with how most analytical use cases treat numeric columns.