Pandas Promotes Int To Float When Filtering

Pandas seems to be promoting an int to a float when filtering. I've provided a simple snippet below but I've got a much more complex example which I believe this promotion leads to

Solution 1:

There is no float comparison happening here. isin is returning NaN's for missing data, and since you are using numpy's int64, the result is getting cast to float64.

In 0.24, pandas added a nullable integer dtype, which you can use here.

---

df1 = df1.astype('Int64')
df2 = df2.astype('Int64')

df1[~df1.isin(df2)]

   col1  col2
0   NaN   NaN
1   NaN   NaN
2   NaN   NaN
3     4    13
4     5    14

---

Just be aware that if you wanted to use numpy operations on the result, numpy would treat the above as an array with dtype object.