-
-
Notifications
You must be signed in to change notification settings - Fork 19.4k
Description
Taking the example from #63417, but opening a separate issue to focus on just the behaviour with mismatching unit, regardless of performance issues.
pandas generally aligns on the index in all operations, except for comparison operations, where we raise an error for mis-aligned indices ("Can only compare identically-labeled Series objects").
But currently, two DatetimeIndex with the same values but different unit is apparently considered as "not identical", and thus you get errors like this:
dti = pd.date_range("2016-01-01", periods=3)
ser = pd.Series(1, index=dti)
ser2 = pd.Series(1, index=dti.as_unit("ns"))
>>> ser == ser2
...
File ~/scipy/repos/pandas/pandas/core/series.py:6656, in Series._cmp_method(self, other, op)
6653 res_name = ops.get_op_result_name(self, other)
6655 if isinstance(other, Series) and not self._indexed_same(other):
-> 6656 raise ValueError("Can only compare identically-labeled Series objects")
6658 lvalues = self._values
6659 rvalues = extract_array(other, extract_numpy=True, extract_range=True)
ValueError: Can only compare identically-labeled Series objectsWith pandas 3.0, since we now no longer default to nanoseconds in all cases (typically microseconds, but also other resolutions depending on how you created the timestamp data), I assume this would be more common.
And generally, my feeling is that the above comparison should Just Work?
cc @pandas-dev/pandas-core