The Interrator Operator Reliability

It's very hard to assign ground truth to the locations of change points in different time series. We invited data scientists who had experience with time series or change point sections itself, to annotate where they ieved that the change points in the time series were. And we did was evaluate the agridms against these multiple ground truths provided by the human annotators. It seems like, not that i have any special experience, but when i look at certain time seriess i feel i have an intuitive understanding of where there are some change points. But those also might be my bias in some weird way.