Is There a Problem With P Value?

In the old days in statistics, we have something called data dredging. It's kind of a bad idea to look around a large data s looking for significance. In machine learning and literature, you are not so obsessed about over fitting. What matters is that when you think you discover something, you should validate it. I don't think there is any need to try to rigorously come up with a p value or anything like that. Whatever you say is always provisional by a scientific attitude. But on the other hand, we cannot be arbitrary, because some difference is just due to chance.