Every tradition that practises pilgrimage also contains a strand that objects to it, and the objection is the same everywhere: if the sacred is everywhere, then the journey is either unnecessary or a substitute for what actually matters.
Islam holds the tension inside its own structure. The hajj is obligatory once for those able, which is as strong an endorsement as a practice can receive, and yet Muslim reformers have repeatedly attacked the visiting of other shrines and tombs as illegitimate, with the argument running from medieval jurists through to modern Salafi teaching and the destruction of tomb structures in Arabia.
Judaism built its pilgrimage around a Temple that has not existed since 70 CE, and the rabbinic tradition then rebuilt the religion so that it no longer required one. The Western Wall is a place of prayer rather than a pilgrimage destination in the older sacrificial sense, and treating the two as equivalent misdescribes what happened after the destruction.
Protestant Christianity rejected pilgrimage outright at the Reformation along with relics and shrines, and the dismantling of English shrines in the 1530s was thorough and deliberate. Protestant practice then regenerated the form under other names: the revival meeting, the mission convention, the tour of the Holy Land.
Buddhism has the sharpest doctrinal problem, because the Buddha is not present anywhere to be visited. The tradition answers by treating the sites of his life, and the relics distributed after his cremation, as occasions for the right state of mind rather than as sources of power in themselves. In practice the distinction blurs continuously, and popular devotion at Bodh Gaya looks a great deal like popular devotion at any major shrine anywhere.
The honest comparative statement is that the objection is old, indigenous to each tradition rather than imported into it, and has never once stopped anyone going.