The claim appears first in a text the churches never admitted to the canon. The Protevangelium of James, from about the middle of the second century, tells the story of Mary childhood, her dedication in the temple, her betrothal to an elderly widower with sons of his own, and a birth that leaves her virginity intact. Most of the later devotional material about Mary parents and her upbringing comes from this book rather than from the gospels, and so does the framework in which perpetual virginity became thinkable.
The doctrine was contested in the fourth century by writers, Helvidius among them, who held that Mary and Joseph lived as an ordinary married couple after the birth. Jerome answered with a treatise arguing that the brothers of Jesus were cousins, a solution that keeps Joseph as well as Mary continent, and his position became standard in the Latin West.
The tradition distinguishes three moments and affirms all three: virginity before the birth, in the birth and after it. The title aeiparthenos, ever-virgin, is used at the Second Council of Constantinople in 553, and the Lateran Synod of 649 states the three phases explicitly. It is affirmed today in Catholic, Eastern Orthodox and Oriental Orthodox teaching, and it appears in the liturgy far more often than in doctrinal argument, which is generally where it is actually transmitted.