The zucchetto, called a pileolus in Latin, has an unglamorous origin. Clergy in the Latin church wore the tonsure, a shaved crown, and medieval churches were cold and draughty, so a small cap covering exactly the shaved area came into use as a practical measure. The tonsure was discontinued as a general requirement in 1972; the cap stayed and became purely a mark of rank.
The colours are unambiguous. White for the pope, scarlet for a cardinal, amethyst for a bishop, and black where the custom of priests and deacons wearing one persists, which is now uncommon outside religious orders. Certain Eastern Catholic and Orthodox usages differ.
Like the mitre, it is removed at the most solemn part of the Mass, from the Preface until after communion, and it is taken off in the presence of the exposed Blessed Sacrament.
The custom that most people have encountered is the exchange. A visitor at a papal audience may bring a new white zucchetto of the correct size and offer it, and the pope will frequently put it on briefly, remove his own and hand it over. The visitor goes home with a cap that has been worn by the pope for four seconds. Popes vary in how readily they do this, and it is entirely informal.
It is worth noting what the zucchetto is not. It looks like a kippah and like the taqiyah worn by Muslim men, and there is no evidence of borrowing in any direction. Head covering for men in a religious context arose independently in several traditions, and in Christianity it arose in spite of a New Testament passage which says the opposite, that a man praying with his head covered dishonours his head.