Old Norse has several words in this area and uses them loosely: jotunn, thurs, risi and troll all appear, sometimes of the same figure in the same poem. Giant is the standard English rendering of the first and it imports a meaning the sources do not emphasise. Size is occasionally mentioned and is almost never what a story turns on.
Three things do the work instead. The first is age and priority: the jotnar were there before the gods, Ymir is the material the world is made from, and their claim on things is older. The second is knowledge. Odin's pursuit of wisdom takes him repeatedly to that side, to Mimir's well, where he leaves an eye, and to a riddle contest with Vafthrudnir that he wins only by asking a question nobody else could answer. The third is possession. The mead of poetry is held by Suttungr and has to be stolen. Idunn and her apples are taken and recovered. Thor's hammer is taken by Thrymr, who names a bride price. A cauldron large enough for the gods' ale is in Hymir's hall.
That last pattern is the most consistent thing about them. The gods want something, a jotunn already has it, and the gods obtain it by theft, disguise or force. Read without the assumption that one side is good, a large part of Norse myth is a record of the Aesir acquiring by trickery what the older family held.
The etymology of jotunn is generally connected with eating, which fits a class of beings characterised by appetite and by having what others want.