The aspirate closes the inherited consonant order, and its position is an argument rather than an accident. The consonants are arranged from the back of the vocal tract forwards, and this sound is made at the vocal folds, which is further back than any point in the mouth, so on the organising principle it should come first. The tradition put it last instead, on the reasoning that it belongs with the sounds of audible friction rather than with the stops, and that the friction group follows the classed consonants. The two principles collide here and the tradition chose one and said so.
In the language the letter opens a family of common roots, among them the ones meaning to strike, to take away and to laugh, and it stands in a range of ordinary vocabulary besides. Historically a good many of its occurrences descend from older voiced aspirates that lost their stop closure, so a Sanskrit word with this letter often corresponds to a word with a voiced aspirate in the related languages.
Its behaviour in junction is irregular by the standards of the rest of the alphabet, and the grammatical tradition treats it as a special case in several rules. That is what one would expect of a sound made at the vocal folds rather than at a point of articulation in the mouth, since most of the rules are stated in terms of where sounds are made.