Mathematicians commonly use three different terms (other than perpendicular) to represent the idea of perpendicularity; orthogonal, normal, and right, but they all come from a common idea.
Ortho was the greek root for erect, or vertical, hence something is orthogonal when it is vertical (in terms of some other base which is the ground). We see this root preserved in "orthocenter" for the point where the altitudes of a triangle intersect, "orthodox" for an accepted practice, and "orthopteran", an order of insects characterized by "straight wings".
A recent thread on the Historia-Matematica discussion group pointed out that Euclid did not address the concept of the orthocenter in the Elements. Archimedes did address the concept, although not by that name, in Lemma 5 of the Liber Assumptorum. Dick Tahta suggested that Lemma 12 is a more appropriate source for the Orthocenter.
A posting by Emili Bifit provides a quote from an article by John Satterly that gives credit for the creation of the name "Orthocenter" to Besant and Ferrers in 1865.
"Note_: As a matter of historical interest our readers may be reminded that the term ``Orthocentre'' was invented by two mathematicians, Besant and Ferrers, in 1865, while out for a walk along the Trumpington Road, a road leading out of Cambridge toward London. In those days it was a tree-lined quiet road with a sidewalk, a favourite place for a conversational walk."
From Mathematical Gazette, Feb 1962, pp 51
Norma was the latin word for what we now call a carpenter's square. It was used to construct lines which were at right angles to another line, so the created line was said to be "normal". The norma was also used as a standard to compare if objects, like a wall, might be erect (perpendicular to the ground) and so those that met the standard were called "normal" and this use extended to the "typical" element of any type of set. Eventually normal came to mean anything that "met the standard".
The latin word for the greek term ortho, was rectus which was also used to mean both straight and erect. We see the imprint of rectus in many math words and common language with both the straight and erect meanings. The rectum is so called because it is the "straight intestine", while a rectangle is a parallelogram with an "erect angle". In fact, the word rectangle was sometimes used for a "right angle" into the 19th century.
The latus rectum in a parabola is the side (latus) through the focus that is straight (rectus - parallel to the directrix). Latus rectum is the Latin translation of the Greek orthea pleura for erect side which was the term Apollonius used in his books on the conic sections. As languages blended in the middle ages, rectos became "recht" and eventually became our word for "right" and for the right angle. The idea of vertical as the "right" position led to the use of right as proper or good. We see this in words like correct, which means, literally, to make straight (or right). The person who is reckless (less right) makes hasty decisions without regard for consequences.
As words changed in the middle ages lots of blending of tones led to various modern spellings of older terms. Many words beginning with the reg prefix are variants of rectus, such as regular (straight) and regent (the one who makes the regulations).
The word perpendicular itself might be translated into English as "very hanging down". It comes from the Latin name for a plumb bob perpendiculum. The three roots are per which is used to give an emphasis of thoroughness or extremeness (hence my "very"), pend which is related to the hanging or weighing and is related to the word for pound and the obvious pendulum. The third root is culum which was an instrumental suffix.
The use of the symbol for perpendicular, according to Cajori, was in 1634 by Pierre Hérigone in his cursus matematicus.