Learning to navigate by the Sun, the Moon and the stars has recently risen to the top of my list of things to do, and I thought it might be useful to describe the results of my study thus far. Be warned that this is the product of about two days of searching around the Web, and that my experience in practical navigation is nil, and much of the terminology in what follows is entirely non-standard.
In any field there is a body off essential knowledge, assumption and fact I call "the Lore". This is frequently hidden behind thickets of obscure terminology, historical usage, and sometimes actual falsehood. Furthermore, much of it is held implicitly by practitioners, which makes it difficult for clewless newbies to get the a hang of it. My first goal in approaching any new area is to identify and make explicit the Lore of the field, and this work is an example of that.
The goal of celestial navigation is to use prior knowledge about the angular positions of the Sun, the Moon and the stars relative to the Greenwich Observatory to establish your own angular position (latitude and longitude). The zero of longitude (the Prime Meridian) runs through Greenwich, and the time as measured on the Prime Meridian, subject to various arcane corrections we needn't go into, is known as "Universal Time" (UT).
Celestial navigation has two basic steps: taking a sight, and reducing a sight. Taking a sight is the act of measuring a celestial body's angle above the horizon at a known Universal Time, typically using a sextant and chronometer. The angle of a body above the horizon is known as its "altitude". Reducing a sight is the act of calculating the possible positions on the surface of the Earth from which that celestial body would have that angle above the horizon at that time.
In understanding the Lore of any field, it is useful to keep in mind the scales of various phenomena. As Aristotle said, "The student should not expect more precision than the subject matter admits of," and this is particularly true when dealing with quantitative subjects.
Science is the discipline of publicly testing ideas by systematic observation and controlled experiment, and while quantification is not necessary to this discipline, it is an extremely powerful tool, because any aspect of any phenomenon that can be quantified can be subject to far more powerful tests and far more systematic observations than otherwise. So quantification is good, and understanding scales is a useful adjunct to quantification.
The scales in celestial navigation are set by the size of the Earth and the acuity of the human eye, perhaps aided by a telescope of modest magnification. The magnification of a sextant eyepiece must be modest because the field of view must be large. Otherwise it would be impossible to use the instrument on the pitching deck of a boat or ship.
In practice, it has proven difficult to measure the altitude of celestial bodies with a precision of more than 0.2 minutes of arc. Because of the rather clever relationship between angle, time and distance, this translates to 0.2 of a nautical mile, or about 560 m, although in reality total errors of 1 - 5 nm seem to be more common, especially for a plastic sextant of the kind I'm buying.
Although I am generally in favour of SI (metric) units, navigation is the one field where the case for metric is more difficult to make.
Our measures of time and angle go back to the Sumerians, whose basic unit of time was the "beru", translated as "double-hour". This is not an accident: a beru is precisely two of our hours, because both are derived from the rotation of the Earth and divisions of the sky.
The Sumerians divided the sky into twelve parts based on stars that rose and set with the Sun at different times of year, one per lunar month, more-or-less. Or rather, "less", as there are 364.25/29.53 = 12.33 lunar months per year. These "heliacal rising" stars were the original clock, provided by careful, quantitative observation of nature by Sumerian savants.
As well as the division of the sky into twelve beru, the Sumerians seem to have noticed that each day the fixed stars completed about 1/360th of their course around the heavens. Precisely what went on is unknown, but it seems reasonable that this had something to do with the adoption of a base-60 (sexagesimal) numbering system for measuring both time and angle, which has come down to us from five thousand years ago essentially unaltered. 60 is a very friendly number, having many divsors, including both 12 and 24.
In modern times, the nautical mile (nm... I trust no one will get it confused with nano-meters in this context) has been defined to equal the distance on the surface of the Earth that subtends one minute of arc, leading to the aforementioned identity between 0.2 degree precision in angular measure and 0.2 nm precision (but not necessarily accuracy!) in navigation. A nautical mile is about 1850 m, as opposed to 1600 m for a statute mile.
Other scales to keep in mind: the Earth has a radius of 6371 km, although it is not precisely spherical due to rotational effects that cause it to bulge about 10 km at the equator, flattening the poles by around 20 km. The distance from Earth to Moon is four hundred thousand kilometers and the distance from Earth to Sun (1 AU) is 150 million kilometers.
The angle subtended by the Moon when viewed from Earth is 0.51 degrees. The angle subtended by the Sun when viewed from Earth is 0.53 degrees. Both of these vary slightly due to orbital eccentricity, but we are concerned here with scales only. "Half a degree" will do for both.
The Sun moves across the sky at 0.25 degrees per minute, the Moon somewhat more quickly. So the Sun traverses its own angular diameter every two minutes.
Having never actually taken a sight, I'm going to skip over this bit for now, although reading over the manual that comes with the Davis Mark 15 I have on order makes me think that my years as an experimentalist aren't entirely for nought.
My goal in my studies in the past couple of days has been to come to a reasonably deep understanding of the process of actually finding a position based on the altitude of a celestial body. The following diagram sums up everything I've figured out so far:
I had a lot of fun creating this diagram and am quite pleased with it as a summary of the process of finding a position from an altitude. You can click on the diagram to get a blowed-up version.
The different colours indicate the different stages of measurement and reduction. Red shows what we actually measure: the apparent or observed altitude of the celestial body, which for now I'm going to assume is the Sun or a star. The Moon is a special case for reasons I'll get to below. If a sight is taken of the Sun, the angle between the lower limb and the horizon is what is typically used.
The observed altitude is a measure of the angle between incoming light from the celestial body and the direction of the horizon. This is shown in red. However, on our spherical Earth the horizon is actually slightly below the horizontal for an observer whose eye is above sea level. This effect is wildly exaggerated in the diagram, allowing our stick-figure to see far around the curve of the globe, but a little bit of elementary trigonometry shows that the dip angle of the horizon is given by
dip in radians = acos(Re/(Re+T)) ~ sqrt(2T/Re) where
T is the height of the eye and
Re is the radius of the Earth, and the approximate relation is based on a Taylor expansion of cosine and dropping second order terms in T. [Multiply this result by 57.3 to get degrees.]
[edit 2011-04-19... corrected for arithmetic error in angle conversion]
For an eye height of even 2 m this gives a dip correction of about 2.7' (minutes of arc), and since we're bothering to measure down to 0.2' this is worth caring about.
Tabulated dip corrections are about ten times the values I am getting with these equations. They include a correction for refraction that is either very significant, or I have made a mistake. It has been known to happen.
In my original calculation I managed to goof in converting radians to degrees. It is in my experience easier to err in the "simple" parts of a calculation or argument where one tends to pay less attention. With the correct conversion factor, my formula agrees with the Nautical Almanac, giving 1.92*sqrt(T) for the dip in minutes of arc with T in meters.
The canonical equation, which includes a refractive correction, is that dip in minutes of arc is 1.56*sqrt(T) where T is in meters (the factor in front is 0.97 for feet).
As well as a refractive correction to the dip correction, there is a refractive correction to the altitude itself. The refractive index of air is only slightly different from unity: 1.00029. The refractive index of a material is the ratio of the velocity in light in vacuum to the velocity of light in the material, and as the light slows down ever so slightly while entering the atmosphere it bends according to Snell's Law.
The actual situation is quite complicated, not a simple kink at the top of the atmosphere as shown in the diagram. The pressure drops off exponentially with height and temperature matters as well, but there are tables for this correction, and in any case it only matters when the sight is reasonably close to the horizon, meaning light has entered the top of the atmosphere at a steep angle and traveled a long way since. Light incident from the zenith, in contrast, is unbent.
From what I've seen, the refraction correction is about 9' at 5 degrees above the horizon, falling to 5' at 11 degrees and 2' at 20. It is negligible above 35 degrees or so, unless you're insanely precise (and hey, who isn't?)
As well as the tabulated refractive correction there are corrections for the semi-diameter of the Sun (16') and parallax corrections for the Moon. The Moon is only 400,000 km away, and the radius of the Earth is 6371 km, so depending on where you are viewing the Moon from it will have a different apparent location relative to the fixed stars. Since the point of celestial navigation is to translate your position into the frame of the fixed stars (and then back to terrestrial co-ordinates via the Universal Time value) any apparent motion of the sighted object against the fixed stars has to be accounted for.
All of this is done to get the true altitude of the celestial body: its angle above the local horizontal. This is shown in blue on the diagram, and marked with "True Altitude" (theta). Note that the local zenith angle of the body, alpha-prime, is complementary to theta, and therefore alpha-prime = 180-theta.
For various stars and the Sun and the Moon, the Nautical Almanac for each year contains the point on Earth at which they stand at the zenith for each hour of the year. One presumes there are also computer programs that perform these calculations in perpetuity, although a self-sufficient information source like ink on paper is likely to be far more reliable than a computer... the whole point of celestial navigation is to free us of dependency on black boxes, particularly black boxes run by the American military.
In a well-ordered world, we would have a program capable of generating all the relevant information, including tables of correction, in a nice booklet format for a given time period to cover a cruise. Think of the trees we would save!
The tabulated zenith position of the celestial body is shown in green on the diagram, and marked as the "ephemeral position" both because it is constantly changing and we get it from an ephemeris such as the Nautical Almanac based on the Universal Time at which we took our sight.
The thing that we want to know is the angle alpha, shown near the center of the Earth on the diagram: this is the angle between the point on the Earth's surface where the celestial body is at the zenith, and the point where we are.
A little bit of reflection will show that alpha is more-or-less exactly equal to the zenith angle alpha-prime for all bodies but the Moon. For more distant bodies, including the Sun, the angle beta (shown on the diagram near the celestial body) between the ray at the zenith and the ray at our location is negligibly small, which means the only thing that matters is the difference in horizon angle at the two locations. For the Moon things are more complicated, and I won't go into them here.
We have a single angle to find our position on a two-dimensional surface, which never works. What we know is that we are somewhere along a circle, shown edge-on as the yellow line in the diagram. An observer at any point on this circle around the ephemeral position of the celestial body will see the body at the same true altitude.
What I have called the "circle of position" is usually called the "line of position" in the celestial navigation literature.
At this point one can do several things. If it is dark and the seeing is good one can take two more star sights. The circles of position derived from this should approximately intersect, and the degree of approximation will give an estimate of position error. This is by far the best thing to do: no scientist worthy of the name ever makes a single measurement and calls it a day. Scientists have a hard-earned distrust of their own results, and as such like to have checks on them, because unlike most people, scientists sometimes make mistakes. But then, unlike most people, we also (sometimes, eventually, mostly, so far) find them and fix them, so that's all right then.
Alternatively one can take the intersection or closest approach between the line of position and one's dead-reckoning course, and use that as an updated position estimate.
Finally, if this is a sun sight at local Noon, both latitude and longitude are determined by it, as you have the added constraint that the Sun is at its highest point in the sky at the Universal Time of the sight, which uniquely identifies your position along the circle.
Two days ago I didn't know any of this, other than in the general terms of geometry and whatnot, and I have done a bit of astronomical observing and dead-reckoning navigation. Most of what I have written here may be wrong. I'll check back in a couple of years after I have more practical experience and see what I missed and where I goofed.
My main goal today was to create the diagram included here, as it captures the Lore of celestial navigation as I understand it in one reasonably complete picture. It makes clear why the various corrections have the signs they do, and why parallax is only important to the Moon.
By fixing the geometry in my head this way I will be able to apply the various corrections and perform the other operations of sight reduction while keeping in mind the actual, physical situation. As a physicist, I use mathematics as a natural language for describing the world, and like any natural language it is far more powerful in the hands of someone fluent in the intentionality of the concepts behind the utterances.