Thomas Walker was born in Rye Field, King & Queens County, Virginia January 15, 1715 the son of Maj. Thomas Walker and Susannah Peachy. The Walkers of Virginia came from Staffordshire, England about 1650 at an early period in the history of the colony of Virginia. Major Walker was a member of the Colonial Assembly 1662, being at that time a Representative from the County of Gloucester. This gentleman, in 1663, claimed that he planted 70,000 mulberry trees and therefore requested bonuses for silk culture. In 1667 following the report of a committee of the House of Burgess sent to count his trees, he was awarded 20,000 pounds of tobacco for his efforts. After studying medicine with his sister's husband Dr. George Gilmer, Thomas set up practice in Fredericksburg and became a noted physician. He also ran a general store and engaged in an import and export trade. In 1741 married Mildred Thornton Meriwether. She was born March 19, 1721 in Louisa, Virginia, the daughter of Col. Francis Thornton and Alice Savage. Mildred married twice, first to Nicholas Meriwether III and then by marriage, Thomas Walker acquired 11,000 acre estate known as Castle Hill.

Castle Hill played host for a short time to the British enemy Banastre Tarleton on June 4, 1781 during the midst of the American Revolution. There Tarleton made a short stay, but was delayed by the insisting of Mildred Walker. This delay gave the young Jack Jouett of Louisa County enough time to reach Charlottesville and send a messenger to warn Thomas Jefferson and her legislators staying at Monticello, who escaped just in time safely into Staunton, Va. in the Shenandoah Valley. Tarleton's short visit at Castle Hill proved to be a critical moment in the Revolution by saving members of the General Assembly and giving the citizens of Charlottesville time to prepare or flee.

In 1749 Thomas became chief agent of Loyal Land Company, which had received a grant of 800,000 acres from the council of Virginia and in the following years he led an expedition to explore lands of this grant. He kept a journal of the trip which was the first record of a white man in what was to become Kentucky. In 1775, during the French and Indian Wars, he became Commissary to Virginia troops under George Washington and was later charged with fraud, but acquitted. Dr. Thomas Walker served on the Committee of Safety in Virginia. In 1777 he was appointed with his son Col. John Walker to visit Indians in Pittsburgh, Pa. for the purpose of gaining their friendship for the Americans. In 1776 the Virginia House of Delegates defined the northern boundary of the Kentucky District as the low-water mark at the mouth of the Big Sandy, on the northern shore of the Ohio River. This boundary followed the Big Sandy River from that point to the junction of the Tug Fork, and from there up to the Laurel Ridge of the Cumberland Mountain to the point where it crossed the Virginia-North Carolina line (known as "seven pines and two black oaks). When Virginia agreed to separate Kentucky in the Compact of 1789, that description was accepted.

In 1779-80, The Virginia-North Carolina dividing line was extended westward to the first crossing of the Cumberland River. From this point west to the Mississippi, Thomas Walker surveyed the line for Virginia. This took him through dense forests, over rugged mountains - a most difficult task. This line almost immediately caused a tremendous amount of dispute for many years between Kentucky and Tennessee. When Kentucky became a state in 1792, it immediately began to "find fault" with the line as drawn by Thomas Walker in 1779. The difficulties continued madly into the 1850's.

In 1859, the Cox-Pebbles team traveled a 320 mile course between January 9th and October 20th. It covered the same terrain that Walker's party had traveled from New Madrid Bend to the Cumberland Gap. They erected 3 foot high stone slabs every five miles to mark the line - beginning at Compromise on the Mississippi River and ending at the spot where the old Wilderness Road passed through the Cumberland Gap. The line is basically what it was as marked by Walker in 1799.
Dr. Thomas Walker died November 9, 1794 at Castle Hill, Albemarle County Virginia, at 79 years of age. Mildred died November 16, 1778 at Castle Hill, Albemarle County, VA, at 57 years of age. They are buried on the estate. The cemetery is situated near the foot of the mountain in the woods surrounded by a brick wall.