John Harvie’s childhood was spent at his father's 2,500-acre estate "Belmont," which adjoined the estates of "Edgehill," belonging to William Randolph, and "Shadwell" belonging to the surveyor, Peter Jefferson.  John was about the same age as Peter Jefferson's oldest son, Thomas.  They became good friends, the relationship growing closer when John's father was appointed as Thomas' guardian after Peter Jefferson's death.  Thomas Jefferson greatly influenced John's decision to study law and actively participate in the government of Virginia.

John was admitted to the bar before settling in Augusta County, where he built a successful law practice.  In 1774, he was named as a commissioner to the Shawnee tribe to negotiate a peace treaty after the Battle of Point Pleasant.  

Augusta County sent him to the Virginia conventions (the revolutionary legislature) in 1775 and 1776. The following year, that body sent him as a delegate to the Continental Congress.  He was a delegate to the Continental Congress in 1777 and 1778, where he signed the Articles of Confederation.  During the remainder of the revolutionary period, Colonel John Harvie was a purchasing agent and supply organizer for Virginia's Militia and Continental Army.

In 1780, he moved to Richmond, Virginia, where he became a land developer and builder, but remained active in government.

In 1789, he was chosen as an elector for the 1789 election from Henrico District.  That district consisted of Charles City, Goochland, Henrico, James City, Louisa, and New Kent Counties, east and west of Richmond.

Colonel John Harvie built several buildings in Richmond and public works in the area.  While inspecting one of these, the Gamble House, on February 6, 1807, he fell from a ladder and died.  He was buried in a family plot at his home.

His family's land later became part of the Hollywood Cemetery at Richmond.