Elijah Miller's ancestors came from Suffolkshire England. He was the second of 10 children and his early education took place at his mother's knee during the revolutionary war. After the War, he attended quaker schools and then went to the Academy at Williamstown (which later became Williams college). His parents moved to Romulus, New York and when Miller finished his schooling, he went there to study law with Daniel Shepard of Aurora. In 1798, Miller was selected to practice in the Supreme Court as an attorney. On January 1, 1800 he married Hanna Foote. He opened his own real estate law office and was considered to be one of the best lawyers in Western New York. He moved to Auburn in 1808 (Lazette and Frances, his only daughters, were born a few years earlier). He joined in the Episcopal Church despite his Quaker Upbringing. His wife died 3 years later. In 1813 Miller was appointed County Clerk and then selected to help choose the new prison site in Auburn as well as be the President of the board of inspectors. He was then appointed the First Judge of the Court of Common Pleas of Cayuga County In 1817, Miller supported the building of the first cotton mill in Auburn, using power from the Owasco River. In 1817 he built a new house in Auburn, Brigham Young, who was at the time a builder but later became the founder of Mormonism helped build the house. He partnered with Seward in his practice but focused on building his investments. He supported the building of canals and railroads throughout Western new York. In 1826 he was nominated for Congress but lost.