Fillmore did not veto any bills during his presidency.

Although Fillmore sought election to a full term in 1852, he had alienated his own party by enforcing the Fugitive Slave Act and was passed over by the Whigs in favor of Winfield Scott, an American military commander known as "Old Fuss and Feathers" who was soundly defeated by Democrat Franklin Pierce.

The Know Nothing Party (or American Party) was primarily an anti-Catholic, anti-immigration, and xenophobic movement, originally started as a secret society. Adherents to the movement were to simply reply "I know nothing" when asked about its specifics by outsiders, providing the group with its common name. During his 1856 candidacy as that party's nominee, Fillmore won only Maryland.

Fillmore considered his political career to be at an end after his defeat in 1856. However, his financial worries were removed on February 10, 1858, when he married Caroline McIntosh, a well-to-do widow. Their combined wealth allowed them to purchase a large house on Niagara Square in Buffalo, where they lived for the remainder of his life.