The Beta Chapter of Phi Lambda Epsilon was organized in 1894 as the first fraternity at the State Teacher's College in Warrensburg, Missouri. In the spring of 1914, the fraternity was disbanded because, after investigation, the faculty found that it only emphasized the social side of college life. During the next school year, some of the former members of Phi Lambda Epsilon, as well as some other male students who had high ideals regarding education, presented their intentions of forming a new fraternity to the faculty. Initially, this idea was disapproved, but through the encouragement and support of President Eldo L. Hendricks, Dean Claude A. Phillips, and Professor C. H. McClure, the faculty approved the establishment of the new fraternity. 

Upon reviewing the academic records of each student, the three felt that the group should affiliate with a nationally recognized fraternal organization stressing scholarship. Several established fraternities were contacted, but the requests to start a new chapter was declined because the State Teacher's College was not a qualified liberal arts college. The three professors then decided that if no national fraternal organization would recognize these students simply because they were enrolled in a teachers college, then they would create a fraternal organization unique unto itself. This fraternity would not stress only scholarship, leadership, or fellowship, but would be built on a tripod of all three qualities. 

On February 14, 1916, Phi Sigma Pi Honorary Professional Fraternity was founded. Dr. Hendricks was the first person to sign a Phi Sigma Pi roll book and is therefore known as Alpha 1. Together, Hendricks, Phillips, and McClure are known as the founding fathers of the fraternity. From the moment that Phi Sigma Pi was born, chapter members at Warrensburg intended to make the organization a national fraternity. Finally, on May 2, 1921, the fraternity became national when the Gamma Chapter was founded at Bradley University in Peoria, Illinois. From this point on, things began developing very rapidly. The Coat of Arms was redesigned and the Grand Seal was created in 1930, and the first copy of the Phi Sigma Pi newsletter was also published. The national song, "Brothers Are We" was copyrighted in 1933. In 1940, the Coat of Arms was redesigned and the Purple and Gold newsletter was created. 

In 1975, the Tau Chapter was informed that it was in direct violation of Final Title IX Regulation Implementing Education Amendments of 1972 Prohibiting Sex Discrimination in Education. Title IX argued that since Phi Sigma Pi was an honor fraternity, it must become co-educational since remaining an all-male organization would have a discriminatory effect upon the future career opportunities of eligible women. Three of the existing eight chapters threatened to withdraw from Phi Sigma Pi if women were admitted, and two were told by there respective universities that they would be shut down if they did not. At the National Convention in Washington D.C. on October 7, 1977 the National Constitution was amended to admit eligible females, and ensured that Phi Sigma Pi would remain an honor fraternity. The Tau Chapter was the first existing chapter to induct a female member, and the Sigma Chapter was the last chapter to admit women. 

By 1984 only six active chapters remained, as some chapters were unable to adjust to the gender change. By 1990 there were 20 active chapters, and many more targeted for development. With the increasing growth of Phi Sigma Pi, the National Council hired a full-time executive director in July 1990. In 1991, the Richard Cecil Todd and Clauda Pennock Todd Tripod Scholarship was established, and in 1992 the first National Headquarters building was purchased. In 1994 the first official alumni chapter, the Eastern North Carolina Alumni Chapter, was founded. In 1996 a permanent national headquarters was built in Lancaster, Pennsylvania.