Phillips attended Charleston High School where he was a teammate of Larry Harris and Curt Price. While there, he averaged 19.7 points for a squad that posted a 72-3 record in three years. At 6-foot-3, Levi had the size and the ability to a good backcourt player. It was believed that he could easily penetrate a defense and make the big play. He had a tendency to glide over the floor and make every play look easy, not too fancy, but graceful. In addition, he was considered a good driver, ran well on the fast break type of player and had the savvy that would be valuable to a team.

His freshman year at WVU, he missed three games to a knee injury but was still able to finish the year as the second leading scorer for the freshmen team with a 17.2 average. He also tied for the most assists with 35 in 10 games. Heading into his sophomore year, it was believed that he could break into the starting line up if his knee held up. He went on to average 8.2 points per game in his sophomore year, but finished the year scoring 26 points in a win over Maryland and 19 in a loss to Florida, and his 79 assists lead the team. That year, he was considered to be very valuable to the team on the fast break because of his great value as an assist man. Unfortunately, Phillips was declared academically ineligible at mid-semester during his junior year in 1971-72. That year, he only played in 5 games. He returned to the basketball program after re-enrolling at WVU in the fall of 1972. After two semesters of study he was in good academic standing. According to Coach Moran, "Levi was perhaps the best all round performer in the WVU program." They were concerned about the one year layoff but they believed he was determined to make his senior year a good one.