Oglesby graduated from Booker T. Washington High School in Columbia, South Carolina. He then went to Ferrum Junior College (Virginia) where he played both the center and forward positions. As a freshman, he tallied 15.3 points and 12.6 rebounds per game in leading the squad to a 21-8 mark. During his sophomore year at Ferrum, he hit an amazing 57 percent of his field goals and averaged 22.6 points and 10.9 rebounds to lead the team in those departments. Ferrum was the only undefeated team in the National Tournament (24-0) and eventually wound up with a 25-2 record.

For his junior year, Oglesby transferred to WVU. He was considered an outstanding junior college player who was going to play a big part in any success the Mountaineers were going to have that season. He was only 6-foot-5, but played as if he were taller because of his timing and long arms. He was physically strong and possessed good agility. The big question when he arrived to the WVU program was how well he was going to blend in with the present WVU personnel and the offensive and defensive alignments. Sam ended up making a great transition to the WVU program and he led the team in rebounding with 238 while playing 873 minutes, a total which led the team. His 14.7 points-per-game average placed him only second to Wil Robinson that year.