During the Saturday meeting, Moon also called on North Korea to créate a space to dialogue with the United States about denuclearization of the Korean peninsula.
Meanwhile, Fred Warmbier, father of Otto Warmbier, an American student who died aged 22 after being detained in North Korea for a petty offence previous year, was not in the box with Pence but rather sitting with the U.S. athletes, according to The Washington Post Tokyo bureau chief Anna Fifield.
Pence stood to welcome the US athletes as the Korean pop hit Gangnam Style blared around the stadium, sparking the "Horse Dance" in the crowd and among the volunteers.
North Korea has spent years developing its military, saying it needs to protect itself from USA aggression.
The invitation was delivered through his younger sister Kim Yo-jong who met Moon at the presidential palace.
"As we speak, an estimated 100,000 North Korean citizens labor in modern-day gulags", Pence said. "That's the team he's going to be cheering for all Olympics".
North and South Korea are technically still at war after their 1950-53 conflict ended in a truce rather than a peace treaty.
If Moon accepts Pyongyang's invitation and meets with the North Korean leader, it would be the 3rd inter-Korean summit.
Moon has been invited to visit Pyongyang for what may become the first top-level summit in over a decade.
In South Korea for the start of the Winter Olympics Friday, U.S. Vice President Mike Pence met with defectors from North Korea - accompanied by the father of Otto Warmbier, an American who died after his release from captivity there previous year.
Prior to the opening ceremony on Friday night, Pence arrived late to a dinner reception between the two Koreas and greeted everyone at a main table except for Kim Yong Nam, North Korea's nominal head of state, Reuters reports.
The event was hosted by South Korean Unification Minister Cho Myoung-gyon for North Korea's high-level delegation.
US President Donald Trump said "all options remain on the table" to ensure North Korea abides by worldwide calls to give up its nuclear development programme.
After a chaotic year of nuclear war threats and nuclear and missile tests from the North, it was a striking visual moment.
After two failed Olympic bids that emphasized the high-sounding notion that the games could help make peace with North Korea, Pyeongchang finally sold its successful try in 2011 on the decidedly capitalistic goal of boosting winter sports tourism in Asia. Already, there are concerns in other corners over how Moon's government has accommodated the North Korean delegation, even at the risk of violating worldwide sanctions in place against Pyongyang.
Asked if she thinks that some of the North Korean athletes might try to defect while at the Winter Games, Choi told Williams they know the government would take their families hostage.
On September 3 2017 North Korea detonated its most powerful nuclear blast to date, sending tensions on the peninsula higher than a mushroom cloud.
The liberal Hankyoreh newspaper played up her direct influence on Kim Jong Un, and interpreted the trip as an attempt by North Korea to reset its global relations and improve ties with both Seoul and Washington.