A high-ranking North Korean delegation of Kim, and North Korea's nominal head of state, Kim Yong Nam, had arrived in South Korea earlier in the day aboard Kim Jong Un's private jet.
Pence was reported to have said he reached the new understanding with South Korean President Moon Jae-in, who has been pushing for diplomatic solution to the standoff over North Korea's nuclear and missile programs, in two substantive conversations during his visit to South Korea.
There is speculation Pyongyang could invite Mr Moon to visit for a summit later this year.
Although Kim Yo Jong ranks diplomatically lower than Kim Yong Nam, as Kim Jong Un's sister she is considered more important in North Korea.
However, no member of North Korea's ruling family has ever before visited the Blue House, which is near where North Korea founder Kim Il Sung - Kim Yo Jong's grandfather - sent dozens of commandos to assassinate former South Korean military strongman Park Chung-hee in 1968.
The nuclear-armed North is on an Olympics-linked charm offensive, sending athletes, a troupe of performers, hundreds of female cheerleaders, the ceremonial head of state and even Kim's sister to South Korea.
The trip by Kim Yo Jong is the latest move in an extraordinary show of Olympic diplomacy with Seoul that could prove to be a major challenge to the Trump administration's hard-line Korea policies.
Japanese Prime Minister Shinzo Abe is attending the opening ceremonies in part to reassure trilateral ties between South Korea, Japan and the US. Although the US, Japan and South Korea agreed to work closely with each other and continue pressuring North Korea, in September 2017 Moon's government announced plans to give $8 million to North Korea in order to provide humanitarian aid, despite the ongoing missile tests.
North Korea showcased new intercontinental ballistic missiles (ICBM) it tested previous year and a new type of short range missile during Thursday's large military parade, analysts said on Friday (Feb 9).
Another focal point is whether Kim Yo Jong has any message for Moon from her brother.
U.N. Secretary-General Antonio Guterres, who sat near Kim Yong Nam at the dinner, had a brief exchange with him and "reiterated his expectation and hope that all parties will use dialogue to achieve the denuclearization of the Korean Peninsula", U.N. deputy spokesman Farhan Haq said.
Usually held in April, the parade took place this year on the eve of the opening ceremony of the Winter Olympics being staged in the South Korean mountain resort of Pyeongchang, just across the heavily fortified border. North Korea is looking to use the opportunity of the Olympics to soften South Korea's attitude towards its northern neighbour and to find a way to bring South Korea to the negotiation table, thus weakening the US-Japan-South Korea trilateral cooperation which has been pressuring North Korea.
The North has refrained from further provocations since testing an intercontinental ballistic missile in November.
Pence says the US and South Korea are "completely aligned".
Moon and Kim Yo Jong had their first face-to-face encounter at the opening ceremony.
Pence is in South Korea, where he is leading the USA delegation to the games' opening ceremonies.
Dressed in long, white padded jackets, they followed the flag with a blue silhouette of the Korean Peninsula against a white background.