The 74-year-old was arrested after photos of him playing checkers at an outdoor table were posted to Facebook last year by a Thai man who was impressed by his tattoos.
The fugitive, Shigeharu Shirai, 72, was arrested by a SWAT team on Wednesday in the sleepy central Thai market town of Lopburi while he was out shopping.
Shirai will face illegal entry charges before he is handed over to Japanese authorities, the police statement said.
However, while Shirai confessed to being a gang-leader, he did not admit murdering his rival.
Photographs posted online by a passerby quickly went viral - and that was when everything went wrong for Shigeharu Shirai.
Police have also said they would investigate other suspected Yakuza gangsters in Thailand even if they did not have an arrest warrant.
The suspect has intricate tattoos all over his body and is missing the little finger on his left hand - both signs of his yakuza membership.
The gangs have been part of Japanese society for centuries now, and have an estimated 60,000 members.
The yakuza emerged in the chaos of postwar Japan, transforming into multibillion-dollar criminal organisations involved in gambling, drugs, prostitution, loan sharking, protection rackets and white-collar crime.
Unlike the Italian Mafia or Chinese triads, yakuza are not illegal and each group has its own headquarters in full view of police. The gangs calls themselves ninky? dantai [chivalrous organisations].
They were tolerated as a necessary evil to keep order on the streets - however dubious the means.
Police General Wirachai Songmetta said Mr Shirai's Japanese associates paid visits to him two to three times a year, each time bearing cash gifts at around $397. He's married to a Thai woman and kept a low profile over the past decade, police said.
He was detained in Lopburi (which is north of Bangkok) for entering Thailand illegally, as he had no passport or visa.