• Since the start of the year, the Nikkei 225 in Japan has gained more than 10%. In comparison, the pan-European Stoxx 600 has risen about 3.43% year-to-date, while the S&P 500 is up 4.2% during the same period.
• "There's a lot of potential in this market that's not appreciated by people," said Nicholas Smith, Japan strategist at CLSA.

Japanese stocks have outperformed their European peers for years, but investors have yet to realize its potential, strategists told CNBC.

Since the start of the year, the Nikkei 225 in Japan has gained more than 10%, as of its Thursday close. In comparison, the pan-European Stoxx 600 has risen about 3.43% year-to-date, while the S&P 500 is up 4.2% during the same period.

The Japanese market has performed "extremely well" since former Prime Minister Shinzo Abe took office for a second term in late-2012, Nicholas Smith, Japan strategist at CLSA, told CNBC's "Street Signs Asia" on Monday.

The Nikkei 225 on Monday crossed the 30,000 level for the first time in more than three decades. Still, some strategists have reportedly raised concerns of a market overheating, and declines in Friday trade brought the index below 30,000.

"There's a lot of potential in this market that's not appreciated by people," Smith said. "It's only in Japan that you could talk about a market being overheated when it's back to where it was 30 years ago."

John Vail, who is chief global strategist at Nikko Asset Management, agreed.

Japan "routinely outperforms Europe by a large margin," he said, adding that Japan is traditionally underweighted and "that's been a massive mistake — versus Europe at least."

Corporate governance in Japan

On the subject of corporate governance in Japan, CLSA's Smith said it's "certainly not world-class" at the moment and things need to improve.

"11% of companies were listed subsidiaries of listed entities and therefore, you need more independent directors in that kind of market," Smith pointed out.

Yet, large numbers of those listed subsidiaries have less than a third of independent directors, he added, citing the Asian Corporate Governance Association's 2020 biannual ranking that placed Japan in the fifth spot across Asia.

"Profits are doing very nicely, this is an attractive market, but it'll be a lot more attractive if they ... get things improved on corporate governance," Smith said.

Meanwhile, Vail said corporate governance has "improved massively," but admitted it was "not perfect."

"Overall, companies do care about profitability now," Vail said. "There's corporate governance problems in every region but somehow they get magnified in Japan."