I agree 100%; of all the "real actors" in I and II, Corey gives the best performance. I mean, Goro is probably my personal favourite, and there are a handful of small-part characters with cool voices in the first game: Chai, Lan Di, Terry, Yamagishi-san, homeless harbour guy.
Even Troy Baker couldn't have saved Shenmue from the "I'm looking for sailors/Do you know anything about Chinese people" memes. It's not Corey's acting, it's the bizarre localisation, combined with tech limitations of the time. Repeatedly asking the same weird question to NPCs, and some terribly hilarious VA perpetrated by other "actors" (I'm looking at you, NPCs) -- that's what gives Shenmue its bad rep.
Also, yes, I'd like to see him return, but if it's the anime that ends up "saving" the series then it also probably makes sense to recast Ryo as the anime guy. It'd be weird but I'd deal with it...by playing in Japanese regardless