tokenizeraddspecialtokens的简单介绍

DoesPythonhaveanequivelenttotheJavaStringTokenizer？Ifso，whatisitandhowdoyouimplementit？ThanksKenFettigKenFettigwroteDoesPyt