这段源码一共切出多少个 Token
手在哪儿停,切出来就是两样东西
一串字符
停早了
停晚了
按玩具语言的规则,把下面这段源码切成 Token,输出总共有几个。
let rate = 12; let msg = "hi there"; total = rate >= 3;
玩具语言的词法规则:关键字 let if else while;标识符以字母或下划线开头;数字是连续的数位;字符串用一对双引号括起来(不含转义);双字符运算符 >= <= == !=;单字符运算符 + - * / = < > ; ( );空白只做分隔。
KEYWORDS = ("let", "if", "else", "while")
TWO = ("==", "!=", ">=", "<=")
ONE = "+-*/=<>;()"
SRC = 'let rate = 12;\nlet msg = "hi there";\ntotal = rate >= 3;'
import re
pat = re.compile(r'\s+|[A-Za-z_]\w*|\d+|"[^"]*"|==|!=|>=|<=|[+\-*/=<>;()]')
n = 0
for m in pat.finditer(SRC):
if not m.group().isspace():
n += 1
print(n)
全部评论