tokenize
Смотреть что такое "tokenize" в других словарях:
tokenize — verb To reduce to a set of tokens by parsing … Wiktionary
Лексический анализ — Запрос «Лексема (информатика)» перенаправляется сюда; см. также другие значения. В информатике лексический анализ процесс аналитического разбора входной последовательности символов (например, такой как исходный код на одном из языков… … Википедия
Лексер — В информатике лексический анализ процесс аналитического разбора входной последовательности символов (например, такой как исходный код на одном из языков программирования) с целью получения на выходе последовательности символов, называемых… … Википедия
Лексический анализатор — В информатике лексический анализ процесс аналитического разбора входной последовательности символов (например, такой как исходный код на одном из языков программирования) с целью получения на выходе последовательности символов, называемых… … Википедия
Applesoft BASIC — was a dialect of BASIC supplied on the Apple II computer, superseding Integer BASIC. Applesoft BASIC was supplied by Microsoft and its name is derived from the names of both Apple and Microsoft. The first version of Applesoft was released in 1977 … Wikipedia
GFA BASIC — is a dialect of the BASIC programming language, by Frank Ostrowski. The first version was finished in 1986. In the mid and late 80 s, it became very popular for the Atari ST homecomputer range (since the ST BASIC shipped with them was more… … Wikipedia
TI-BASIC — is the unofficial name of a BASIC like language built into Texas Instruments (TI) s graphing calculators. , , (Except for the TI 85 and TI 86 variants, TI only refers to it as the calculators extensive programming capability .) Though the TI… … Wikipedia
Tag (metadata) — For other uses, see Tag. A tag cloud with terms related to Web 2.0 In online computer systems terminology, a tag is a non hierarchical keyword or term assigned to a piece of information (such as an Internet bookmark, digital image, or computer… … Wikipedia
Just BASIC — is a dialect of the highly popular BASIC programming language of the 1970 s for 32 bit Windows computer systems. It s the freeware version of Liberty BASIC, popular since 1992. It allows you to create programs with a comprehensive tutorial, help… … Wikipedia
Repetition blindness — (RB) is a phenomenon observed in Rapid Serial Visual Presentation. Subjects are less likely to detect the repetition of a target stimulus than they are to detect a second, different target.For example, a subject s chances of correctly reporting… … Wikipedia
Shunting yard algorithm — The shunting yard algorithm is a method for parsing mathematical equations specified in infix notation. It can be used to produce output in Reverse Polish notation (RPN) or as an abstract syntax tree (AST). The algorithm was invented by Edsger… … Wikipedia
