characters in the data that the map will process. As per customer requirement which character we have to use based on that we can take the tokens. . Create a Syntax Token for Western European Languages. Create a Syntax Token for Asian Languages. Note: We have some pre define tokens are available in Map Editor. Input file tokens/characters and pre defined tokens should be match.
If particular token is not available
in predefined then we should create syntax tokens as per requirements in Map Editor. To create Syntax Tokens for the Maps, follow below steps: Select string elementGo to Edit menuselect syntax tokens.