Using mathics-scanner¶
This is used as the scanner inside Mathics but it can also be used for tokenizing and formatting Wolfram Language code. In fact we intend to write one. This library is also quite usefull if you need to work with Wolfram Language named character and convert them to various formats.
For tokenizing and scanning Wolfram Language code, use the
mathics_scanner.Tokenizer
class.To convert between Wolfram Language named characters and Unicode/ASCII, use the
mathics_scanner.characters.replace_wl_with_plain_text
andmathics_scanner.characters.replace_unicode_with_wl
functions.To convert between qualified names of named characters (such
FormalA
for\[FormalA]
) and Wolfram’s internal representation use them̀athics_scanner.characters.named_characters
dictionary.