Unique Words
Unique Words lets you list unique Unicode word tokens in their first-seen order. Enter your own values or load the worked example.
- Copy or download the computed result
- Adjust input text
- See the processing method and limits
Method and calculation
List unique Unicode word tokens in their first-seen order.
Inputs and settings
Unique Words accepts Input text.
Worked example
Unique Words example — Input text: "Hello world\nA second line\nHello world". Computed output: hello world a second line
Output and limitations
A repeated vocabulary list becomes one token per row. Matching ignores English letter case, so Apple and apple collapse into one lowercase entry. Tokenization retains apostrophes inside words but does not preserve punctuation or the layout of the original prose.
Technical references: JavaScript Unicode normalization
Unique Words FAQs
Unique Words applies the following rule to your input: List unique Unicode word tokens in their first-seen order. The worked example on this page uses the same processor as the result button.
Unique Words has these boundaries: Input is treated as plain text; it is never executed. Start by checking the input text field against the units or format shown.
Unique Words provides this sample: hello world a second line. Use the sample button to reproduce it, then replace the inputs with your own values.