Hyphen #282

Split out of #187:

A related point is the treatment of hyphen:

To do:

• Do we need to suggest language-specific tokenisation (and provide details as to when to treat hyphen as a word character)?
• Do we need to handle e.g. U+2010 as well as ASCII hyphen (U+002D which is also used as a minus sign)? Similarly to apostrophe, we ideally want to avoid having charset-specific variants of an algorithm (we used to and they got out of step in some cases) so probably the answer is to treat characters that can't be encoded in the specified character set as characters we won't see, at least in some cases.
• Update https://snowballstem.org/texts/introduction.html
• Create a page like https://snowballstem.org/texts/apostrophe.html (or adjust this to be about word tokenisation more generally).
• arabic
• armenian
• basque
• catalan (has rules to remove attached pronouns such as: `'-te')
• danish
• dutch (The original C version of the Kraaij-Pohlmann has some code to remove hyphens in some cases which is not currently in the Snowball version)
• dutch_porter (provided for compatibility with older Snowball so probably shouldn't change)
• english
• esperanto ("a suffix may follow a non-Esperanto word if separated by a hyphen"; optionally also for suffixes to stems containing digits)
• estonian
• finnish
• french
• german
• greek
• hindi
• hungarian _If the last letter of a foreign word is silent (not pronounced) or part of a complex cluster of letters, a hyphen is used when attaching suffixes (e.g. guillotine-nal 'with a guillotine', Montesquieu-vel 'with M.'). If an adjective is formed from a proper name with only one element, it will be lowercase (e.g. voltaire-es 'Voltaire-esque'
• indonesian Hyphen used in reduplication - e.g. anak-anak meaning "children" is the plural of anak meaning "child"; Some reduplications change the prefix and/or suffix and/or vowel sounds
• irish (has a rule to remove prefixes: 'h-' 'n-' 't-' //nAthair -> n-athair, but alone are problematic)
• italian
• lithuanian
• lovins
• nepali
• norwegian
• persian (Persian doesn't use hyphen)
• polish
• porter (reference implementation so shouldn't change)
• portuguese (glossary says "in Portuguese [a-suffixes] are separated by hyphen from the preceding word, which makes them easy to eliminate")
• romanian (algorithm description says "It is assumed that hyphenated forms are split into separate words prior to stemming.")
• russian (stop.txt entry: нибудь | indef. suffix preceded by hyphen)
• serbian
• spanish
• swedish
• tamil
• turkish
• yiddish

Algorithms not annotated above currently have no special treatment of hyphens.