Natural Language Processing: Regular Expressions - Deepstash

Keep reading for FREE

Natural Language Processing: Regular Expressions

  • /word/ - matches any string containing the substring "word". Note: it's case sensitive.
  • /[ab]/ - disjunction of characters: "a" or" b"
  • /cat|dog/ - disjunction of string cat and dog. To be read as: String containing "cat" or "dog"
  • /pupp(y|ies)/ - To be read as: string containing "puppy" or "puppies"
  • /[a-z]/ - shows an interval: any letter between "a" and "z"
  • /[^a] / - "^" shows negation when it's the first character inside brackets. To be read as: not containing "a"
  • /^The/ - Here "^" matches only the string which starts with the expression that follows it. E.g. The dog barks.
  • /s.$/ - "$" matches the end of the line. E.g. expression that matches any string ending with .s
  • /words?/ - "?" implies zero or one instances of the previous character. E.g. word or words
  • /b.y/ - "." shows any character between b and y. E.g. for the expression "busy life", the answer would be "us"
  • /a*/ - "*" shows any string of 0 or more "a"'s. To be read as: "String containing no a's or 1 ore more a's
  • /[ab+] / - "+" shows 1 ore more occurences of the previous character. To be read as:Any character containing 1 ore more ab's.
  • /\bthe \b/ / "\b" matches a word boundary. To be read as "matches a string which contains "the" isolated by other words. E.g. "in the club" but not "the other", 
  •  \d - contains any digit
  • \D - contains any non-digit
  • {n} - n occurences of the previous char or expression
  • {,n} - up to n occurences of the previous char or expresssion
  • {n, m) - from n to m occurences of the previous char or expression

Natural Language Processing: Word Formation

Morphology is the study of word structure.

Words consist of one or more morphemes (cats= cat+ s). A morpheme is the smallest unit of language.

Morphemes can be of several types:

  • stem (can stand on its own)
  • affix (can't stand on its own)

Word formation can develop through:

  • inflection: forms of the same word. E.g. word, words; work, worked
  • derivation: not applicable to all words in a class; the meaning changes: e.g. act -> actor
  • compunds: stardust = star + dust
  • clitisication: word + clitic. E.g.: we're, you're

For counting words:

  • tokens: distinct occurences of word strings
  • types: distinct words

Lemmatization is a vocabulary reduction process of mapping words to their stem. E.g. sang, sung, sings to sing  

Stemming is the process of reducing words to stems. E.g. information to inform, retrieval to retriev

Types of errors:

  • ommision: related words are not reduced to the same stem. E.g. European and Europe
  • commision: unrelated words reduced to the same stems. E.g. policy and police are reduced to polic 

Note: look at Minimum Distance Algorithm to determine the distance between words

It's time to
Read like a Pro.

Jump-start your

reading habits

, gather your

knowledge

,

remember what you read

and stay ahead of the crowd!

Save time with daily digests

No ads, all content is free

Save ideas & add your own

Get access to the mobile app

2M+ Installs

4.7 App Rating